图像编辑与可控生成 · CVPR2026 ·2026.06.03

解耦再耦合:解决主体驱动文生图生成中的相似性-可控性悖论

Disentangling to Re-couple: Resolving the Similarity-Controllability Paradox in Subject-Driven Text-to-Image Generation

Shuang Li、Chao Deng、Hang Chen、Liqun Liu、Zhenyu Hu、Te Cao 等 11 人

与你有关的部分

把主体身份与修改指令在提示词层面解耦(实体词指代主体、代词指代修改对象),再用专门设计的奖励信号配合强化学习重新耦合,这套「先解耦再耦合」的流程可直接迁移到任何需要同时保真与可控的个性化生成/编辑管线中。

原文依据 we propose DisCo, a novel framework that first Disentangles and then re-Couples visual and textual information
可迁移到 subject-driven text-to-image generationpersonalized image generationinstruction-based image editing

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

主体驱动的文生图(T2I)生成旨在保持主体身份的同时,依据文本提示词编辑其上下文。该任务的核心挑战是「相似性-可控性悖论」:增强文本控制往往会损害主体的保真度,反之亦然。我们认为该悖论源于文本提示词角色模糊——提示词常被要求同时描述主体与期望的修改,从而给模型带来相互冲突的信号。为解决此问题,我们提出 DisCo,一种先解耦、再重新耦合视觉与文本信息的新框架。首先,我们的文本-视觉解耦模块分离信息来源:主体身份仅由参考图像结合主体的实体词提取,而文本提示词被简化为只包含修改指令,其中主体以通用代词指代,从而消除描述歧义。然而,这种严格分离可能导致主体与其上下文之间出现不自然的组合。为此,我们设计了专门的奖励信号,并利用强化学习将视觉定义的主体与文本生成的上下文无缝重新耦合。我们的方法有效解决了该悖论,能够同时实现高保真的主体保持与精确的文本控制。大量实验表明,本方法达到最优性能,可生成高度真实且连贯的图像。

英文原文摘要 1522 字符

Subject-Driven Text-to-Image (T2I) Generation aims to preserve a subject's identity while editing its context based on a text prompt. A core challenge in this task is the "similarity-controllability paradox", where enhancing textual control often degrades the subject's fidelity, and vice-versa. We argue this paradox stems from the ambiguous role of text prompts, which are often tasked with describing both the subject and the desired modifications, leading to conflicting signals for the model. To resolve this, we propose DisCo, a novel framework that first Disentangles and then re-Couples visual and textual information. First, our textual-visual decoupling module isolates the sources of information: subject identity is extracted exclusively from the reference image with the entity word of the subject, while the text prompt is simplified to contain only the modification command, where the subject refers to general pronouns, eliminating descriptive ambiguity. However, this strict separation can lead to unnatural compositions between the subject and its contexts. We address this by designing a dedicated reward signal and using reinforcement learning to seamlessly recouple the visually-defined subject and the textually-generated context. Our approach effectively resolves the paradox, enabling simultaneous high-fidelity subject preservation and precise textual control. Extensive experiments demonstrate that our method achieves state-of-the-art performance, producing highly realistic and coherent images.

标签

会议 CVPR2026
页码 7741-7751
Bibkey
来源 CVF Open Access