生成模型与扩散 · CVPR2026 ·2026.06.03

视觉感知思维链:在统一模型中实现高保真视觉一致性

Visual-Aware CoT: Achieving High-Fidelity Visual Consistency in Unified Models

Zixuan Ye、Quande Liu、Cong Wei、Yuanxing Zhang、Xintao Wang、Pengfei Wan 等 8 人

与你有关的部分

将视觉一致性显式纳入统一模型的推理过程,通过自适应视觉规划生成检查清单,并利用迭代视觉修正与流式GRPO强化学习提升多参考生成中的视觉保真度,为可控生成提供了可复用的自反思框架。

原文依据 we integrate the visual context consistency into the reasoning of unified models
可迁移到 多参考图像生成可控图像生成视觉一致性评估

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

近期,思维链(CoT)的引入大幅提升了统一模型的生成能力。然而,观察到当前生成过程中的思考主要关注与文本提示的文本一致性,而忽略了多模态生成(如多参考生成)过程中与视觉参考图像的视觉上下文一致性。这种一致性的缺失导致无法保持关键视觉特征(如人物身份、物体属性、风格)。为此,我们将视觉上下文一致性融入统一模型的推理过程,通过以下方式明确激励模型保持这种一致性:1)自适应视觉规划:生成结构化视觉检查清单,以确定需要保持一致性的视觉元素;2)迭代视觉修正:在检查清单的指导下进行自反思,并以迭代方式精炼生成结果。为实现这一目标,我们使用监督微调来教导模型如何规划视觉检查、进行自反思与自精炼,并使用流式GRPO通过定制的视觉检查奖励进一步增强视觉一致性。实验表明,我们的方法在多模态生成中优于零样本统一模型以及使用文本思维链的模型,展现出更高的视觉上下文一致性。

英文原文摘要 1392 字符

Recently, the introduction of Chain-of-Thought (CoT) has largely improved generation ability of unified models. However, it is observed that the current thinking process during generation mainly focuses on the text consistency with the text prompt, ignoring the visual context consistency with the visual reference images during the multi-modal generation, e.g., multi-reference generation. The lack of such consistency results in the failure in maintaining key visual features (like human ID, object attribute, style). To this end, we integrate the visual context consistency into the reasoning of unified models, explicitly motivating the model to sustain such consistency by 1) Adaptive Visual Planning: generating structured visual check list to figure out the visual element of needed consistency keeping, and 2) Iterative Visual Correction: performing self-reflection with the guidance of check lists and refining the generated result in an iterative manner. To achieve this, we use supervised finetuning to teach the model how to plan the visual checking, conduct self-reflection and self-refinement, and use flow-GRPO to further enhance the visual consistency through a customized visual checking reward. The experiments show that our method outperforms both zero-shot unified models and those with text CoTs in multi-modal generation, demonstrating higher visual context consistency.

标签

会议 CVPR2026
页码 9116-9126
Bibkey
来源 CVF Open Access