Re-Align:面向上下文图像生成与编辑的结构化推理引导对齐
Re-Align: Structured Reasoning-guided Alignment for In-Context Image Generation and Editing
将结构化思维链推理与强化学习结合,用于对齐多模态模型的理解与生成能力,可复用于需要精确遵循图文交错指令的图像生成与编辑任务。
原文依据 In-context image generation and editing (ICGE) enables users to specify visual concepts through interleaved image-text prompts
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论上下文图像生成与编辑(ICGE)允许用户通过图文交错的提示指定视觉概念,要求精确理解并忠实执行用户意图。尽管近期统一多模态模型展现出有前景的理解能力,但这些优势往往无法有效迁移到图像生成。我们提出 Re-Align,一个统一框架,通过结构化推理引导的对齐弥合理解与生成之间的差距。其核心是上下文思维链(IC-CoT),一种结构化推理范式,解耦语义引导与参考关联,提供清晰的文本目标并缓解参考图像间的混淆。此外,Re-Align 引入一种有效的强化学习训练方案,利用代理奖励衡量结构化推理文本与生成图像之间的对齐,从而提升模型在 ICGE 任务上的整体性能。大量实验验证,Re-Align 在上下文图像生成与编辑任务上均优于同等模型规模与资源的竞争方法。
英文原文摘要 1155 字符
In-context image generation and editing (ICGE) enables users to specify visual concepts through interleaved image-text prompts, demanding precise understanding and faithful execution of user intent. Although recent unified multimodal models exhibit promising understanding capabilities, these strengths often fail to transfer effectively to image generation. We introduce Re-Align, a unified framework that bridges the gap between understanding and generation through structured reasoning-guided alignment. At its core lies the In-Context Chain-of-Thought (IC-CoT), a structured reasoning paradigm that decouples semantic guidance and reference association, providing clear textual target and mitigating confusion among reference images. Furthermore, Re-Align introduces an effective RL training scheme that leverages a surrogate reward to measure the alignment between structured reasoning text and the generated image, thereby improving the model's overall performance on ICGE tasks. Extensive experiments verify that Re-Align outperforms competitive methods of comparable model scale and resources on both in-context image generation and editing tasks.