一致性评论器:通过参考引导的注意力对齐纠正生成图像中的不一致
The Consistency Critic: Correcting Inconsistencies in Generated Images via Reference-Guided Attentive Alignment
把参考图引导的后编辑作为修正生成不一致的手段,并给出基于注意力对齐损失与细节编码器的可复用修正模块,可嵌入智能体做多轮局部编辑,为定制化生成的一致性修复提供思路。
原文依据 we accordingly devise an attention alignment loss and a detail encoder to precisely rectify inconsistencies
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论以往工作已探索了给定参考图像的各种定制化生成任务,但在生成一致的细粒度细节方面仍存在局限。本文旨在通过引入参考引导的后编辑方法来解决生成图像的不一致问题,并提出 ImageCritic。我们首先构建了一个参考-退化-目标三元组数据集,该数据集通过基于视觉语言模型的选择与显式退化获得,能够有效模拟现有生成模型中常见的偏差或不一致。此外,在深入考察模型的注意力机制与内在表征的基础上,我们相应地设计了注意力对齐损失与细节编码器,以精确纠正不一致。ImageCritic 可集成到智能体框架中,在复杂场景下自动检测不一致并通过多轮局部编辑加以纠正。大量实验表明,ImageCritic 能有效解决多种定制化生成场景中的细节相关问题,相较现有方法取得显著提升。
英文原文摘要 1159 字符
Previous works have explored various customized generation tasks given a reference image, but they still face limitations in generating consistent fine-grained details. In this paper, our aim is to solve the inconsistency problem of generated images by applying a reference-guided post-editing approach and present our ImageCritic. We first construct a dataset of reference-degraded-target triplets obtained via VLM-based selection and explicit degradation, which effectively simulates the common inaccuracies or inconsistencies observed in existing generation models. Furthermore, building on a thorough examination of the model's attention mechanisms and intrinsic representations, we accordingly devise an attention alignment loss and a detail encoder to precisely rectify inconsistencies. ImageCritic can be integrated into an agent framework to automatically detect inconsistencies and correct them with multi-round and local editing in complex scenarios. Extensive experiments demonstrate that ImageCritic can effectively resolve detail-related issues in various customized generation scenarios, providing significant improvements over existing methods.