Meta-CoT:提升图像编辑的粒度与泛化能力
Meta-CoT: Enhancing Granularity and Generalization in Image Editing
Meta-CoT 将任意图像编辑操作分解为(任务、目标、所需理解能力)三元组,并进一步拆解为五个元任务,配合思维链一致性奖励,使模型在少量元任务训练后即可泛化到未见编辑任务;该分解思路可直接用于构建更细粒度的编辑指令数据与训练策略。
原文依据 we propose Meta-CoT, a paradigm that performs a two-level decomposition of any single-image editing operation
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论统一的多模态理解/生成模型通过将细粒度理解融入其思维链(CoT)过程,已展现出改进的图像编辑性能。然而,一个关键问题仍未得到充分探索:何种形式的思维链与训练策略能够同时提升理解粒度和泛化能力?为此,我们提出 Meta-CoT,一种对任意单图像编辑操作进行两级分解的范式,具有两个关键特性:(1)可分解性。我们观察到任何编辑意图都可以表示为一个三元组——(任务、目标、所需理解能力)。受此启发,Meta-CoT 同时对编辑任务和目标进行分解,生成任务特定的思维链,并在所有目标上遍历编辑操作。这种分解增强了模型对编辑操作的理解粒度,并引导其在训练过程中学习三元组的每个元素,从而大幅提升编辑能力。(2)泛化性。在第二级分解中,我们进一步将编辑任务分解为五个基本元任务。我们发现,在这五个元任务以及三元组的另外两个元素上进行训练,足以在多样化的、未见过的编辑任务上实现强泛化。为了进一步使模型的编辑行为与其思维链推理对齐,我们引入了思维链-编辑一致性奖励,鼓励在编辑过程中更准确、有效地利用思维链信息。实验表明,我们的方法在 21 个编辑任务上实现了整体 15.8% 的提升,并且在仅使用少量元任务训练时能有效泛化到未见过的编辑任务。我们的代码、基准和模型将公开发布。
英文原文摘要 1787 字符
Unified multi-modal understanding/generative models have shown improved image editing performance by incorporating fine-grained understanding into their Chain-of-Thought (CoT) process. However, a critical question remains underexplored: what forms of CoT and training strategy can jointly enhance both the understanding granularity and generalization? To address this, we propose Meta-CoT, a paradigm that performs a two-level decomposition of any single-image editing operation with two key properties: (1) Decomposability. We observe that any editing intention can be represented as a triplet -- (task, target, required understanding ability). Inspired by this, Meta-CoT decomposes both the editing task and the target, generating task-specific CoT and traversing editing operations on all targets. This decomposition enhances the model's understanding granularity of editing operations and guides it to learn each element of the triplet during training, substantially improving the editing capability.(2) Generalizability. In the second decomposition level, we further break down editing tasks into five fundamental meta-tasks. We find that training on these five meta-tasks, together with the other two elements of the triplet, is sufficient to achieve strong generalization across diverse, unseen editing tasks.To further align the model's editing behavior with its CoT reasoning, we introduce the CoT-Editing Consistency Reward, which encourages more accurate and effective utilization of CoT information during editing. Experiments demonstrate that our method achieves an overall 15.8% improvement across 21 editing tasks, and generalizes effectively to unseen editing tasks when trained on only a small set of meta-tasks. Our code, benchmark, and model will be released publicly.