CogniEdit:面向细粒度图像编辑的稠密梯度流优化
CogniEdit: Dense Gradient Flow Optimization for Fine-Grained Image Editing
把 GRPO 的稀疏步级奖励改为跨连续去噪步传播梯度的稠密奖励,为扩散编辑提供轨迹级监督;该思路可直接迁移到任何需要细粒度指令遵循的扩散采样优化中。
原文依据 dense reward optimization that propagates gradients across consecutive denoising steps
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论基于指令的扩散模型图像编辑已取得显著成果,但现有方法难以处理指定颜色、位置和数量等精确属性的细粒度指令。近期方法采用组相对策略优化(GRPO)进行对齐,然而它们仅在单个采样步上进行优化,提供的反馈稀疏,限制了轨迹级控制。我们提出统一框架 CogniEdit,将多模态推理与稠密奖励优化相结合,使梯度在连续去噪步之间传播,从而在采样过程中实现轨迹级梯度流。该方法包含三个部分:(1)多模态大语言模型,用于将复杂指令分解为可执行的指令项;(2)动态词元焦点重定位,自适应地强调细粒度属性;(3)基于 GRPO 的稠密优化,在连续步之间传播梯度以实现轨迹级监督。在基准数据集上的大量实验表明,CogniEdit 在细粒度指令遵循与视觉质量、可编辑性保持之间取得了最优的平衡。代码已开源:https://github.com/yl4467/CogniEdit。
英文原文摘要 1287 字符
Instruction-based image editing with diffusion models has achieved impressive results, yet existing methods struggle with fine-grained instructions specifying precise attributes such as colors, positions, and quantities. While recent approaches employ Group Relative Policy Optimization (GRPO) for alignment, they optimize only at individual sampling steps, providing sparse feedback that limits trajectory-level control. We propose a unified framework **CogniEdit**, combining multi-modal reasoning with dense reward optimization that propagates gradients across consecutive denoising steps, enabling trajectory-level gradient flow through the sampling process. Our method comprises three components: (1) Multi-modal Large Language Models for decomposing complex instructions into actionable directives, (2) Dynamic Token Focus Relocation that adaptively emphasizes fine-grained attributes, and (3) Dense GRPO-based Optimization that propagates gradients across consecutive steps for trajectory-level supervision. Extensive experiments on benchmark datasets demonstrate that our CogniEdit achieves state-of-the-art performance in balancing fine-grained instruction following with visual quality and editability preservation. Our code is available at https://github.com/yl4467/CogniEdit.