魔鬼藏在注意力共享中：通过注意力协同提升复杂非刚性图像编辑的保真度
The Devil is in Attention Sharing: Improving Complex Non-rigid Image Editing Faithfulness via Attention Synergy
针对扩散模型注意力共享中位置嵌入与语义特征失衡导致的注意力坍缩问题，提出按去噪步量化编辑幅度并动态调制位置嵌入影响的注意力协同流程，可直接复用于免训练编辑中保真度与编辑强度的平衡控制。
原文依据 attention collapse in existing attention sharing mechanisms, where either positional embeddings or semantic features dominate visual content retrieval
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论基于大型扩散模型的免训练图像编辑已变得实用，但忠实执行复杂的非刚性编辑（例如姿态或形状变化）仍然极具挑战。我们发现了一个关键的根本原因：现有注意力共享机制中的注意力坍缩，即位置嵌入或语义特征中的一方主导了视觉内容检索，从而导致过度编辑或编辑不足。为解决这一问题，我们提出了 SynPS，一种协同利用位置嵌入与语义信息以实现忠实的非刚性图像编辑的方法。我们首先提出一种编辑度量，用于量化每个去噪步骤所需的编辑幅度。基于该度量，我们设计了一条注意力协同流程，动态调制位置嵌入的影响，使 SynPS 能够平衡语义修改与保真度保持。通过自适应地整合位置与语义线索，SynPS 有效避免了过度编辑与编辑不足。在公开基准以及新构建的基准上的大量实验证明了我们方法优越的性能与保真度。我们的代码将公开发布。
英文原文摘要 1180 字符
Training-free image editing with large diffusion models has become practical, yet faithfully performing complex non-rigid edits (e.g., pose or shape changes) remains highly challenging. We identify a key underlying cause: attention collapse in existing attention sharing mechanisms, where either positional embeddings or semantic features dominate visual content retrieval, leading to over-editing or under-editing.To address this issue, we introduce SynPS, a method that Synergistically leveragesPositional embeddings and Semantic information for faithful non-rigid image editing. We first propose an editing measurement that quantifies the required editing magnitude at each denoising step. Based on this measurement, we design an attention synergy pipeline that dynamically modulates the influence of positional embeddings, enabling SynPS to balance semantic modifications and fidelity preservation.By adaptively integrating positional and semantic cues, SynPS effectively avoids both over- and under-editing. Extensive experiments on public and newly curated benchmarks demonstrate the superior performance and faithfulness of our approach. Our code will be publicly released.