文本到图像扩散模型中的邻域感知局部概念擦除
Neighbor-Aware Localized Concept Erasure in Text-to-Image Diffusion Models
提供一种免训练的三阶段概念擦除流程:频谱加权嵌入调制 + 注意力引导空间门控 + 空间门控硬擦除,可直接用于扩散模型的安全对齐与细粒度类别保真,避免擦除时误伤语义近邻概念。
原文依据 a spectrally-weighted embedding modulation that attenuates target concept directions while stabilizing neighbor concept representations
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论文本到图像扩散模型中的概念擦除旨在移除不期望的概念,同时保持整体生成能力。局部擦除方法试图将编辑限制在目标概念所占的空间区域内。然而,我们观察到抑制某一概念可能会无意中削弱语义相关的邻域概念,从而降低细粒度领域中的保真度。我们提出邻域感知局部概念擦除(NLCE),这是一个免训练框架,旨在移除目标概念的同时更好地保留邻域概念。它分三个阶段运行:(1)频谱加权嵌入调制,在衰减目标概念方向的同时稳定邻域概念表征;(2)注意力引导的空间门控,用于识别表现出残余概念激活的区域;(3)空间门控硬擦除,仅在必要之处消除残留痕迹。这一邻域感知流程能够在保持周围概念邻域结构的同时实现局部概念移除。在细粒度数据集(Oxford Flowers、Stanford Dogs)上的实验表明,我们的方法能够有效移除目标概念,同时更好地保留密切相关的类别。在名人身份、显式内容和艺术风格上的额外结果证明了该方法对更广泛擦除场景的鲁棒性和泛化能力。
英文原文摘要 1380 字符
Concept erasure in text-to-image diffusion models seeks to remove undesired concepts while preserving overall generative capability. Localized erasure methods aim to restrict edits to the spatial region occupied by the target concept. However, we observe that suppressing a concept can unintentionally weaken semantically related neighbor concepts, reducing fidelity in fine-grained domains. We propose Neighbor-Aware Localized Concept Erasure (NLCE), a training-free framework designed to better preserve neighboring concepts while removing target concepts. It operates in three stages: (1) a spectrally-weighted embedding modulation that attenuates target concept directions while stabilizing neighbor concept representations, (2) an attention-guided spatial gate that identifies regions exhibiting residual concept activation, and (3) a spatially-gated hard erasure that eliminates remaining traces only where necessary. This neighbor-aware pipeline enables localized concept removal while maintaining the surrounding concept neighborhood structure. Experiments on fine-grained datasets (Oxford Flowers, Stanford Dogs) show that our method effectively removes target concepts while better preserving closely related categories. Additional results on celebrity identity, explicit content and artistic style demonstrate robustness and generalization to broader erasure scenarios.