EMMA：具备全面语义指标与多样类别的概念擦除基准
EMMA: Concept Erasure Benchmark with Comprehensive Semantic Metrics and Diverse Categories
为概念擦除方法提供了一套覆盖隐式提示、视觉相似非目标概念以及性别/族裔偏见的评测协议，做 T2I 安全生成或模型编辑的研究者可直接用其 13 项指标检验方法是否真正从表征中移除概念，而非仅对直接提示失效。
原文依据 we introduce EMMA, a benchmark that evaluates five key dimensions of concept erasure over 13 metrics
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论文本到图像（T2I）生成的广泛采用引发了对隐私、偏见与版权侵权的担忧。概念擦除技术通过从预训练模型中选择性地移除不期望的概念而无需完整重训练，提供了一种有前景的解决方案。然而，这些方法往往只在有限的概念集合上评估，且依赖过于简单直接的提示。为检验概念擦除技术的边界，并评估其是否真正从模型表征中移除了目标概念，我们提出 EMMA，一个在 13 项指标上评估概念擦除五个关键维度的基准。EMMA 超越了图像质量与时间效率等标准指标，在具有挑战性的条件下测试鲁棒性，包括间接描述、视觉相似的非目标概念以及潜在的性别与族裔偏见，从而对方法行为给出具有社会意识的剖析。利用 EMMA，我们分析了五个领域（物体、名人、艺术风格、NSFW 与版权）中的五种概念擦除方法。结果表明，现有方法在隐式提示（即当被擦除概念被间接指代时仍生成该概念）与视觉相似的非目标概念（即无法生成与被擦除概念相似的非目标概念）上表现不佳，而部分方法相较原始模型放大了性别与族裔偏见。代码与提示词见 https://github.com/lobsterlulu/EMMA。
英文原文摘要 1466 字符
The widespread adoption of text-to-image (T2I) generation has raised concerns about privacy, bias, and copyright violations. Concept erasure techniques offer a promising solution by selectively removing undesired concepts from pre-trained models without requiring full retraining. However, these methods are often evaluated on a limited set of concepts, relying on overly simplistic and direct prompts. To test the boundaries of concept erasure techniques, and assess whether they truly remove targeted concepts from model representations, we introduce EMMA, a benchmark that evaluates five key dimensions of concept erasure over 13 metrics. EMMA goes beyond standard metrics like image quality and time efficiency, testing robustness under challenging conditions, including indirect descriptions, visually similar non-target concepts, and potential gender and ethnicity bias, providing a socially aware analysis of method behavior. Using EMMA, we analyze five concept erasure methods across five domains (objects, celebrities, art styles, NSFW, and copyright). Our results show that existing methods struggle with implicit prompts (i.e., generating the erased concept when it is indirectly referenced) and visually similar non-target concepts (i.e., failing to generate non-target concepts resembling the erased one), while some amplify gender and ethnicity bias compared to the original model. Code and prompts are available at https://github.com/lobsterlulu/EMMA.