GenBreak：利用大语言模型对文本到图像生成进行红队测试
GenBreak: Red Teaming Text-to-Image Generation Using Large Language Models
对做 T2I 安全评测的人，这篇给出了一个可复用的红队提示词生成范式：用监督微调加多奖励强化学习训练一个 LLM 攻击者，同时优化提示词的隐蔽性与生成图像的毒性，可直接用于黑盒评测商用 T2I 生成器的安全对齐强度。
原文依据 fine-tunes a red-team large language model (LLM) to systematically explore underlying vulnerabilities in T2I generators
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论Stable Diffusion 等文本到图像（T2I）模型发展迅速，已广泛用于内容创作。然而，这些模型可能被滥用来生成裸体或暴力等有害内容，带来显著的安全风险。尽管多数平台部署了内容审核系统，其底层漏洞仍可能被攻击者利用。近期针对 T2I 模型的红队测试与对抗攻击研究存在一个关键局限：现有方法难以在提示词隐蔽性与图像毒性之间取得平衡。部分研究虽能成功生成高毒性图像，但所用对抗提示词容易被安全过滤器检测到；另一些研究专注于绕过安全机制，却无法产生真正有害的输出，忽视了对真正高风险提示词的发现。因此，目前仍缺乏可靠工具来评估经过防御的 T2I 模型的安全性。为填补这一空白，我们提出 GenBreak，一个微调红队大语言模型（LLM）以系统性地探索 T2I 生成器潜在漏洞的框架。我们的方法将基于精选数据集的监督微调与通过和替代 T2I 模型交互的强化学习相结合。通过整合多种奖励信号，我们引导 LLM 构造对抗提示词，在保持语义连贯性与多样性的同时，提升规避能力与图像毒性。这些提示词在对商用 T2I 生成器的黑盒攻击中表现出很强的有效性，揭示了实际存在的安全弱点。代码见 https://github.com/wangdandan567/RT-diffuser。
英文原文摘要 1647 字符
Text-to-image (T2I) models such as Stable Diffusion have advanced rapidly and are widely used in content creation. However, these models can be misused to generate harmful content, including nudity or violence, posing significant safety risks. While most platforms employ content moderation systems, underlying vulnerabilities can still be exploited by adversaries. Recent research on red-teaming and adversarial attacks against T2I models faces a critical limitation: existing methods struggle to balance prompt stealthiness with image toxicity. Some studies successfully generate highly toxic images but use adversarial prompts that are easily detected by safety filters, while others focus on bypassing safety mechanisms but fail to produce genuinely harmful outputs, neglecting the discovery of truly high-risk prompts. Consequently, there remains a lack of reliable tools for evaluating the safety of defended T2I models. To address this gap, we propose GenBreak, a framework that fine-tunes a red-team large language model (LLM) to systematically explore underlying vulnerabilities in T2I generators. Our approach combines supervised fine-tuning on curated datasets with reinforcement learning via interaction with a surrogate T2I model. By integrating multiple reward signals, we guide the LLM to craft adversarial prompts that enhance both evasion capability and image toxicity, while maintaining semantic coherence and diversity. These prompts demonstrate strong effectiveness in black-box attacks against commercial T2I generators, revealing practical safety weaknesses. Code is available at https://github.com/wangdandan567/RT-diffuser.