看见并修复缺陷：通过智能体数据合成让视觉语言模型与扩散模型理解视觉伪影
See and Fix the Flaws: Enabling VLMs and Diffusion Models to Comprehend Visual Artifacts via Agentic Data Synthesis
提出在扩散 Transformer 内做 patch 级嵌入操纵来可控注入伪影，并自动生成局部/全局解释，可直接用于构建伪影检测与修复模型的训练数据，也可作为评估生成图像真实性的诊断工具。
原文依据 introduces artifacts via artifact injection tools through novel patch-wise embedding manipulation within a diffusion transformer
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论尽管扩散模型近期取得了进展，AI 生成的图像仍常常包含损害真实感的视觉伪影。虽然更充分的预训练和更大的模型可能减少伪影，但无法保证伪影能被完全消除，这使得伪影缓解成为一个极为关键的研究领域。以往的伪影感知方法依赖人工标注的伪影数据集，这类数据集成本高昂且难以扩展，凸显出对自动化手段可靠获取伪影标注数据集的需求。本文提出 ArtiAgent，它能高效地生成真实图像与注入伪影图像构成的成对数据。该框架包含三个智能体：感知智能体从真实图像中识别并定位实体与子实体；合成智能体通过扩散 Transformer 中新颖的逐块嵌入操纵，借助伪影注入工具引入伪影；策展智能体对合成伪影进行筛选，并为每个实例生成局部与全局解释。利用该流水线，我们收集了 10 万对实例，并在多种应用中验证了其有效性与通用性。代码已发布于 https://github.com/krafton-ai/ArtiAgent。
英文原文摘要 1250 字符
Despite recent advances in diffusion models, AI generated images still often contain visual artifacts that compromise realism. Although more thorough pre-training and bigger models might reduce artifacts, there is no assurance that they can be completely eliminated, which makes artifact mitigation a highly crucial area of study. Previous artifact-aware methodologies depend on human-labeled artifact datasets, which are costly and difficult to scale, underscoring the need for an automated approach to reliably acquire artifact-annotated datasets. In this paper, we propose **ArtiAgent**, which efficiently creates pairs of real and artifact-injected images. It comprises three agents: a perception agent that recognizes and grounds entities and subentities from real images, a synthesis agent that introduces artifacts via artifact injection tools through novel patch-wise embedding manipulation within a diffusion transformer, and a curation agent that filters the synthesized artifacts and generates both local and global explanations for each instance. Using our pipeline, we collect 100K pairwise instances and demonstrate both efficacy and versatility across diverse applications. Code is available at https://github.com/krafton-ai/ArtiAgent.