ZINA：多模态细粒度幻觉检测与编辑
ZINA: Multimodal Fine-grained Hallucination Detection and Editing
为视觉语言模型输出提供细粒度幻觉片段定位与错误类型分类（六类），并给出修正建议，可直接用于VLM生成结果的自动核查与后处理编辑；其基于图结构建模错误类型依赖的合成数据生成方式，也可迁移到其他需要细粒度标注稀缺的多模态任务。
原文依据 we propose ZINA, a novel method that identifies hallucinated spans at a fine-grained level, classifies their error types into six categories
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多模态大语言模型（MLLMs）经常产生幻觉，即输出偏离视觉内容。鉴于这些幻觉可能呈现多种形式，在细粒度层面检测幻觉对于全面评估与分析至关重要。为此，我们提出了一项新任务：面向 MLLMs 的多模态细粒度幻觉检测与编辑。此外，我们提出了 ZINA，一种新方法，能够在细粒度层面识别幻觉片段、将其错误类型划分为六个类别，并给出恰当的修正建议。为训练和评估该任务下的模型，我们构建了 VisionHall 数据集，其中包含由 211 名标注人员人工标注的、来自十二个 MLLMs 的 6.9k 条输出，以及使用一种捕捉错误类型间依赖关系的图方法生成的 20k 条合成样本。我们证明，ZINA 在检测与编辑两项任务上均优于现有方法，包括 GPT-4o 和 Llama-3.2。
英文原文摘要 980 字符
Multimodal Large Language Models (MLLMs) often generate hallucinations, where the output deviates from the visual content. Given that these hallucinations can take diverse forms, detecting hallucinations at a fine-grained level is essential for comprehensive evaluation and analysis. To this end, we propose a novel task of multimodal fine-grained hallucination detection and editing for MLLMs. Moreover, we propose ZINA, a novel method that identifies hallucinated spans at a fine-grained level, classifies their error types into six categories, and suggests appropriate refinements. To train and evaluate models for this task, we constructed VisionHall, a dataset comprising 6.9k outputs from twelve MLLMs manually annotated by 211 annotators, and 20k synthetic samples generated using a graph-based method that captures dependencies among error types. We demonstrated that ZINA outperformed existing methods, including GPT-4o and Llama-3.2, in both detection and editing tasks.