DiGraphHal-Bench：评估多模态大语言模型在复杂有向图上的表现
DiGraphHal-Bench: Evaluating Multimodal Large Language Models on Complex Directed Graphs
该论文提出的有向图VQA基准和两阶段自动数据整理流程，可用于评估和提升多模态大模型在结构化视觉信息（如图表、流程图）上的细粒度推理与幻觉抑制能力，为视觉-语言模型在复杂图结构理解方向提供可复用的评测方法和数据构建思路。
原文依据 we present DiGraphHal-Bench, the first large-scale Visual Question Answering (VQA) benchmark for evaluating both hallucination phenomena and fine-grained reasoning of MLLMs on real-world directed graphs.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论以往关于多模态大语言模型（MLLM）幻觉的研究主要考察自然图像中的跨模态不一致性，而复杂图结构上的幻觉仍未得到充分探索。同时，缺乏对融合结构、视觉和语义信息的细粒度推理的稳健评估。为填补这些空白，我们提出了 DiGraphHal-Bench，这是首个大规模视觉问答（VQA）基准，用于评估 MLLM 在真实世界有向图上的幻觉现象和细粒度推理。DiGraphHal-Bench 包含来自六个以上不同领域的高质量程序化图，并围绕四种高级能力和十二项细粒度任务的分类体系进行组织。为确保基准保真度，我们提出了一种新颖的两阶段自动数据整理流程，权衡数据规模与质量，从而保证可靠评估。实验表明，最优的 MLLM 在细粒度图推理中明显出现幻觉。尽管监督微调（SFT）大幅缓解了这些幻觉并增强了复杂推理，但性能仍远未达到最优。消融研究强调了基础能力对整合推理的重要性，我们的基准为推进稳健的多模态图理解奠定了基础。
英文原文摘要 1379 字符
While prior research on Multimodal Large Language Model (MLLM) hallucinations has primarily examined cross-modal inconsistencies in natural images, hallucination over complex graph structures remains underexplored.Concurrently, there is a lack of robust evaluation for fine-grained reasoning integrating structural, visual, and semantic information.To address these gaps, we present DiGraphHal-Bench, the first large-scale Visual Question Answering (VQA) benchmark for evaluating both hallucination phenomena and fine-grained reasoning of MLLMs on real-world directed graphs. DiGraphHal-Bench comprises high-quality procedural graphs from over six distinct domains and is organized around a taxonomy of four high-level capabilities and twelve fine-grained tasks. To ensure benchmark fidelity, we propose a novel two-stage automatic data curation pipeline that reconciles the trade-off between data scale and quality, thereby guaranteeing reliable evaluation.Experiments reveal that state-of-the-art MLLMs hallucinate notably in fine-grained graph reasoning. Although SFT substantially mitigates these hallucinations and strengthens complex reasoning, performance remains far from optimal. Ablation studies highlight the importance of fundamental capabilities for integrative reasoning, and our benchmark provides a foundation for advancing robust multi-modal graph understanding.