CRIT：基于图的自动数据合成以增强跨模态多跳推理
CRIT: Graph-Based Automatic Data Synthesis to Enhance Cross-Modal Multi-Hop Reasoning
提供了一套基于图自动生成跨模态多跳推理训练/评测数据的流水线，可直接用于构造需要图文互补才能作答的 VLM 训练数据，缓解模型推理链缺乏视觉依据的幻觉问题。
原文依据 we introduce CRIT, a new dataset and benchmark built with a graph-based automatic pipeline for generating complex cross-modal reasoning tasks
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论现实世界的推理往往需要跨模态整合信息，在多跳过程中将文本上下文与视觉线索联系起来。然而，大多数多模态基准未能刻画这一能力：它们通常依赖单张图像或一组图像，答案仅凭单一模态即可推断。这一局限同样反映在训练数据中，交错图文内容很少强制要求互补性的多跳推理。因此，视觉语言模型（VLM）经常产生幻觉，并给出缺乏视觉证据支撑的推理轨迹。为弥补这一空白，我们提出 CRIT，这是一个新的数据集与基准，通过基于图的自动流水线构建，用于生成复杂的跨模态推理任务。CRIT 涵盖从自然图像、视频到富文本来源的多样领域，并包含一个经人工校验的测试集以支持可靠评测。在该基准上的实验表明，即使最优模型在这类推理任务上也表现吃力。在 CRIT 上训练的模型在跨模态多跳推理上取得显著提升，包括在 SPIQA 及其他标准多模态基准上的大幅改进。
英文原文摘要 1183 字符
Real-world reasoning often requires combining information across modalities, connecting textual context with visual cues in a multi-hop process. Yet, most multimodal benchmarks fail to capture this ability: they typically rely on single images or set of images, where answers can be inferred from a single modality alone. This limitation is mirrored in the training data, where interleaved image-text content rarely enforces complementary, multi-hop reasoning. As a result, Vision-Language Models (VLMs) frequently hallucinate and produce reasoning traces poorly grounded in visual evidence. To address this gap, we introduce CRIT, a new dataset and benchmark built with a graph-based automatic pipeline for generating complex cross-modal reasoning tasks. CRIT consists of diverse domains ranging from natural images, videos, and text-rich sources, and includes a manually verified test set for reliable evaluation. Experiments on this benchmark reveal that even state-of-the-art models struggle on such reasoning tasks. Models trained on CRIT show significant gains in cross-modal multi-hop reasoning, including strong improvements on SPIQA and other standard multimodal benchmarks.