多模态模型会被多图视觉谜题难住吗？
Will Multimodal Models Be Dazzled by Multi-Image Visual Puzzles?
对做多图/多帧视觉语言推理的人，MIRACLE 提供了强制跨图依赖的评测协议与实例分组设计，可直接用来诊断 MLLM 在高视觉信息密度（谜题、超多图输入）下的结构性关系推理短板，并作为多图训练数据的构造参考。
原文依据 even top-performing models like Gemini-2.5-Pro achieve only 55.91% points
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多模态大语言模型（MLLMs）的快速发展，暴露出已有基准在评估多图复杂推理方面的局限。为填补这一空白，我们提出 MIRACLE，一个用于多图复杂推理与理解逻辑评估的新基准，包含 4000 道问题，覆盖视觉比较、时序排序、空间关系等多种推理类型。MIRACLE 通过系统化的数据收集流程强调图像间的强依赖关系，并辅以精细的实例分组与问题设计，从而强制进行跨图推理。对领先 MLLM 的评估显示，即便是 Gemini-2.5-Pro 这样的顶尖模型也仅取得 55.91% 的分数，凸显了多图推理的重大挑战。此外，在以高视觉信息密度为特征的场景中，例如谜题任务和超多图输入条件，所有模型都出现显著性能下降，这凸显了 MLLM 在处理复杂结构关系与协同推理方面的局限，揭示了其在高负载视觉推理设定下认知能力的不足。我们希望 MIRACLE 能激励社区推动多图推理的边界。MIRACLE 已发布于 https://huggingface.co/datasets/queyuecanyang/MIRACLE。
英文原文摘要 1370 字符
The rapid advancement of Multimodal Large Language Models (MLLMs) has revealed the limitations of existing benchmarks in evaluating complex reasoning over multiple images. To address this gap, we introduce MIRACLE, a novel benchmark for Multi-Image complex Reasoning And Comprehension Logic Evaluation, featuring 4,000 questions across diverse reasoning types such as visual comparison, temporal sequencing, and spatial relations. MIRACLE emphasizes strong inter-image dependencies through a systematic data collection process, followed by delicate instance grouping and question design that enforce cross-image reasoning.Evaluation on leading MLLMs shows that even top-performing models like Gemini-2.5-Pro achieve only 55.91% points, highlighting the significant challenges of multi-image reasoning. Moreover, in scenarios characterized by high visual information density, such as puzzle tasks and ultra multi-image input conditions, all models exhibit a significant drop in performance, which highlights the limitations of MLLMs in handling complex structural relations and collaborative reasoning, revealing deficiencies in their cognitive capabilities under high-load visual reasoning settings. We hope MIRACLE will inspire the community to push the boundaries of multi-image reasoning. MIRACLE is available at https://huggingface.co/datasets/queyuecanyang/MIRACLE.