当可视化成为推理的第一步：MIRA，一个面向视觉思维链的基准
When Visualizing is the First Step to Reasoning: MIRA, a Benchmark for Visual Chain-of-Thought
该基准把「生成中间视觉草图/路径图」作为推理的必要环节，为视觉研究者提供了检验 MLLM 是否真正利用视觉中间表征的评测协议（三级输入：直接输入、Text-CoT、Visual-CoT），可用于设计需要视觉想象的推理模型与训练数据。
原文依据 tasks in MIRA require models to generate and utilize intermediate images --- such as sketches, structural diagrams, or path drawings --- to guide their reasoning process
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论我们提出 MIRA（Multimodal Imagination for Reasoning Assessment，多模态想象推理评估），这是一个新的基准，用于评估模型在必须生成中间视觉图像才能成功推理的场景下的表现。与仅依赖文本的传统思维链（CoT）方法不同，MIRA 中的任务要求模型生成并利用中间图像——例如草图、结构示意图或路径图——来引导其推理过程。这一设定与人类通过「画图来思考」解决复杂问题的方式高度一致。为此，MIRA 聚焦于本身具有挑战性、且涉及复杂结构、空间关系或难以仅用语言表达的推理步骤的任务（例如跟踪骰子在棋盘上的移动，并在每次投掷后对朝下一面的点数求和）。为确保评测数据的高质量，我们收录了 546 个多模态问题，并标注了中间视觉图像与最终答案。我们还为 MIRA 提出了一个统一的评测协议，涵盖三个层次的评测输入：仅输入图像与问题的直接输入、输入图像与思考提示的纯文本思维链（Text-CoT）输入，以及同时输入标注图像线索与文本思考提示的 Visual-CoT 输入。为探测模型在该基准上的能力上限，我们还报告了不同 k 设置下的 pass@k 与多数投票准确率。实验结果表明，现有多模态大语言模型（MLLM），包括最强的闭源模型（如 GPT-5、o3、Gemini 2.5 Pro）以及强开源权重模型（如 Qwen2.5-VL、GLM 4.5V），在仅依赖文本提示时表现不佳。然而，当提供中间视觉线索时，模型性能持续提升，在所有模型与任务上平均相对增益达 33.7%。我们还通过扩大搜索空间以及设计与 Visual-CoT 对齐的文本提示来探测上限，但与我们的 Visual-CoT 设定相比，二者仅带来有限提升。这些结果凸显了想象出的视觉信息在 MIRA 上实现成功推理的关键作用。
英文原文摘要 2167 字符
We propose MIRA (Multimodal Imagination for Reasoning Assessment), a new benchmark designed to evaluate models in scenarios where generating intermediate visual images is essential for successful reasoning. Unlike traditional Chain-of-thought (CoT) methods that rely solely on text, tasks in MIRA require models to generate and utilize intermediate images --- such as sketches, structural diagrams, or path drawings --- to guide their reasoning process. This setup closely mirrors how humans solve complex problems through "drawing to think". To solve this, MIRA focuses on tasks that are intrinsically challenging and involve complex structures, spatial relationships, or reasoning steps that are difficult to express through language alone (e.g., tracking a die's movement on a board and summing the face-down values after each roll). To ensure that our evaluation data is of high-quality, we include 546 multimodal problems, annotated with intermediate visual images and final answers. We also propose a unified evaluation protocol for MIRA that spans three levels of evaluation input: direct input with image and question only, text-only CoT (Text-CoT) input with image and thinking prompts, and Visual-CoT input with both annotated image clues and textual thinking prompts. To probe the upper bound of model capacity on our benchmark, we also report pass@k and majority voting accuracies under different k settings. Experimental results show that existing multimodal large language models (MLLMs), including strongest private models (e.g., GPT-5, o3, Gemini 2.5 Pro) as well as strong open-weight models (e.g., Qwen2.5-VL, GLM 4.5V), perform poorly when relying solely on textual prompts. However, when intermediate visual cues are provided, model performance improves consistently, yielding an average relative gain of 33.7% across all models and tasks. We also probe the upper bound by expanding the search space and designing textual prompts aligned with Visual-CoT, but both yield only limited improvements compared to our Visual-CoT setting. These results underscore the critical role of imagined visual information in enabling successful reasoning on MIRA.