Think 360°:超越深度:评估多模态大模型的宽度中心推理能力
Think 360deg: Beyond Depth: Evaluating the Width-centric Reasoning Capability of MLLMs
该基准提出的细粒度思维树评测协议可用于诊断视觉语言模型在需要广泛搜索与多约束优化的视觉推理任务上的不足,为设计支持宽推理的视觉模型提供评测工具。
原文依据 we present a holistic multimodal benchmark that evaluates the reasoning capabilities of MLLMs with an explicit focus on reasoning width
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论本文提出了一个整体性的多模态基准,用于评估多模态大模型的推理能力,并明确聚焦于推理宽度——这是对更常被研究的推理深度的一个补充维度。具体而言,推理深度衡量模型执行长链、顺序推理的能力,其中每一步都紧密且严格地与下一步相连。推理宽度则更关注模型进行广泛试错搜索或多约束优化的能力:它必须系统地遍历许多可能的并行推理路径,应用多种约束来剪除无望的分支,并识别有效的解决路径以进行高效迭代或回溯。为实现这一目标,我们精心整理了1200多个高质量多模态案例,涵盖异构领域,并提出了一种细粒度的思维树评测协议,该协议联合量化推理宽度和深度。我们在难度层级、问题类型和所需技能上评估了12个主要模型家族(超过30个先进的多模态大模型)。结果表明,尽管当前模型在通用或常识视觉问答任务上表现出强劲性能,但它们仍难以将深度顺序思维链与广泛探索性搜索相结合,以执行真正的基于洞察的推理。最后,我们分析了特征性失败模式,为构建不仅推理更深而且更宽的多模态大模型提供了可能的方向。
英文原文摘要 1456 字符
In this paper, we present a holistic multimodal benchmark that evaluates the reasoning capabilities of MLLMs with an explicit focus on reasoning width, a complementary dimension to the more commonly studied reasoning depth. Specifically, reasoning depth measures the model's ability to carry out long-chain, sequential reasoning in which each step is tightly and rigorously linked to the next. Reasoning width tends to focus more on the model's capacity for broad trial-and-error search or multi-constrained optimization: it must systematically traverse many possible and parallelized reasoning paths, apply diverse constraints to prune unpromising branches, and identify valid solution routes for efficient iteration or backtracking. To achieve it, we carefully curate 1200+ high-quality multimodal cases spanning heterogeneous domains, and propose a fine-grained tree-of-thought evaluation protocol that jointly quantifies reasoning width and depth. We evaluate 12 major model families (over 30 advanced MLLMs) across difficulty tiers, question types, and required skills. Results show that while current models exhibit strong performance on general or common-sense VQA tasks, they still struggle to combine deep sequential thought chains with wide exploratory search to perform genuine insight-based reasoning. Finally, we analyze characteristic failure modes to provide possible directions for building MLLMs that reason not only deeper but also wider.