通过外部推理框架突破多模态大语言模型的区域感知瓶颈
Breaking the Regional Perception Bottleneck of Multimodal Large Language Models via External Reasoning Framework
把蒙特卡洛树搜索这类推理时搜索引入 MLLM 的像素级视觉定位，用多模态特征对齐打分和区域特征加权投票替代回归解码器，为纯 MLLM 做 grounding 提供了一条不增加训练成本、只扩展推理阶段的可复用路线。
原文依据 a reasoning framework for MLLM-based grounding built upon a multimodal Monte Carlo Tree Search algorithm
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论高质量的像素级响应仍是多模态大语言模型（MLLM）在区域感知方面的主要瓶颈。现有方法通常将回归解码器附加到 MLLM 特征上，虽然取得了较强的视觉定位性能，但破坏了端到端设计并增加了训练成本。研究者们采用参数与数据扩展来提升纯 MLLM 以自然语言生成像素坐标的能力，然而在视觉定位任务上的性能提升仍明显弱于标准问答任务。我们的分析表明，主要瓶颈在于常规扩展无法有效增强像素级区域感知所需的关键推理阶段。为此，我们提出 R-Ground，一个基于多模态蒙特卡洛树搜索算法的 MLLM 视觉定位推理框架。R-Ground 利用结构化推理动作、多模态特征对齐评分和区域特征加权投票，在指定的推理阶段进行扩展。大量实验表明，R-Ground 实现了有效的推理扩展，使 7B 的 MLLM 在视觉定位任务上达到甚至超越 72B 模型。
英文原文摘要 1201 字符
High-quality pixel-level responses remain a major bottleneck for multimodal large language models (MLLMs) in regional perception. Existing approaches generally attach regression decoders to MLLM features, achieving strong grounding performance but compromising end-to-end design and increasing training costs. Researchers have applied parameter and data scaling to improve pure MLLMs' ability to generate pixel coordinates in natural language, yet the performance gains on grounding tasks remain markedly weaker than those in standard QA tasks. Our analysis shows the primary bottleneck is that conventional scaling fails to effectively enhance the key reasoning stage required for pixel-level regional perception. To address this, we propose R-Ground, a reasoning framework for MLLM-based grounding built upon a multimodal Monte Carlo Tree Search algorithm. R-Ground leverages structured reasoning actions, multimodal feature alignment scoring, and regional feature weighted voting to perform scaling at the designated reasoning stage. Extensive experiments demonstrate that R-Ground achieves effective reasoning scaling, enabling a 7B MLLM to match or even surpass a 72B model on the grounding task.