ReasonMap：迈向基于交通地图的细粒度视觉推理
ReasonMap: Towards Fine-Grained Visual Reasoning from Transit Maps
ReasonMap 提供了一个需要细粒度视觉理解与空间推理的基准，可用于评估和诊断 MLLM 在视觉接地上的不足；其视觉掩码分析和强化微调基线为改进多模态模型的视觉推理能力提供了可复用的评测与训练思路。
原文依据 ReasonMap encompasses high-resolution transit maps from 30 cities and includes 1,008 question-answer pairs spanning two question types and three templates.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多模态大语言模型（MLLMs）在语义场景理解和文本-图像对齐方面已取得显著进展，其推理变体进一步提升了在涉及数学和逻辑的复杂任务上的表现。然而，它们在同时需要细粒度视觉理解和空间推理的任务上的能力仍未被充分探索。为弥补这一空白，我们提出了 ReasonMap，一个专门用于评估这些能力的新基准。ReasonMap 涵盖来自 30 个城市的高分辨率交通地图，包含 1,008 个问答对，跨越两种问题类型和三种模板。此外，我们设计了一个两级评估流程，以恰当评估答案的正确性和质量。我们对 16 个流行 MLLM 的全面评估揭示了一个反直觉的模式：在开源模型中，基础变体优于其推理微调版本，而在闭源模型中则观察到相反的趋势。在视觉掩码设置下的进一步分析证实，强大的性能需要直接的视觉接地，而非仅依赖语言先验。我们进一步建立了一个基于强化微调的训练基线，为未来的探索提供参考。我们希望这项基准研究能为视觉推理提供新的见解，并有助于探究开源与闭源模型之间的差距。
英文原文摘要 1414 字符
Multimodal large language models (MLLMs) have demonstrated significant progress in semantic scene understanding and text-image alignment, with reasoning variants enhancing performance on more complex tasks involving mathematics and logic. However, their proficiency in tasks requiring both fine-grained visual understanding and spatial reasoning remains underexplored. To bridge this gap, we introduce ReasonMap, a novel benchmark specifically designed to evaluate these capabilities. ReasonMap encompasses high-resolution transit maps from 30 cities and includes 1,008 question-answer pairs spanning two question types and three templates. Furthermore, we design a two-level evaluation pipeline that properly assesses answer correctness and quality. Our comprehensive evaluation of 16 popular MLLMs reveals a counterintuitive pattern: among open-source models, base variants outperform their reasoning-tuned counterparts, whereas the opposite trend is observed in closed-source models. Further analysis under the visual-masking setting confirms that strong performance necessitates direct visual grounding, rather than relying solely on language priors. We further establish a training baseline with reinforcement fine-tuning, providing a reference for future exploration. We hope this benchmark study offers new insights into visual reasoning and helps investigate the gap between open- and closed-source models.