从室内到开放世界：揭示多模态大模型中的空间推理鸿沟
From Indoor to Open World: Revealing the Spatial Reasoning Gap in MLLMs
对做多模态大模型空间推理的人，这篇提供了用双目+LiDAR+IMU/GPS 采集的行人视角视频自动生成分层空间推理问答的流程，以及合成异常场景与盲测两种诊断协议，可直接用来检验模型是否真的依赖视觉 grounding 而非语言先验。
原文依据 we introduce a large-scale benchmark built from pedestrian-perspective videos captured with stereo cameras, LiDAR, and IMU/GPS sensors
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论尽管多模态大模型（MLLMs）在语义任务上取得了令人瞩目的表现，但其空间智能——对于构建稳健且具有落地能力的 AI 系统至关重要——仍发展不足。现有基准难以诊断这一局限：它们要么聚焦于过度简化的定性推理，要么依赖特定领域的室内数据，其根源在于缺乏带有可验证度量真值的室外数据集。为弥补这一空白，我们提出了一个大规模基准，其数据来自使用双目相机、LiDAR 与 IMU/GPS 传感器采集的行人视角视频。该数据集提供度量精确的三维信息，使得自动生成空间推理问题成为可能，这些问题覆盖从定性关系推理到定量度量与运动学理解的分层谱系。评测显示，在结构化室内基准上观察到的性能提升在开放世界场景中消失。进一步利用合成异常场景与盲测开展的分析证实，当前 MLLMs 严重依赖语言先验，而非基于视觉的落地推理。因此，我们的基准为诊断这些局限并推进物理落地的空间智能提供了一个有原则的平台。项目主页：https://mingrui-wu.github.io/osi-bench/
英文原文摘要 1326 字符
While Multimodal Large Language Models (MLLMs) have achieved impressive performance on semantic tasks, their spatial intelligence--crucial for robust and grounded AI systems--remains underdeveloped. Existing benchmarks fall short of diagnosing this limitation: they either focus on overly simplified qualitative reasoning or rely on domain-specific indoor data, constrained by the lack of outdoor datasets with verifiable metric ground truth. To bridge this gap, we introduce a large-scale benchmark built from pedestrian-perspective videos captured with stereo cameras, LiDAR, and IMU/GPS sensors. This dataset provides metrically precise 3D information, enabling the automatic generation of spatial reasoning questions that span a hierarchical spectrum--from qualitative relational reasoning to quantitative metric and kinematic understanding. Evaluations reveal that the performance gains observed in structured indoor benchmarks vanish in open-world settings. Further analysis using synthetic abnormal scenes and blinding tests confirms that current MLLMs depend heavily on linguistic priors instead of grounded visual reasoning. Our benchmark thus provides a principled platform for diagnosing these limitations and advancing physically grounded spatial intelligence. Project page: https://mingrui-wu.github.io/osi-bench/