Multi-SpatialMLLM：面向多模态大语言模型的多帧空间理解
Multi-SpatialMLLM: Multi-Frame Spatial Understanding with Multi-Modal Large Language Models
把深度、跨帧对应、动态感知等基础空间技能统一进 MLLM 训练，并用 2700 万级 3D/4D 数据管线与统一指标基准评测多帧空间推理，可直接复用于机器人多帧奖励标注与具身感知。
原文依据 equip MLLMs with multi-frame spatial understanding by integrating fundamental spatial skills, including depth perception, visual correspondence, and dynamic perception
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多模态大语言模型（MLLM）在视觉任务上进展迅速，但其空间理解仍局限于单张图像，难以适用于需要多帧推理的物理世界应用。本文提出一个框架，通过整合深度感知、视觉对应与动态感知等基础空间技能，使 MLLM 具备多帧空间理解能力。我们设计了一套新的数据管线，并收集了涵盖多样 3D 与 4D 场景、超过 2700 万样本的 MultiSPA 数据集以支持训练。除 MultiSPA 之外，我们还提出一个全面的基准，在统一指标下测试广泛的空间任务。由此得到的模型 Multi-SpatialMLLM 相较基线方法与专有系统取得显著提升，展现出可扩展且可泛化的多帧感知能力。我们进一步观察到多任务收益以及在挑战性场景中涌现的空间能力，并展示了该模型如何作为机器人领域的多帧奖励标注器发挥作用。
英文原文摘要 1081 字符
Multi-modal large language models (MLLMs) have rapidly advanced in visual tasks, yet their spatial understanding remains limited to single images, leaving them ill-suited for physical-world applications that require multi-frame reasoning. In this paper, we propose a framework to equip MLLMs with multi-frame spatial understanding by integrating fundamental spatial skills, including depth perception, visual correspondence, and dynamic perception. We design a novel data pipeline and collect the MultiSPA dataset of more than 27 million samples spanning diverse 3D and 4D scenes to enable training. Alongside MultiSPA, we introduce a comprehensive benchmark that tests a wide spectrum of spatial tasks under uniform metrics. Our resulting model, Multi-SpatialMLLM, achieves significant gains over baselines and proprietary systems, demonstrating scalable and generalizable multi-frame perception. We further observe multi-task benefits and emergent spatial capabilities in challenging scenarios, and showcase how our model can serve as a multi-frame reward annotator for robotics.