HiSpatial：在视觉语言模型中驯服分层 3D 空间理解
HiSpatial: Taming Hierarchical 3D Spatial Understanding in Vision-Language Models
把度量尺度点图作为辅助输入注入 VLM，并用四级递进的空间 VQA 数据（约 5M 图像、45M 物体）做监督微调，是提升视觉语言模型 3D 空间理解的一条可复用数据与输入表征路线。
原文依据 develop an RGB-D VLM incorporating metric-scale point maps as auxiliary inputs to further enhance spatial understanding
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论要让视觉语言模型（VLM）具备类人的空间智能，需要从 2D 观测中推断 3D 结构、识别 3D 空间中的物体属性与关系，并执行高层空间推理。本文提出一个原则性的分层框架，将 VLM 的 3D 空间理解学习分解为四个复杂度递进的层级，从几何感知一直到抽象空间推理。在该框架指导下，我们构建了一条自动化流水线，处理约 500 万张图像、超过 4500 万个物体，生成覆盖多样任务与场景的 3D 空间 VQA 对，用于 VLM 的监督微调。我们还开发了一个 RGB-D VLM，引入度量尺度点图作为辅助输入，以进一步增强空间理解。大量实验表明，我们的方法在多个空间理解与推理基准上取得最优性能，超越了专门的空间模型以及 Gemini-2.5-pro 和 GPT-5 等大型专有系统。此外，我们的分析揭示了各分层任务层级之间清晰的依赖关系，为多层级任务设计如何促进 3D 空间智能的涌现提供了新的洞见。
英文原文摘要 1247 字符
Achieving human-like spatial intelligence for vision-language models (VLMs) requires inferring 3D structures from 2D observations, recognizing object properties and relations in 3D space, and performing high-level spatial reasoning. In this paper, we propose a principled hierarchical framework that decomposes the learning of 3D spatial understanding in VLMs into four progressively complex levels, from geometric perception to abstract spatial reasoning. Guided by this framework, we construct an automated pipeline that processes approximately 5M images with over 45M objects to generate 3D spatial VQA pairs across diverse tasks and scenes for VLM supervised fine-tuning. We also develop an RGB-D VLM incorporating metric-scale point maps as auxiliary inputs to further enhance spatial understanding. Extensive experiments demonstrate that our approach achieves state-of-the-art performance on multiple spatial understanding and reasoning benchmarks, surpassing specialized spatial models and large proprietary systems such as Gemini-2.5-pro and GPT-5. Moreover, our analysis reveals clear dependencies among hierarchical task levels, offering new insights into how multi-level task design facilitates the emergence of 3D spatial intelligence.