基于视觉定位的3D感知空间视觉语言建模
Grounded 3D-Aware Spatial Vision-Language Modeling
把「生成时插入区域 token」的隐式视觉定位与区域提示的单目 3D 框预测结合，可作为 VLM 空间推理的可复用范式：在解码过程中动态引用视觉证据，并用内参感知归一化与稠密几何监督把 2D 定位升级为 3D 推断。
原文依据 inserts the corresponding region tokens into the text stream, allowing the model to reference visual evidence on the fly
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论本文提出 GR3D，一种在单一框架内同时具备三种互补视觉定位能力的空间视觉语言模型：显式 2D 视觉定位、隐式 2D 视觉定位与单目 3D 视觉定位。GR3D 引入隐式视觉定位机制，在生成过程中识别实体提及并将对应的区域 token 插入文本流，使模型在生成空间思维链回答时能够即时引用视觉证据。与此同时，一种区域提示的单目 3D 视觉定位设计从已定位的区域查询出发，预测相机视角下的 3D 边界框，并由内参感知归一化与稠密几何监督提供支撑。这些视觉定位能力共同使 GR3D 能够将复杂的空间理解问题分解为先进行有视觉定位的 2D 感知、再进行 3D 推断。GR3D 在需要视觉定位与不需要视觉定位的空间基准上均取得一致提升，表明视觉定位是一种有效的归纳偏置，可增强视觉语言模型的空间理解能力。这些视觉定位能力共同提升了超出视觉定位任务本身的一般空间理解能力。
英文原文摘要 1160 字符
We present GR3D, a spatial vision language model equipped with three complementary grounding capabilities--explicit 2D grounding, implicit 2D grounding, and monocular 3D grounding--within a single framework. GR3D introduces an implicit grounding mechanism that identifies entity mentions during generation and inserts the corresponding region tokens into the text stream, allowing the model to reference visual evidence on the fly when producing spatial chain-of-thought responses. In parallel, a region-prompted monocular 3D grounding design predicts 3D bounding boxes in the camera view from grounded region queries, supported by intrinsic-aware normalization and dense geometric supervision. Together, these grounding capabilities enable GR3D to decompose complex spatial understanding problems into grounded 2D perception followed by 3D inference. GR3D achieves consistent improvements across grounded and non-grounded spatial benchmarks, demonstrating grounding as an effective inductive bias for strengthening spatial understanding in VLMs. These grounding capabilities collectively enhance general spatial understanding beyond the grounding task itself.