超越 3D VQA：向视觉语言模型注入 3D 空间先验以增强几何推理
Beyond 3D VQAs: Injecting 3D Spatial Priors into Vision-Language Models for Enhanced Geometric Reasoning
把点对应与深度一致性作为深层监督信号注入 LLM 的 Transformer 各层，为提升 VLM 三维空间推理提供了一条不依赖 3D VQA 数据、可复用的训练路径；其逐层对应匹配诊断也可直接用于分析其他 VLM 的内部几何表征。
原文依据 GASP employs a small correspondence head, applied as a deep supervision signal across all layers
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉语言模型（VLM）往往难以进行稳健的 3D 空间推理。主流的依赖 3D 视觉问答（VQA）数据集微调的方法可能过拟合数据集特有的偏差，而集成专门的 3D 视觉编码器通常不够灵活且繁琐。本文认为，真正的空间理解应当源于对基本几何先验的学习，而不仅仅来自高层次的 VQA 监督。我们提出 GASP（Geometric-Aware Spatial Priors），一个将这些先验直接注入 LLM 的 Transformer 层的框架。GASP 采用一个小型对应头，作为跨所有层的深层监督信号，并以利用大规模视频场景真值几何的双重目标进行训练：基于真值点对应的对比损失强制实现 2D 视角不变性，而深度一致性监督则消解 3D 几何歧义。我们的分析首先给出诊断性结果，表明标准 VLM 的内部对应匹配准确率非常低（通常低于 5%）。随后我们证明，我们的训练显著改善了该行为，将逐层对应峰值提升至 70% 以上，并在基线仍低于 5% 的情况下保持超过 85% 的时序鲁棒性。这些内部改进转化为下游空间基准上的显著收益——包括 All-Angles Bench 上 +18.2% 和 VSI-Bench 上 +29.0%——且完全未在任何 3D VQA 数据上训练。我们的发现表明，从基本几何先验中学习是通向具有更可靠 3D 空间推理能力的 VLM 的一条有前景且可泛化的路径。
英文原文摘要 1652 字符
Vision-Language Models (VLMs) often struggle with robust 3D spatial reasoning. Prevailing methods that rely on fine-tuning with 3D visual question-answering (VQA) datasets may overfit dataset-specific biases, while integrating specialized 3D visual encoders is often inflexible and cumbersome. In this paper, we argue that genuine spatial understanding should emerge from learning fundamental geometric priors, not only from high-level VQA supervision. We propose GASP (Geometric-Aware Spatial Priors), a framework that injects these priors directly into the LLM's transformer layers. GASP employs a small correspondence head, applied as a deep supervision signal across all layers, and is trained with a dual objective leveraging ground-truth geometry from large-scale video scenes: a contrastive loss on ground-truth point correspondences enforces 2D view-invariance, while a depth consistency supervision resolves 3D geometric ambiguities. Our analysis first provides a diagnostic showing that standard VLMs' internal correspondence matching accuracy is very low (often below 5%). We then demonstrate that our training substantially improves this behavior, boosting peak layer-wise correspondence to over 70% and maintaining over 85% temporal robustness while baselines remain below 5%. These internal improvements translate to significant gains on downstream spatial benchmarks---including +18.2% on All-Angles Bench and +29.0% on VSI-Bench---all without training on any 3D VQA data. Our findings indicate that learning from fundamental geometric priors is a promising and generalizable pathway towards VLMs with more reliable 3D spatial reasoning.