以 3D 思考：基于有限视角的几何想象驱动空间推理
Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Views
把 VLM 推理过程中的隐状态与 VGGT 等 3D 基础模型输出的几何潜变量对齐，使模型在无 3D 先验输入时也能进行「脑内 3D 想象」，为视觉研究者提供了一条把 3D 几何表征注入多模态推理的可复用路径。
原文依据 we perform supervised training to align the 3D latent generated by VLM while reasoning with that of a 3D foundation model
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论尽管视觉语言模型（VLM）近期在广泛的多模态任务上取得了显著进展，但从有限视角理解 3D 空间关系仍是一项重大挑战。此前的推理方法通常依赖纯文本（如拓扑认知地图）或 2D 视觉线索，然而其有限的表征能力制约了在需要 3D 空间想象的特定任务上的表现。为解决这一局限，我们提出 3DThinker，一个能够在推理过程中像人类一样有效利用图像中蕴含的丰富几何信息的框架。该框架首次实现了在推理过程中进行 3D 心智建模，且无需任何 3D 先验输入，也不依赖显式标注的 3D 数据进行训练。具体而言，我们的训练分为两个阶段。首先，我们进行监督训练，将 VLM 在推理时生成的 3D 潜变量与 3D 基础模型（如 VGGT）的 3D 潜变量对齐。随后，我们仅基于结果信号优化整个推理轨迹，从而精炼底层的 3D 心智建模。在多个基准上的大量实验表明，3DThinker 持续优于强基线，并为将 3D 表征统一到多模态推理中提供了新视角。我们的代码已发布于 https://github.com/zhangquanchen/3DThinker。
英文原文摘要 1389 字符
Though recent advances in vision-language models (VLMs) have achieved remarkable progress across a wide range of multimodal tasks, understanding 3D spatial relationships from limited views remains a significant challenge. Previous reasoning methods typically rely on pure text (e.g., topological cognitive maps) or on 2D visual cues. However, their limited representational capacity hinders performance in specific tasks that require 3D spatial imagination. To address this limitation, we propose 3DThinker, a framework that can effectively exploit the rich geometric information embedded within images while reasoning, like humans do. Our framework is the first to enable 3D mentaling during reasoning without any 3D prior input, and it does not rely on explicitly labeled 3D data for training. Specifically, our training consists of two stages. First, we perform supervised training to align the 3D latent generated by VLM while reasoning with that of a 3D foundation model (e.g., VGGT). Then, we optimize the entire reasoning trajectory solely based on outcome signals, thereby refining the underlying 3D mentaling. Extensive experiments across multiple benchmarks show that 3DThinker consistently outperforms strong baselines and offers a new perspective toward unifying 3D representations into multimodal reasoning. Our code is available at https://github.com/zhangquanchen/3DThinker.