SpaceDrive：将空间感知注入基于视觉语言模型的自动驾驶
SpaceDrive: Infusing Spatial Awareness into VLM-based Autonomous Driving
把 3D 坐标（多视角深度、历史自车状态、文本提示）统一编码为位置编码，既叠加到 2D 视觉 token 上，又替代数字 token 作为 VLM 的输入输出，使 VLM 能直接回归连续轨迹坐标；这一「空间位置编码替代数字 token」的思路可迁移到任何需要 VLM 做几何量输出的视觉任务。
原文依据 treats spatial information as explicit positional encodings (PEs) instead of textual digit tokens
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论基于视觉语言模型（VLM）的端到端自动驾驶方法，凭借其在大规模预训练中获得的通用视觉理解与强推理能力，经历了快速发展。然而，我们发现当前的 VLM 难以理解细粒度的三维空间关系，而这正是与物理世界交互的系统的一项基本要求。为解决该问题，我们提出 SpaceDrive，一个具备空间感知能力的、基于 VLM 的驾驶框架，它将空间信息视为显式的位置编码（PE），而非文本数字 token，从而实现对语义表示与空间表示的联合推理。SpaceDrive 对所有由多视角深度估计、历史自车状态和文本提示得到的三维坐标采用统一的位�置编码器。这些三维位置编码首先被叠加，以增强对应的二维视觉 token。同时，它们作为一种与任务无关的坐标表示，替代逐位数字 token，作为 VLM 的输入与输出。该机制使模型能够在空间推理中更好地索引特定的视觉语义，并直接回归轨迹坐标而非逐位生成数字，从而提升规划精度。大量实验验证，SpaceDrive 在 nuScenes 数据集上取得了最优的开环性能，并在 Bench2Drive 闭环基准上取得了 78.02 的驾驶得分，在现有基于 VLM 的方法中位列第二。代码已发布于：https://github.com/zhenghao2519/SpaceDrive。
英文原文摘要 1581 字符
End-to-end autonomous driving methods built on vision language models (VLMs) have undergone rapid development driven by their universal visual understanding and strong reasoning capabilities obtained from the large-scale pretraining. However, we find that current VLMs struggle to understand fine-grained 3D spatial relationships which is a fundamental requirement for systems interacting with the physical world. To address this issue, we propose SpaceDrive, a spatial-aware VLM-based driving framework that treats spatial information as explicit positional encodings (PEs) instead of textual digit tokens, enabling joint reasoning over semantic and spatial representations. SpaceDrive employs a universal positional encoder to all 3D coordinates derived from multi-view depth estimation, historical ego-states, and text prompts. These 3D PEs are first superimposed to augment the corresponding 2D visual tokens. Meanwhile, they serve as a task-agnostic coordinate representation, replacing the digit-wise numerical tokens as both inputs and outputs for the VLM. This mechanism enables the model to better index specific visual semantics in spatial reasoning and directly regress trajectory coordinates rather than generating digit-by-digit, thereby enhancing planning accuracy. Extensive experiments validate that SpaceDrive achieves state-of-the-art open-loop performance on the nuScenes dataset and the second-best Driving Score of 78.02 on the Bench2Drive closed-loop benchmark over existing VLM-based methods. Code is available at: https://github.com/zhenghao2519/SpaceDrive.