DVGT：面向自动驾驶的视觉几何 Transformer
DVGT: Driving Visual Geometry Transformer
无需相机内外参、直接从无位姿多视角图像序列回归首帧自车坐标系下的全局稠密点图与逐帧自车位姿，这种「无显式几何先验 + 度量尺度直接预测」的设定可复用于任意相机配置下的在线建图与占用/深度预训练。
原文依据 reconstructs a global dense 3D point map from a sequence of unposed multi-view visual inputs
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论从视觉输入中感知并重建三维场景几何对自动驾驶至关重要。然而，目前仍缺乏一个面向自动驾驶、能够适应不同场景与相机配置的稠密几何感知模型。为填补这一空白，我们提出面向自动驾驶的视觉几何 Transformer（DVGT），它从一段无位姿的多视角视觉输入序列中重建全局稠密三维点图。我们首先使用 DINO 主干网络为每张图像提取视觉特征，并采用交替的视图内局部注意力、跨视图空间注意力与跨帧时序注意力来推断图像之间的几何关系。随后，我们使用多个解码头，在第一帧的自车坐标系下解码出全局点图，并解码出每一帧的自车位姿。与依赖精确相机参数的传统方法不同，DVGT 不依赖显式的三维几何先验，从而能够灵活处理任意相机配置。DVGT 直接从图像序列预测具有度量尺度的几何，无需再与外部传感器进行后对齐。在包括 nuScenes、OpenScene、Waymo、KITTI 与 DDAD 在内的大规模混合自动驾驶数据集上训练后，DVGT 在多种场景下显著优于现有模型。
英文原文摘要 1264 字符
Perceiving and reconstructing 3D scene geometry from visual inputs is crucial for autonomous driving. However, there still lacks a driving-targeted dense geometry perception model that can adapt to different scenarios and camera configurations. To bridge this gap, we propose a Driving Visual Geometry Transformer (DVGT), which reconstructs a global dense 3D point map from a sequence of unposed multi-view visual inputs. We first extract visual features for each image using a DINO backbone, and employ alternating intra-view local attention, cross-view spatial attention, and cross-frame temporal attention to infer geometric relations across images. We then use multiple heads to decode a global point map in the ego coordinate of the first frame and the ego poses for each frame. Unlike conventional methods that rely on precise camera parameters, DVGT is free of explicit 3D geometric priors, enabling flexible processing of arbitrary camera configurations. DVGT directly predicts metric-scaled geometry from image sequences, eliminating the need for post-alignment with external sensors. Trained on a large mixture of driving datasets including nuScenes, OpenScene, Waymo, KITTI, and DDAD, DVGT significantly outperforms existing models on various scenarios.