空间检索增强的自动驾驶
Spatial Retrieval Augmented Autonomous Driving
把离线检索的地理图像作为额外输入模态，与车载传感器特征对齐融合，为遮挡、暗光、雨雾等感知退化场景提供先验；做自动驾驶感知/规划的研究者可以把它当作即插即用的额外输入分支，或直接使用其扩展的 nuScenes 基准来评测跨模态融合。
原文依据 we propose the spatial retrieval paradigm, introducing offline retrieved geographic images as an additional input
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论现有自动驾驶系统依赖车载传感器（相机、LiDAR、IMU 等）进行环境感知。然而，这一范式受限于行驶时的感知视野，在视野范围有限、遮挡或黑暗与降雨等极端条件下往往失效。相比之下，人类驾驶员即使在能见度很差时也能回忆起道路结构。为使模型具备这种回忆能力，我们提出空间检索范式，引入离线检索的地理图像作为额外输入。这些图像可以方便地从离线缓存（例如 Google Maps 或已存储的自动驾驶数据集）中获取，无需额外传感器，因而对现有自动驾驶任务是即插即用的扩展。在实验中，我们首先用通过 Google Maps API 检索到的地理图像扩展 nuScenes 数据集，并将新数据与自车轨迹对齐。我们在五个核心自动驾驶任务上建立了基线：目标检测、在线建图、占用预测、端到端规划以及生成式世界建模。大量实验表明，这一扩展模态能够提升某些任务的性能。我们将开源数据集构建代码、数据与基准，以推动对这一新自动驾驶范式的进一步研究。
英文原文摘要 1302 字符
Existing autonomous driving systems rely on onboard sensors (cameras, LiDAR, IMU, etc) for environmental perception. However, this paradigm is limited by the drive-time perception horizon and often fails under limited view scope, occlusion or extreme conditions such as darkness and rain. In contrast, human drivers are able to recall road structure even under poor visibility. To endow models with this recall ability, we propose the spatial retrieval paradigm, introducing offline retrieved geographic images as an additional input. These images are easy to obtain from offline caches (e.g., Google Maps or stored autonomous driving datasets) without requiring additional sensors, making it a plug-and-play extension for existing AD tasks. For experiments, we first extend the nuScenes dataset with geographic images retrieved via Google Maps APIs and align the new data with ego-vehicle trajectories. We establish baselines across five core autonomous driving tasks: object detection, online mapping, occupancy prediction, end-to-end planning, and generative world modeling. Extensive experiments show that the extended modality could enhance the performance of certain tasks. We will open-source dataset curation code, data, and benchmarks for further study of this new autonomous driving paradigm.