自动驾驶与遥感 · CVPR2026 ·2026.06.03

RAG-TP:基于检索增强生成的车辆轨迹预测通用框架

RAG-TP: A General Framework for Vehicle Trajectory Prediction via Retrieval-Augmented Generation

Ziyi Wang、Yang Zhang、Guijian Tang、Chao Zhang、Shibo Zhang、Xueqiong Li 等 7 人

与你有关的部分

把轨迹预测从依赖在线感知先验改为从结构化知识库检索历史经验,并用检索驱动的 MoE 加交叉注意力做动态融合;对做自动驾驶预测、以及想把 RAG 思路引入视觉时序任务的人,这是一个可复用的「离线知识库 + 推理时检索」范式。

原文依据 we introduce RAG-TP, a framework reframing the problem from relying on uncertain online perception to retrieving from a large-scale, structured knowledge base
可迁移到 车辆轨迹预测自动驾驶运动预测跨域泛化

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

车辆轨迹预测对安全高效的自动驾驶至关重要。然而,其对实时在线先验的严重依赖阻碍了泛化性与可扩展性。为突破这一瓶颈,我们提出 RAG-TP,该框架将问题从依赖不确定的在线感知重新表述为从大规模结构化知识库中检索。RAG-TP 通过动态查询富含场景拓扑与运动模式的异构知识库,以检索到的历史经验作为先验,从而增强推理时的预测。我们进一步设计了基于新型检索驱动混合专家(MoE)的动态融合模块。与传统参数化设计不同,该机制动态地将检索到的知识单元视为专家,通过交叉注意力对其进行加权与整合,以生成用于最终多模态轨迹解码的稠密上下文。通过将在线推理与离线知识解耦,该方法将预测建立在庞大的结构化数据库之上,缓解模型幻觉、补偿不可靠先验,并显著增强鲁棒性与域适应能力。大量实验表明,RAG-TP 在有地图与无地图设定下均取得优异性能,与专门的无地图方法相比极具竞争力,同时与最优(SOTA)的有地图模型表现相当。它在跨域与零样本泛化方面尤其展现出显著优势。我们的工作为构建可扩展且鲁棒的自动驾驶预测系统提供了一条有前景的技术路径。

英文原文摘要 1632 字符

Vehicle trajectory prediction is critical for safe and efficient autonomous driving. However, its generalization and scalability are hindered by heavy reliance on real-time, online priors. To break this bottleneck, we introduce RAG-TP, a framework reframing the problem from relying on uncertain online perception to retrieving from a large-scale, structured knowledge base. RAG-TP enhances inference-time predictions by dynamically querying a heterogeneous knowledge base rich with scene topologies and motion patterns, using retrieved historical experiences as priors. We further design a dynamic fusion module based on a novel Retrieval-Driven Mixture-of-Experts (MoE). Unlike conventional parametric designs, this mechanism dynamically treats retrieved knowledge units as experts, weighting and integrating them via cross-attention to generate a dense context for final multi-modal trajectory decoding. By decoupling online inference from offline knowledge, this approach grounds predictions in a vast structured database, mitigating model hallucination, compensating for unreliable priors, and significantly enhancing robustness and domain adaptation. Extensive experiments show RAG-TP achieves excellent performance in map-based and map-free settings, demonstrating highly competitive results against specialized map-free methods while performing on par with state-of-the-art (SOTA) map-based models. It demonstrates significant advantages, particularly in cross-domain and zero-shot generalization. Our work provides a promising technical pathway toward building scalable and robust prediction systems for autonomous driving.

标签

会议 CVPR2026
页码 24865-24874
Bibkey
来源 CVF Open Access