EE-RL：面向端到端自动驾驶的探索者与专家模型视觉语言引导强化学习
EE-RL: Vision Language Guided Reinforcement Learning with Explorer and Expert model for End-to-End Autonomous Driving
将微调后的视觉语言模型作为稀疏关键场景的推理专家，并用StateHash按RGB相似度与运动学相似度跳过冗余VLM推理，这一「按需调用VLM」的机制可迁移到其他需要昂贵大模型推理的在线决策或感知流程中。
原文依据 the StateHash algorithm is designed to separately measure RGB image similarity and vehicle kinematic similarity, thereby skipping unnecessary VLM reasoning
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论直接将原始传感器数据映射为车辆控制命令的端到端驾驶框架已展现出显著潜力。然而，在稀疏关键场景（即罕见但安全敏感的事件发生的场景）中，其性能往往退化。为解决该问题，我们提出探索者-专家强化学习（EE-RL），一种新颖的端到端框架，集成了基于强化学习的探索者、基于微调视觉语言模型（VLM）的专家以及双经验回放缓冲区。EE-RL 采用协同学习策略，由探索者与两个专家从常规驾驶场景中联合生成经验以引导策略学习。随着训练推进，其中一个 VLM 专注于对稀疏关键场景进行推理，从而提升两类场景下的学习效率与策略优化。此外，我们设计了 StateHash 算法，分别度量 RGB 图像相似度与车辆运动学相似度，从而跳过不必要的 VLM 推理，实现更密集、更有效的专家经验生成。在 CARLA Leaderboard 上的实验表明，EE-RL 显著优于最优基线，在 Town03 上的驾驶分数与违规分数分别提升 +19.82% 与 +20.98%。EE-RL 在 Town05-06 上进一步实现闯红灯事故概率为 0%、平均驾驶分数达 80.09，展现出鲁棒性、泛化能力以及处理稀疏关键场景的能力。
英文原文摘要 1509 字符
End-to-end driving frameworks, which directly map raw sensor data to vehicle control commands, have shown remarkable potential. However, their performance often deteriorates in sparse-critical scenarios, where rare but safety-sensitive events occur. To address this problem, we propose Explorer-Expert Reinforcement Learning (EE-RL), a novel end-to-end framework that integrates an RL-based explorer, a fine-tuned vision-language model (VLM)-based expert, and a dual replay buffer. EE-RL adopts a collaborative learning strategy in which an explorer and two experts jointly generate experiences from regular driving scenarios to guide policy learning. As training progresses, one of the VLMs focuses on reasoning about sparse-critical scenarios, thereby enhancing learning efficiency and policy optimization in both scenarios. Additionally, the StateHash algorithm is designed to separately measure RGB image similarity and vehicle kinematic similarity, thereby skipping unnecessary VLM reasoning and enabling denser, more effective expert experience generation. Experiments on the CARLA Leaderboard demonstrate that EE-RL significantly outperforms state-of-the-art (SOTA) baselines, achieving +19.82% and +20.98% improvements in driving and infraction scores on Town03, respectively. The EE-RL further achieves 0% accident probability for red-light running and an average driving score of 80.09 in Town05-06, demonstrating robustness, generalizability, and the capability to handle sparse-critical scenarios.