强化学习与具身智能 · CVPR2026 ·2026.06.03

通过显式从失败中学习释放自动驾驶中的 VLA 潜力

Unleashing VLA Potentials in Autonomous Driving via Explicit Learning from Failures

Yuechen Luo、Fang Li、Qimao Chen、Shaoqing Xu、Jiaxin Liu、Ziying Song 等 8 人

与你有关的部分

对做自动驾驶 VLA 的研究者,这篇论文给出了一种把失败诊断的结构化文本反馈转成高奖励修正样本、再注入 RL 训练批次的做法,可用来突破 SFT 后 RL 探索受限导致的性能平台期,尤其适用于长尾场景中奖励稀疏的规划任务。

原文依据 By injecting these corrected, high-reward samples back into the RL training batch, our approach provides a targeted gradient
可迁移到 自动驾驶规划视觉语言动作模型强化学习微调

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

面向自动驾驶的视觉-语言-动作(VLA)模型在强化学习(RL)优化过程中常常遭遇性能平台期。这种停滞源于探索能力受到先前监督微调(SFT)的约束,从而在长尾场景中导致“持续性失败”。在这些关键情形下,所有被探索的动作都只能得到零值的驾驶得分。这种信息稀疏的奖励只表明发生了失败,却无法指出失败的根因——究竟是规划错误、推理有缺陷,还是轨迹执行不佳。为应对这一局限,我们提出了显式从失败中学习的 VLA(ELF-VLA),这是一个用结构化诊断反馈增强 RL 的框架。我们的方法不再依赖模糊的标量奖励,而是生成详细、可解释的报告,指出具体的失败模式。随后 VLA 策略利用这一显式反馈生成反馈引导的精炼。通过将这些经过修正的高奖励样本注入 RL 训练批次,我们的方法提供了有针对性的梯度,使策略能够解决无引导探索无法解决的关键场景。大量实验表明,我们的方法释放了 VLA 模型的潜在能力,在公开的 Navsim 基准上于总体 PDMS、EPDMS 得分和高层规划准确率方面取得了最优性能。

英文原文摘要 1430 字符

Vision-Language-Action (VLA) models for autonomous driving often hit a performance plateau during Reinforcement Learning (RL) optimization. This stagnation arises from exploration capabilities constrained by previous Supervised Fine-Tuning (SFT), leading to "persistent failures" in long-tail scenarios. In these critical situations, all explored actions yield a zero-value driving score. This information-sparse reward signals a failure, yet fails to identify its root cause--whether it is due to incorrect planning, flawed reasoning, or poor trajectory execution. To address this limitation, we propose **VLA** with **E**xplicit **L**earning from **F**ailures (**ELF-VLA**), a framework that augments RL with structured diagnostic feedback. Instead of relying on a vague scalar reward, our method produces detailed, interpretable reports that identify the specific failure mode. The VLA policy then leverages this explicit feedback to generate a **Feedback-Guided Refinement**. By injecting these corrected, high-reward samples back into the RL training batch, our approach provides a targeted gradient, which enables the policy to solve critical scenarios that unguided exploration cannot. Extensive experiments demonstrate that our method unlocks the latent capabilities of VLA models, achieving state-of-the-art (SOTA) performance on the public Navsim benchmark for overall PDMS, EPDMS score and high-level planning accuracy.

标签

会议 CVPR2026
页码 24833-24842
Bibkey
来源 CVF Open Access