反事实 VLA:具备自适应推理的自反思视觉-语言-动作模型
Counterfactual VLA: Self-Reflective Vision-Language-Action Model with Adaptive Reasoning
把反事实自反思作为 VLA 执行前的推理环节,并配合 rollout-filter-label 数据挖掘管线,可为视觉驱动的端到端规划提供「先想后做」的可复用范式,尤其适合需要安全约束的轨迹生成任务。
原文依据 performs counterfactual reasoning conditioned on both the meta-actions and the visual context
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近期增强推理的视觉-语言-动作(VLA)模型通过生成中间推理轨迹,提升了端到端自动驾驶的可解释性。然而这些模型主要描述其感知到的内容与意图执行的动作,很少质疑其规划动作是否安全或恰当。本文提出反事实 VLA(CF-VLA),一种自反思 VLA 框架,使模型能够在执行前对规划动作进行推理并加以修正。CF-VLA 首先生成按时间分段的元动作以概括驾驶意图,随后以元动作与视觉上下文为条件进行反事实推理。该步骤模拟潜在结果、识别不安全行为,并输出修正后的元动作以指导最终轨迹生成。为高效获得这种自反思能力,我们提出 rollout-filter-label 管线,从基础(非反事实)VLA 的 rollout 中挖掘高价值场景,并为后续训练轮次标注反事实推理轨迹。在大规模驾驶数据集上的实验表明,CF-VLA 将轨迹精度提升最多 17.6%,安全指标提升 20.5%,并展现出自适应思考能力:仅在具有挑战性的场景中启用反事实推理。通过将推理轨迹从单样本描述转变为因果自我修正信号,CF-VLA 朝着学会在行动前思考的自反思自动驾驶智能体迈出了一步。
英文原文摘要 1518 字符
Recent reasoning-augmented Vision-Language-Action (VLA) models have improved the interpretability of end-to-end autonomous driving by generating intermediate reasoning traces. Yet these models primarily describe what they perceive and intend to do, rarely questioning whether their planned actions are safe or appropriate. This work introduces Counterfactual VLA (CF-VLA), a self-reflective VLA framework that enables the model to reason about and revise its planned actions before execution. CF-VLA first generates time-segmented meta-actions that summarize driving intent, and then performs counterfactual reasoning conditioned on both the meta-actions and the visual context. This step simulates potential outcomes, identifies unsafe behaviors, and outputs corrected meta-actions that guide the final trajectory generation. To efficiently obtain such self-reflective capabilities, we propose a rollout-filter-label pipeline that mines high-value scenes from a base (non-counterfactual) VLA's rollouts and labels counterfactual reasoning traces for subsequent training rounds. Experiments on large-scale driving datasets show that CF-VLA improves trajectory accuracy by up to 17.6%, enhances safety metrics by 20.5%, and exhibits adaptive thinking: it only enables counterfactual reasoning in challenging scenarios. By transforming reasoning traces from one-shot descriptions to causal self-correction signals, CF-VLA takes a step toward self-reflective autonomous driving agents that learn to think before they act.