RehearseVLA：面向 VLA 的物理一致世界模型模拟后训练
RehearseVLA: Simulated Post-Training for VLAs with Physically-Consistent World Model
用世界模型生成时序一致的未来视觉观测来替代真实环境交互，为 VLA 做安全的 RL 后训练；其中 VLM 引导的即时反射器同时充当连续奖励与动作终止判别器，这一『视觉预测+奖励/终止联合建模』思路可迁移到任何需要在线评估的视觉策略学习场景。
原文依据 a physically-consistent world simulator that generates temporally consistent future visual observations
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论通过模仿学习训练的视觉语言动作（VLA）模型依赖大规模示教数据集，因此在数据稀缺场景下性能显著下降。尽管基于强化学习（RL）的后训练已被证明能有效缓解数据稀缺问题，但其在 VLA 模型上的应用受制于真实环境不可重置的特性。这一局限在工业自动化等高风险领域尤为关键，因为交互往往引发难以承受或无法回退的状态变化。此外，现有 VLA 方法缺乏可靠的任务完成检测机制，导致冗余动作并降低整体任务成功率。为应对这些挑战，我们提出 RehearseVLA，一个基于 RL 的后训练框架，用低成本的世界模型虚拟模拟器替代物理交互。RehearseVLA 包含两个关键组件：（1）物理一致的世界模拟器，生成时序一致的未来视觉观测；（2）由视觉语言模型（VLM）引导的即时反射器，提供连续奖励信号并预测动作终止。该模拟环境使 VLA 模型能够安全探索并泛化到初始模仿学习分布之外。我们的方法在每任务仅需五个专家示教的情况下即取得显著性能提升。复杂机器人操作任务上的实验表明，RehearseVLA 有效克服了依赖真实世界交互的传统 VLA 模型所面临的数据低效、安全约束与执行低效问题，为资源受限场景下的后训练提供了实用且可扩展的解决方案。代码已发布于 https://github.com/iSEE-Laboratory/RehearseVLA。
英文原文摘要 1848 字符
Vision-Language-Action (VLA) models trained via imitation learning suffer from significant performance degradation in data-scarce scenarios due to their reliance on large-scale demonstration datasets. Although reinforcement learning (RL)-based post-training has proven effective in addressing data scarcity, its application to VLA models is hindered by the non-resettable nature of real-world environments. This limitation is particularly critical in high-risk domains such as industrial automation, where interactions often induce state changes that are costly or infeasible to revert. Furthermore, existing VLA approaches lack a reliable mechanism for detecting task completion, leading to redundant actions that reduce overall task success rates. To address these challenges, we propose RehearseVLA, an RL-based post-training framework that replaces physical interaction with a low-cost world model-based virtual simulator. RehearseVLA consists of two key components: (1) a physically-consistent world simulator that generates temporally consistent future visual observations, and (2) a vision-language model (VLM)-guided instant reflector that provides continuous reward signals and predicts action termination. This simulated environment enables VLA models to safely explore and generalize beyond their initial imitation learning distribution. Our method achieves notable performance gains with as few as five expert demonstrations per task. Experiments on complex robotic manipulation tasks demonstrate that RehearseVLA effectively overcomes the data inefficiency, safety constraints, and inefficient execution of conventional VLA models that rely on real-world interaction, offering a practical and scalable solution for post-training in resource-constrained settings. Our code is available at https://github.com/iSEE-Laboratory/RehearseVLA.