强化学习与具身智能 · CVPR2026 ·2026.06.03

全局先验遇上局部一致性:面向高效机器人操作的双记忆增强视觉-语言-动作模型

Global Prior Meets Local Consistency: Dual-Memory Augmented Vision-Language-Action Model for Efficient Robotic Manipulation

Zaijing Li、Bing Hu、Rui Shao、Gongwei Chen、Dongmei Jiang、Pengwei Xie 等 8 人

与你有关的部分

用检索到的任务级先验替代高斯噪声来缩短扩散策略的生成路径,并用局部一致性记忆约束时序平滑,可直接迁移到视觉-语言-动作模型的少步动作生成与长程操作任务。

原文依据 GPM replaces Gaussian noise with task-level priors retrieved from semantically similar trajectories, thereby shortening the generative path
可迁移到 机器人操作视觉-语言-动作建模扩散策略生成

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

分层视觉-语言-动作(VLA)模型已迅速成为机器人操作的主导范式。它通常由用于感知与理解的视觉-语言骨干网络以及用于动作生成的生成式策略组成。然而,其性能日益受到动作生成过程的瓶颈制约。(i)推理效率低。各向同性噪声先验与目标动作分布之间存在显著分布差距,这增加了去噪步数以及不可行样本的出现概率。(ii)鲁棒性差。现有策略仅以当前观测为条件,忽略了历史序列的约束,因而缺乏对任务进度和时序一致性的感知。为解决这些问题,我们提出OptimusVLA,一种带有全局先验记忆(GPM)和局部一致性记忆(LCM)的双记忆VLA框架。GPM用从语义相似轨迹中检索到的任务级先验替代高斯噪声,从而缩短生成路径并减少函数评估次数(NFE)。LCM动态建模已执行的动作序列以推断任务进度,并注入学习到的一致性约束,以强制轨迹的时序连贯性与平滑性。在三个仿真基准上,OptimusVLA持续优于强基线:在LIBERO上达到98.6%的平均成功率,在CALVIN上比pi_0提升13.5%,在RoboTwin 2.0 Hard上达到38%的平均成功率。在真实世界评估中,OptimusVLA在泛化性和长程套件上排名最佳,分别比pi_0高出42.9%和52.4%,同时实现2.9倍的推理加速。

英文原文摘要 1683 字符

Hierarchical Vision-Language-Action (VLA) models have rapidly become a dominant paradigm for robotic manipulation. It typically comprising a Vision-Language backbone for perception and understanding, together with a generative policy for action generation. However, its performance is increasingly bottlenecked by the action generation proceess. (i) Low inference efficiency. A pronounced distributional gap between isotropic noise priors and target action distributions, which increases denoising steps and the incidence of infeasible samples. (ii) Poor robustness. Existing policies condition solely on the current observation, neglecting the constraint of history sequence and thus lacking awareness of task progress and temporal consistency. To address these issues, we introduce OptimusVLA, a dual-memory VLA framework with Global Prior Memory (GPM) and Local Consistency Memory (LCM). GPM replaces Gaussian noise with task-level priors retrieved from semantically similar trajectories, thereby shortening the generative path and reducing the umber of function evaluations (NFE). LCM dynamically models executed action sequence to infer task progress and injects a learned consistency constraint that enforces temporal coherence and smoothness of trajectory. Across three simulation benchmarks, OptimusVLA consistently outperforms strong baselines: it achieves 98.6% average success rate on LIBERO, improves over pi_0 by 13.5% on CALVIN, and attains 38% average success rate on RoboTwin 2.0 Hard. In Real-World evaluation, OptimusVLA ranks best on Generalization and Long-horizon suites, surpassing pi_0 by 42.9% and 52.4%, respectively, while delivering 2.9x inference speedup.

标签

会议 CVPR2026
页码 35135-35145
Bibkey
来源 CVF Open Access