强化学习与具身智能 · CVPR2026 ·2026.06.03

SaPaVe:面向机器人视觉-语言动作模型的主动感知与操作

SaPaVe: Towards Active Perception and Manipulation in Vision-Language Action Models for Robotics

Mengzhen Liu、Enshen Zhou、Cheng Chi、Yi Han、Shanyu Rong、Liming Chen 等 9 人

与你有关的部分

将相机运动作为独立的语义控制动作与操作动作解耦,并用3D几何感知模块提升动态视角下的执行鲁棒性,为视觉-语言-动作模型提供可复用的主动视角控制范式与ActiveViewPose-200K数据。

原文依据 we first train semantic camera control on our proposed large-scale dataset, then jointly optimizes both action types via hybrid data
可迁移到 vision-language-actionactive perceptionrobotic manipulationcamera control

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

主动感知与操作是机器人交互复杂场景的关键能力。现有方法难以将语义驱动的主动感知与鲁棒、视角不变的操作执行统一起来。为此,我们提出SaPaVe,一个以数据高效方式联合学习上述能力的端到端框架。与共享动作空间的做法相反,我们的核心是将相机动作与操作动作解耦,并采用自底向上的学习策略:首先在我们提出的大规模数据集上训练语义相机控制,然后通过混合数据联合优化两类动作。为支撑该策略,我们引入ActiveViewPose-200K,包含20万组图像-语言-相机运动三元组用于语义相机运动学习,并提出一个3D几何感知模块以提升动态视角下的执行鲁棒性。我们进一步提出ActiveManip-Bench,这是首个填补主动操作评测空白的基准。在仿真与真实环境中的大量实验表明,SaPaVe优于GR00T N1和pi0等近期VLA模型,在真实任务中成功率最高提升31.25%。结果表明,当感知与执行以解耦而协同的策略训练时,紧密耦合的感知与执行能够实现高效且可泛化的主动操作。

英文原文摘要 1342 字符

Active perception and manipulation are crucial for robots to interact with complex scenes. Existing methods struggle to unify semantic-driven perception actively with robust, viewpoint-invariant execution accordingly. To this end, we propose SaPaVe, an end-to-end framework that jointly learns these capabilities in a data-efficient manner. Central to our approach is a decoupling of camera and manipulation actions, contrary to shared-action-space, and learning in a bottom-up strategy: we first train semantic camera control on our proposed large-scale dataset, then jointly optimizes both action types via hybrid data. To support this, we introduce ActiveViewPose-200K, comprising 200k image-language-camera movement pairs for semantic camera movement learning, and a 3D geometry-aware module that improves execution robustness under dynamic viewpoints. We further present ActiveManip-Bench, the first benchmark filling the gap to evaluate active manipulation. Extensive experiments in both simulation and real-world settings show that SaPaVe outperforms recent VLA models such as GR00T N1 and pi0, achieving up to 31.25% higher success rates in real-world tasks. Our results show that tightly coupled perception and execution, when trained with decoupled yet coordinated strategies, enable efficient and generalizable active manipulation.

标签

会议 CVPR2026
页码 37164-37174
Bibkey
来源 CVF Open Access