强化学习与具身智能 · CVPR2026 ·2026.06.03

BiPreManip:通过前瞻性协作学习基于可供性的双臂预备操作

BiPreManip: Learning Affordance-Based Bimanual Preparatory Manipulation through Anticipatory Collaboration

Yan Shen、Feng Jiang、Zichen He、Xiaoqi Li、Yuchen Liu、Zhiyu Li 等 8 人

与你有关的部分

把视觉可供性作为双臂之间的共享中间表示,先预测目标动作的可供性、再反推预备动作,这种「可供性前瞻推理」可直接迁移到其他长程操作与工具使用任务中,作为跨臂/跨步骤的协调接口。

原文依据 we propose a visual affordance-based framework that first envisions the final goal-directed action
可迁移到 机器人操作可供性预测视觉运动策略学习长程任务规划

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

许多日常物体难以被直接抓取(例如平放的 iPad),也难以进行功能性操作(例如打开放在桌上的笔帽)。这类任务需要双臂之间顺序化、非对称的协调:一只手臂执行预备操作,从而为另一只手臂的目标导向动作创造条件——例如先把 iPad 推到桌子边缘再将其拿起,或先抬起笔身以便另一只手取下笔帽。在本工作中,我们提出协作预备操作(Collaborative Preparatory Manipulation),这是一类双臂操作任务,要求理解物体的语义与几何、预判空间关系,并在双臂之间规划长程协调动作。为应对这一挑战,我们提出一个基于视觉可供性的框架:该框架首先设想最终的目标导向动作,然后引导一只手臂执行一系列预备操作,以促成另一只手臂随后的操作。这种以可供性为中心的表征使双臂之间能够进行前瞻性推理与协调,并可跨类别各异的多种物体有效泛化。在仿真与真实世界中的大量实验表明,与有竞争力的基线相比,我们的方法显著提升了任务成功率与泛化能力。

英文原文摘要 1340 字符

Many everyday objects are difficult to directly grasp (e.g., a flat iPad) or manipulate functionally (e.g., opening the cap of a pen lying on a desk). Such tasks require sequential, asymmetric coordination between two arms, where one arm performs preparatory manipulation that enables the other's goal-directed action--for instance, pushing the iPad to the table's edge before picking it up, or lifting the pen body to allow the other hand to remove its cap. In this work, we introduce Collaborative Preparatory Manipulation, a class of bimanual manipulation tasks that demand understanding object semantics and geometry, anticipating spatial relationships, and planning long-horizon coordinated actions between the two arms. To tackle this challenge, we propose a visual affordance-based framework that first envisions the final goal-directed action and then guides one arm to perform a sequence of preparatory manipulations that facilitate the other arm's subsequent operation. This affordance-centric representation enables anticipatory inter-arm reasoning and coordination, generalizing effectively across various objects spanning diverse categories. Extensive experiments in both simulation and the real world demonstrate that our approach substantially improves task success rates and generalization compared to competitive baselines.

标签

会议 CVPR2026
页码 42430-42440
Bibkey
来源 CVF Open Access