AffordGen:借助可供性对应为可泛化物体操作生成多样化示范
AffordGen: Generating Diverse Demonstrations for Generalizable Object Manipulation with Affordance Correspondence
利用视觉基础模型在大规模三维网格上建立关键点语义对应,从而把已有操作轨迹迁移到几何形状不同的新物体上,为视觉研究者提供了一条用三维生成模型扩充机器人操作数据、并训练闭环视觉运动策略的可复用路径。
原文依据 utilizing the semantic correspondence of meaningful keypoints across large-scale 3D meshes to generate new robot manipulation trajectories
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论尽管现代模仿学习方法在机器人操作中取得了近期成功,但其性能往往受限于数据多样性不足所导致的几何变化。所提出的 AffordGen 框架借助强大的三维生成模型与视觉基础模型(VFM),利用大规模三维网格上具有语义意义的关键点之间的语义对应关系来生成新的机器人操作轨迹,从而克服这一局限。随后,这一大规模、可供性感知的数据集被用于训练一个鲁棒的闭环视觉运动策略,将可供性的语义泛化能力与端到端学习的反应式鲁棒性相结合。仿真与真实世界实验表明,使用 AffordGen 训练的策略取得了高成功率,并能零样本泛化到真正未见过的物体,显著提升了机器人学习中的数据效率。
英文原文摘要 907 字符
Despite the recent success of modern imitation learning methods in robot manipulation, their performance is often constrained by geometric variations due to limited data diversity. Leveraging powerful 3D generative models and vision foundation models (VFMs), the proposed AffordGen framework overcomes this limitation by utilizing the semantic correspondence of meaningful keypoints across large-scale 3D meshes to generate new robot manipulation trajectories. This large-scale, affordance-aware dataset is then used to train a robust, closed-loop visuomotor policy, combining the semantic generalizability of affordances with the reactive robustness of end-to-end learning. Experiments in simulation and the real world show that policies traine with AffordGen achieve high success rates and enable zero-shot generalization to truly unseen objects, significantly improving data efficiency in robot learning.