AffordGrasp:面向可供性的跨模态扩散抓取合成
AffordGrasp: Cross-Modal Diffusion for Affordance-Aware Grasp Synthesis
把可供性(affordance)作为空间条件注入扩散过程来生成手部抓取姿态,其双条件扩散与分布调整模块可迁移到任何需要几何+语义联合约束的 3D 姿态/接触生成任务。
原文依据 integrates an affordance-aware latent representation of hand poses with a dual-conditioning diffusion process
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论生成既能准确反映物体几何、又能体现用户指定交互语义的人手抓取姿态,对于 AR/VR 与具身智能中自然的手物交互至关重要。然而,现有的语义抓取方法难以应对三维物体表示与文本指令之间的巨大模态鸿沟,且往往缺乏明确的空间或语义约束,导致抓取在物理上不可行或语义上不一致。本文提出 AffordGrasp,一个基于扩散的框架,能够高精度地生成物理稳定且语义忠实的人手抓取。我们首先引入一套可扩展的标注流水线,自动为手物交互数据集补充捕捉交互意图的细粒度结构化语言标签。在这些标注的基础上,AffordGrasp 将可供性感知的手部姿态潜变量表示与双条件扩散过程相结合,使模型能够对物体几何、空间可供性与指令语义进行联合推理。一个分布调整模块进一步强化物理接触一致性与语义对齐。我们在由 HO-3D、OakInk、GRAB 与 AffordPose 衍生的四个指令增强基准上评估 AffordGrasp,观察到其在抓取质量、语义准确性与多样性上相较最优方法均有显著提升。
英文原文摘要 1409 字符
Generating human grasping poses that accurately reflect both object geometry and user-specified interaction semantics is essential for natural hand-object interactions in AR/VR and embodied AI. However, existing semantic grasping approaches struggle with the large modality gap between 3D object representations and textual instructions, and often lack explicit spatial or semantic constraints, leading to physically invalid or semantically inconsistent grasps. In this work, we present AffordGrasp, a diffusion-based framework that produces physically stable and semantically faithful human grasps with high precision. We first introduce a scalable annotation pipeline that automatically enriches hand-object interaction datasets with fine-grained structured language labels capturing interaction intent. Building upon these annotations, AffordGrasp integrates an affordance-aware latent representation of hand poses with a dual-conditioning diffusion process, enabling the model to jointly reason over object geometry, spatial affordances, and instruction semantics. A distribution adjustment module further enforces physical contact consistency and semantic alignment. We evaluate AffordGrasp across four instruction-augmented benchmarks derived from HO-3D, OakInk, GRAB, and AffordPose, and observe substantial improvements over state-of-the-art methods in grasp quality, semantic accuracy, and diversity.