QueryMe：查询驱动的多模态证据开放词表3D物体功能可供性定位
QueryMe: Query-Driven Open-Vocabulary 3D Object Affordances Grounding from Multimodal Evidence
将2D人-物交互图像投影到3D空间，并用多模态查询结构在点云中检索几何一致的功能部件，为开放词表3D功能可供性定位提供了可复用的跨模态对齐与查询机制。
原文依据 project human-object interaction images into 3D space, employ an Adaptive Spatial Attention module to focus on key interaction regions
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论开放词表3D物体功能可供性定位旨在根据任意语义描述识别物体的功能区域。然而，现有方法通常依赖固定的训练类别和几何先验，缺乏几何不变性和类比推理能力。由于从2D图像迁移功能可供性知识到3D点云时存在显著的域间差距，现有方法难以泛化到具有多样形状或未见类别的物体，也无法进行有效的类别推理。为解决这些挑战，我们提出了QueryMe，一个查询驱动的框架，从多模态证据空间中学习以实现开放词表3D功能可供性定位。所提方法将人-物交互图像投影到3D空间，采用自适应空间注意力模块聚焦关键交互区域，并引入多模态查询结构在点云中检索可用的几何一致功能部件，有效融合视觉、语言和几何线索。借助基于注意力的查询机制，我们的方法自适应地定位功能可供性区域，并通过几何相似性进行类比推理，从而对未见场景和物体展现出强泛化能力。实验结果表明，QueryMe持续优于最优方法，在未见功能可供性定位任务上AUC相比先前工作提升了4.19%。
英文原文摘要 1524 字符
Open-vocabulary 3D object affordance grounding aims to identify functional regions of objects given arbitrary semantic descriptions. However, existing methods often rely on fixed training categories and geometric priors, lacking geometric invariance and analogical reasoning capabilities. Since there exists a significant domain gap when transferring affordance knowledge learned from 2D images to 3D point clouds, existing methods struggle to generalize well to objects with diverse shapes or unseen categories, and fail to perform effective category reasoning.To address these challenges, we propose **QueryMe**, a **Query**-driven framework that learns from **M**ultimodal **e**vidence spaces to achieve open-vocabulary 3D affordance grounding.The proposed approach is to project human-object interaction images into 3D space, employ an Adaptive Spatial Attention module to focus on key interaction regions, and introduce a multimodal query structure to retrieve available geometrically consistent functional parts within the point cloud, effectively fusing visual, linguistic, and geometric cues.Leveraging attention-based query mechanisms, our method adaptively localizes affordance regions and performs analogy reasoning through geometric similarity, thereby exhibiting strong generalization to unseen scenes and objects. Experimental results demonstrate that QueryMe consistently outperforms state-of-the-art approaches, with the AUC improving by 4.19% compared to previous work for unseen affordance grounding tasks.