曲率感知描述:利用测地注意力进行3D场景理解
Curvature-Aware Captioning: Leveraging Geodesic Attention for 3D Scene Understanding
将双曲几何中的洛伦兹空间与斜空间注意力引入3D稠密描述任务,为点云-语言跨模态对齐提供了一种保持层次语义的几何先验,可复用于其他需要建模场景实例间层级关系的视觉-语言任务。
原文依据 Bidirectional geodesic cross-attention within Lorentz space models hierarchical semantic relationships across scene instances
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论准确的3D场景描述是机器人导航和增强现实的基础,然而当前的稠密描述方法在处理稀疏点云数据时面临显著局限。现有方法采用欧几里得嵌入空间,难以同时保留细粒度的局部几何细节并建模指数增长的全局语义层次,导致定位不准确或场景描述不连贯、浅显。本文提出一种新颖的曲率感知描述框架,集成新颖的非欧几里得测地注意力机制,以解决定位与上下文建模之间的冲突。具体而言,斜空间中的自注意力在建立长程依赖的同时强制维度同质性。洛伦兹空间中的双向测地交叉注意力建模场景实例间的层次语义关系,从而同时实现物体定位的精确性和场景描述的连贯性。理论分析证实,斜流形与洛伦兹双曲面之间的曲率互补性解决了欧几里得-双曲冲突,通过各向同性优化确保特征稳定性,同时保留固有的层次关系。在ScanRefer和Nr3D基准上的大量实验证明了最优性能,在定位准确性和描述丰富性方面均取得显著提升。
英文原文摘要 1449 字符
Accurate 3D scene description is fundamental to robotic navigation and augmented reality, yet current dense captioning methods face significant limitations in processing sparse point cloud data.Existing approaches that apply Euclidean embedding spaces struggle to simultaneously preserve fine-grained local geometric details and model exponentially growing global semantic hierarchies, leading to either inaccurate localization or disjointed, shallow scene descriptions.In this work, we propose a novel Curvature-Aware Captioning framework, integrating novel non-Euclidean geodesic attention mechanisms, to resolve the localization-contextualization conflict. Specifically, self-attention within Oblique space enforces dimensional homogeneity while establishing long-range dependencies. Bidirectional geodesic cross-attention within Lorentz space models hierarchical semantic relationships across scene instances, enabling simultaneous precision in object localization and coherence in scene descriptions.Theoretical analysis confirms that the curvature complementarity between the Oblique manifold and Lorentz hyperboloid resolves the Euclidean-hyperbolic conflict, ensuring feature stability via isotropic optimization while preserving inherent hierarchical relationships.Extensive experiments on ScanRefer and Nr3D benchmarks demonstrate state-of-the-art performance, with significant gains in both localization accuracy and descriptive richness.