SAG-GNN：面向无描述子 2D-3D 匹配的语义感知引导图神经网络
SAG-GNN: Semantic-Aware Guided GNN for Descriptor-Free 2D-3D Matching
把低维语义概率分布作为 3D 点的紧凑先验注入无描述子 2D-3D 匹配，用语义消歧替代大描述子库，可直接迁移到无描述子视觉定位与位姿估计流程中降低存储并提升跨场景泛化。
原文依据 we propose the Semantic-Aware Guided Graph Neural Network (SAG-GNN), integrating high-level semantics into descriptor-free 2D-3D matching
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论图像到点云的匹配（2D-3D 匹配）在图像关键点与 3D 点之间建立精确对应关系，用于 6-DoF 相机位姿估计。现有方法要么因依赖场景特定的坐标回归而需要逐场景重训练，导致泛化性差；要么因基于描述子的匹配依赖大规模描述子集合，带来高昂的存储与维护成本。因此，无描述子方法因避免沉重存储并提升泛化性而受到关注；然而，大多数此类方法仅依赖低层几何线索，限制了性能。鉴于语义在提供上下文、消解歧义以及增强挑战性场景下的鲁棒性方面的优势，我们提出语义感知引导图神经网络（SAG-GNN），将高层语义融入无描述子 2D-3D 匹配。具体而言，我们设计了一种紧凑的语义提取方案，将每个 3D 点编码为低维语义概率分布，以极小的存储代价提供有效引导。双向对齐融合模块将几何特征与语义上下文融合，得到更统一、更一致的表示。此外，语义先验从高层语义视角引导交互框架内的 2D-3D 信息交换。大量室内与室外实验验证，SAG-GNN 在无描述子 2D-3D 匹配与视觉定位上取得最优结果，且存储开销低、泛化性强。代码见 https://github.com/tinxu0203/SAG-GNN。
英文原文摘要 1596 字符
Image-to-point cloud matching (2D-3D matching) establishes accurate correspondences between image keypoints and 3D points for 6-DoF camera pose estimation. Existing methods either suffer from poor generalization due to scene-specific coordinate regression requiring per-scene retraining, or incur high storage and maintenance costs from descriptor-based matching that relies on large descriptor sets. Consequently, descriptor-free approaches have gained attention by avoiding heavy storage while improving generalizability; however, most rely only on low-level geometric cues, which limits performance. Leveraging the benefits of semantics in providing context, resolving ambiguities, and enhancing robustness in challenging scenes, we propose the Semantic-Aware Guided Graph Neural Network (SAG-GNN), integrating high-level semantics into descriptor-free 2D-3D matching. Specifically, we design a compact semantic extraction scheme encoding each 3D point as a low-dimensional semantic probability distribution, offering effective guidance with minimal storage. A bidirectionally-aligned fusion block merges geometric features with semantic context for more unified and consistent representations. Additionally, semantic priors guide the 2D-3D information exchange within the interaction framework from a high-level semantic perspective. Extensive indoor and outdoor experiments validate that SAG-GNN achieves state-of-the-art results in descriptor-free 2D-3D matching and visual localization, with low storage and strong generalization. Code is available at https://github.com/tinxu0203/SAG-GNN.