3D 视觉与重建 · CVPR2026 ·2026.06.03

UniCorrn:跨 2D 与 3D 的统一对应 Transformer

UniCorrn: Unified Correspondence Transformer Across 2D and 3D

Prajnan Goswami、Tianye Ding、Feng Liu、Huaizu Jiang

与你有关的部分

用共享权重的双流解码器(外观流+位置流)统一 2D-2D、2D-3D、3D-3D 匹配,可直接复用到跨模态特征匹配与点云配准的特征融合与查询式对应估计环节。

原文依据 We propose a dual-stream decoder that maintains separate appearance and positional feature streams.
可迁移到 image matchingpoint cloud registration2D-3D matchingfeature correspondence

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

图像到图像(2D-2D)、图像到点云(2D-3D)以及点云到点云(3D-3D)几何匹配中的视觉对应关系构成了众多三维视觉任务的基础。尽管这些问题结构相似,现有方法却采用针对具体任务的设计,并为每种模态组合使用独立模型。我们提出 UniCorn,这是首个在所有三项任务上以共享权重统一几何匹配的对应模型。我们的关键洞见是 Transformer 注意力天然能够捕捉跨模态特征相似性。我们提出一种双流解码器,分别维护外观特征流与位置特征流。该设计通过可堆叠层实现端到端学习,同时支持跨异构模态的灵活查询式对应估计。我们的架构采用模态特定骨干网络,后接共享的编码器与解码器组件,并在融合深度图生成的伪点云与真实三维对应标注的多样数据上联合训练。UniCorn 在 2D-2D 匹配上取得有竞争力的性能,并在配准召回率上于 7Scenes(2D-3D)超越此前最优方法 8%、于 3DLoMatch(3D-3D)超越 10%。代码见 https://github.com/neu-vi/UniCorrn 。

英文原文摘要 1304 字符

Visual correspondence across image-to-image (2D-2D), image-to-point cloud (2D-3D), and point cloud-to-point cloud (3D-3D) geometric matching forms the foundation for numerous 3D vision tasks. Despite sharing a similar problem structure, current methods use task-specific designs with separate models for each modality combination. We present UniCorn, the first correspondence model with shared weights that unifies geometric matching across all three tasks. Our key insight is that Transformer attention naturally captures cross-modal feature similarity. We propose a dual-stream decoder that maintains separate appearance and positional feature streams. This design enables end-to-end learning through stack-able layers while supporting flexible query-based correspondence estimation across heterogeneous modalities. Our architecture employs modality-specific backbones followed by shared encoder and decoder components, trained jointly on diverse data combining pseudo point clouds from depth maps with real 3D correspondence annotations. UniCorn achieves competitive performance on 2D-2D matching and surpasses prior state-of-the-art by 8% on 7Scenes (2D-3D) and 10% on 3DLoMatch (3D-3D) in registration recall. Code is provided at \href https://github.com/neu-vi/UniCorrn github.com/neu-vi/UniCorrn .

标签

会议 CVPR2026
页码 9943-9954
Bibkey
来源 CVF Open Access