MV-RoMa:从成对匹配到多视图轨迹重建
MV-RoMa: From Pairwise Matching into Multi-View Track Reconstruction
把成对匹配结果当作几何先验,用多视图编码器加像素级注意力精化器,在不做全量交叉注意力的前提下联合估计源图到多个共视目标图的稠密对应,可直接为 SfM 提供一致的多视图轨迹,替代逐对匹配后串联的做法。
原文依据 multi-view encoder that leverages pair-wise matching results as a geometric prior
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论在图像之间建立一致的对应关系,对运动恢复结构(SfM)等三维视觉任务至关重要;然而现有匹配器大多以成对方式工作,当它们的预测跨视图串联时,往往产生碎片化且几何不一致的轨迹。我们提出 MV-RoMa,一种多视图稠密匹配模型,它联合估计从源图像到多个共视目标图像的稠密对应关系。具体而言,我们设计了一种高效的模型架构,避免了多视图特征交互中全量交叉注意力的高计算开销:(i) 多视图编码器,利用成对匹配结果作为几何先验;(ii) 多视图匹配精化器,使用像素级注意力精化对应关系。此外,我们提出一种后处理策略,将模型输出的多视图一致对应关系整合为用于 SfM 的高质量轨迹。在多样且具有挑战性的基准上,MV-RoMa 产生的对应关系比现有稀疏与稠密匹配方法更可靠,三维重建结果也显著更稠密、更准确。
英文原文摘要 1127 字符
Establishing consistent correspondences across images is essential for 3D vision tasks such as structure-from-motion (SfM), yet most existing matchers operate in a pairwise manner, often producing fragmented and geometrically inconsistent tracks when their predictions are chained across views. We propose MV-RoMa, a multi-view dense matching model that jointly estimates dense correspondences from a source image to multiple co-visible targets. Specifically, we design an efficient model architecture which avoids high computational cost of full cross-attention for multi-view feature interaction: (i) multi-view encoder that leverages pair-wise matching results as a geometric prior, and (ii) multi-view matching refiner that refines correspondences using pixel-wise attention. Additionally, we propose a post-processing strategy that integrates our model's consistent multi-view correspondences as high-quality tracks for SfM. Across diverse and challenging benchmarks, MV-RoMa produces more reliable correspondences and substantially denser, more accurate 3D reconstructions than existing sparse and dense matching methods.