3D 基础模型中涌现的极端视角几何
Emergent Extreme-View Geometry in 3D Foundation Models
论文揭示 3D 基础模型在非重叠极端视角下已具备涌现的几何理解能力,并给出只微调骨干网络 bias 项、冻结解码器头的轻量对齐方案,可直接用于提升相对姿态估计;同时 MegaUnScene 基准为评测 3DFM 的跨场景泛化提供了新协议。
原文依据 we introduce a lightweight alignment scheme that refines their internal 3D representation by tuning only a small subset of backbone bias terms
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论3D 基础模型(3DFMs)近来改变了三维视觉,使直接从图像联合预测深度、姿态与点图成为可能。然而,它们在极端、非重叠视角下进行推理的能力仍基本未被探索。在本工作中,我们研究了它们的内部表征,发现 3DFMs 表现出对极端视角几何的涌现理解,尽管从未针对此类条件进行训练。为进一步增强这些能力,我们引入一种轻量对齐方案,仅通过调优骨干网络中一小部分偏置项来精炼其内部三维表征,同时保持所有解码器头冻结。这种有针对性的适配大幅提升了极端视角下的相对姿态估计,且不降低单图像深度或点的质量。此外,我们贡献了 MegaUnScene,这是一个由现有 3DFMs 未见过的互联网场景构成的新基准,包含分别用于相对姿态估计和稠密三维重建的专用测试划分。
英文原文摘要 981 字符
3D foundation models (3DFMs) have recently transformed 3D vision, enabling joint prediction of depths, poses, and point maps directly from images. Yet their ability to reason under extreme, non-overlapping views remains largely unexplored. In this work, we study their internal representations and find that 3DFMs exhibit an emergent understanding of extreme-view geometry, despite never being trained for such conditions. To further enhance these capabilities, we introduce a lightweight alignment scheme that refines their internal 3D representation by tuning only a small subset of backbone bias terms, leaving all decoder heads frozen. This targeted adaptation substantially improves relative pose estimation under extreme viewpoints without degrading per-image depth or point quality. Additionally, we contribute MegaUnScene, a new benchmark of Internet scenes unseen by existing 3DFMs, with dedicated test splits for both relative pose estimation and dense 3D reconstruction.