Selfi：通过三维几何特征对齐的自改进重建引擎
Selfi: Self-improving Reconstruction Engine via 3D Geometric Feature Alignment
通过重投影一致性损失将 VGGT 特征蒸馏到几何对齐的特征空间，可作为一种通用的自监督特征对齐手段，提升多视图几何一致性，直接用于新视角合成和相机姿态估计。
原文依据 we train a lightweight feature adapter using a reprojection-based consistency loss, which distills VGGT outputs into a new geometrically-aligned feature space
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论新视角合成传统上依赖于具有显式三维归纳偏置的模型，并结合事先通过运动恢复结构（SfM）获得的已知相机参数。近期如 VGGT 等视觉基础模型采取了一种正交的方法——三维知识通过训练数据和损失目标隐式获得，能够直接从一组未标定图像中前馈预测相机参数和三维表示。虽然灵活，但 VGGT 特征缺乏显式的多视图几何一致性，我们发现改善这种三维特征一致性有益于新视角合成和姿态估计任务。我们提出 Selfi，一种通过特征对齐的自改进三维重建流程，利用 VGGT 骨干网络自身的输出作为伪真值，将其转变为高保真三维重建引擎。具体而言，我们使用基于重投影的一致性损失训练一个轻量级特征适配器，将 VGGT 输出蒸馏到一个新的几何对齐特征空间，该空间能够捕获三维空间邻近性。这使得在新视角合成和相机姿态估计两方面均达到最优性能，表明特征对齐是下游三维推理中极为有益的步骤。更多细节见项目页面：https://denghilbert.github.io/selfi
英文原文摘要 1306 字符
Novel View Synthesis (NVS) has traditionally relied on models with explicit 3D inductive biases combined with known camera parameters from Structure-from-Motion (SfM) beforehand. Recent vision foundation models like VGGT take an orthogonal approach -- 3D knowledge is gained implicitly through training data and loss objectives, enabling feed-forward prediction of both camera parameters and 3D representations directly from a set of uncalibrated images. While flexible, VGGT features lack explicit multi-view geometric consistency, and we find that improving such 3D feature consistency benefits both NVS and pose estimation tasks. We introduce Selfi, a self-improving 3D reconstruction pipeline via feature alignment, transforming a VGGT backbone into a high-fidelity 3D reconstruction engine by leveraging its own outputs as pseudo-ground-truth. Specifically, we train a lightweight feature adapter using a reprojection-based consistency loss, which distills VGGT outputs into a new geometrically-aligned feature space that captures spatial proximity in 3D. This enables state-of-the-art performance in both NVS and camera pose estimation, demonstrating that feature alignment is a highly beneficial step for downstream 3D reasoning. More details on our project page: https://denghilbert.github.io/selfi