TALO:推动 3D 视觉基础模型走向全局一致的在线重建
TALO: Pushing 3D Vision Foundation Models Towards Globally Consistent Online Reconstruction
把 3D 基础模型逐窗口的前馈预测用薄板样条(TPS)做高自由度长程对齐,并用点无关的子图配准提升对噪声几何的鲁棒性;做在线/流式 3D 重建或需要跨时序窗口融合深度与位姿时可直接复用这套即插即用框架。
原文依据 we propose a higher-DOF and long-term alignment framework based on Thin Plate Spline, leveraging globally propagated control points
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论3D 视觉基础模型已展现出强大的泛化能力,能够通过单次前馈从无标定图像中重建关键的 3D 属性。然而,当部署在驾驶场景等在线环境中时,预测是在时序窗口上进行的,因此跨时间保持一致性并非易事。近期的策略通过求解全局变换来对齐相邻预测,但我们的分析揭示了它们在假设有效性、局部对齐范围以及噪声几何下鲁棒性方面的根本局限。在本工作中,我们提出了一种基于薄板样条的高自由度、长程对齐框架,利用全局传播的控制点来校正空间变化的不一致性。此外,我们采用了一种与点无关的子图配准设计,该设计对噪声几何预测具有内在鲁棒性。所提框架完全即插即用,兼容多种 3D 基础模型与相机配置(例如单目或环视)。大量实验表明,我们的方法在多个数据集、骨干网络模型和相机设置下始终产生更连贯的几何与更低的轨迹误差,凸显了其鲁棒性与泛化能力。代码见 https://github.com/Xian-Bei/TALO。
英文原文摘要 1301 字符
3D vision foundation models have shown strong generalization in reconstructing key 3D attributes from uncalibrated images through a single feed-forward pass. However, when deployed in online settings such as driving scenarios, predictions are made over temporal windows, making it non-trivial to maintain consistency across time. Recent strategies align consecutive predictions by solving global transformation, yet our analysis reveals their fundamental limitations in assumption validity, local alignment scope, and robustness under noisy geometry. In this work, we propose a higher-DOF and long-term alignment framework based on Thin Plate Spline, leveraging globally propagated control points to correct spatially varying inconsistencies. In addition, we adopt a point-agnostic submap registration design that is inherently robust to noisy geometry predictions. The proposed framework is fully plug-and-play, compatible with diverse 3D foundation models and camera configurations (e.g., monocular or surround-view). Extensive experiments demonstrate that our method consistently yields more coherent geometry and lower trajectory errors across multiple datasets, backbone models, and camera setups, highlighting its robustness and generality. Code is available at https://github.com/Xian-Bei/TALO.