2D-LFM:无需三维监督的提升基础模型
2D-LFM: Lifting Foundation Model without 3D Supervision
把经典 structure-from-motion 的对应关系结构以位置编码形式注入模型,使二维关键点提升为物体级三维几何,可作为无需三维标注的三维先验或伪标签来源,用于关键点/骨架驱动的三维重建与姿态估计。
原文依据 inject correspondence structure into the model via a positional encoding inspired by classical structure-from-motion
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近期的视觉基础模型给人一种印象:从 RGB 图像进行三维重建的问题已基本解决。然而,这些系统在物体特定的三维结构上表现不佳,即由物体的关键点或骨架所隐含的细粒度几何。本文表明,当模型仅获得二维关键点时,其恢复出的三维结构比最优的 RGB 深度基础模型更准确。PAUL 等经典提升方法展示了这一原理,但无法扩展到单一类别之外;而 3D-LFM 等方法虽可扩展,却需要大量三维监督。我们提出首个仅使用二维监督即可学习物体特定三维几何的提升基础模型。其核心思想是通过受经典 structure-from-motion 启发的位置编码,将对应关系结构注入模型。这一简单的归纳偏置使稳健、与物体无关的三维提升成为可能,其效果可与近期依赖三维监督的方法相媲美甚至超越,从而揭示出基于关键点的提升仍是三维理解中强大且尚未被充分利用的范式。
英文原文摘要 1069 字符
Recent vision foundation models give the impression that 3D reconstruction from RGB is largely solved. Yet these systems struggle with object-specific 3D structure: the fine-grained geometry implied by an object's landmarks or skeleton. In this paper, we show that when a model is given only 2D landmarks, it can recover more accurate 3D structure than state-of-the-art depth-from-RGB foundation models. Classical lifting approaches such as PAUL demonstrate this principle but do not scale beyond single categories, while methods like 3D-LFM scale but require extensive 3D supervision. We present the first lifting foundation model that learns object-specific 3D geometry using only 2D supervision. The key idea is to inject correspondence structure into the model via a positional encoding inspired by classical structure-from-motion. This simple inductive bias enables robust, object-agnostic 3D lifting that rivals or exceeds recent 3D-supervised approaches, revealing that landmark-based lifting remains a powerful and under-exploited paradigm for 3D understanding.