Depth Any Panoramas:面向全景深度估计的基础模型
Depth Any Panoramas: A Foundation Model for Panoramic Depth Estimation
以 DINOv3-Large 为骨干、配合可插拔 range mask head 与几何一致性优化,构建跨室内外、跨合成/真实的全景度量深度基础模型;其三级伪标签清洗流程与 UE5+文生图合成数据构造方式,可直接迁移到其他缺乏真值的深度/几何任务。
原文依据 we introduce a three-stage pseudo-label curation pipeline to generate reliable ground truth for unlabeled images
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论本文提出一个全景度量深度基础模型,可在多样化的场景距离下泛化。我们从数据构建与框架设计两方面探索了数据在环(data-in-the-loop)范式。我们构建了一个大规模数据集,融合了公开数据集、来自自研 UE5 模拟器与文生图模型的高质量合成数据,以及来自网络的真实全景图像。为缩小室内/室外以及合成/真实数据之间的域差异,我们引入一个三阶段伪标签清洗流程,为无标注图像生成可靠的真值。在模型方面,我们采用 DINOv3-Large 作为骨干网络,以利用其强大的预训练泛化能力,并引入可插拔的距离掩码头(range mask head)、以清晰度为中心的优化以及以几何为中心的优化,以提升对不同距离的鲁棒性并强化跨视角的几何一致性。在多个基准(如 Stanford2D3D、Matterport3D 和 Deep360)上的实验表明,模型具有强劲的性能与零样本泛化能力,在多样化的真实场景中尤其能给出鲁棒且稳定的度量预测。项目主页见:https://insta360-research-team.github.io/DAP_website/
英文原文摘要 1197 字符
In this work, we present a panoramic metric depth foundation model that generalizes across diverse scene distances. We explore a data-in-the-loop paradigm from the view of both data construction and framework design. We collect a large-scale dataset by combining public datasets, high-quality synthetic data from our UE5 simulator and text-to-image models, and real panoramic images from the web. To reduce domain gaps between indoor/outdoor and synthetic/real data, we introduce a three-stage pseudo-label curation pipeline to generate reliable ground truth for unlabeled images. For the model, we adopt DINOv3-Large as the backbone for its strong pre-trained generalization, and introduce a plug-and-play range mask head, sharpness-centric optimization, and geometry-centric optimization to improve robustness to varying distances and enforce geometric consistency across views. Experiments on multiple benchmarks (e.g., Stanford2D3D, Matterport3D, and Deep360) demonstrate strong performance and zero-shot generalization, with particularly robust and stable metric predictions in diverse real-world scenes. The project page can be found at: https://insta360-research-team.github.io/DAP_website/