Iris:将真实世界先验引入扩散模型用于单目深度估计
Iris: Bringing Real-World Priors into Diffusion Model for Monocular Depth Estimation
把扩散模型的生成先验按频段拆开使用:低频用真实数据蒸馏、高频用合成真值约束,并通过高到低的时间步调度共享权重,这套「先验-几何」两阶段确定性采样思路可直接迁移到其他以扩散先验做稠密回归的视觉任务(如表面法向、光流)。
原文依据 the prior stage uses Spectral-Gated Distillation (SGD) to transfer low-frequency real priors while leaving high-frequency details unconstrained
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论本文提出 Iris,一个将真实世界先验融入扩散模型的确定性单目深度估计(MDE)框架。传统前馈方法依赖海量训练数据,却仍会丢失细节。此前的基于扩散的方法利用了丰富的生成先验,却难以完成合成到真实的域迁移。相比之下,Iris 能保留精细细节、从合成场景到真实场景具有很强的泛化能力,并且在训练数据有限时仍保持高效。为此,我们引入两阶段的「先验到几何确定性」(PGD)调度:先验阶段使用频谱门控蒸馏(SGD)迁移低频真实先验,同时不对高频细节施加约束;几何阶段应用频谱门控一致性(SGC),在利用合成真值进行精化的同时强制高频保真。两个阶段共享权重,并以从高到低的时间步调度执行。大量实验结果证实,Iris 在单目深度估计性能上取得显著提升,并具有强大的野外泛化能力。
英文原文摘要 1107 字符
In this paper, we propose Iris, a deterministic framework for Monocular Depth Estimation (MDE) that integrates real-world priors into the diffusion model. Conventional feed-forward methods rely on massive training data, yet still miss details. Previous diffusion-based methods leverage rich generative priors yet struggle with synthetic-to-real domain transfer. Iris, in contrast, preserves fine details, generalizes strongly from synthetic to real scenes, and remains efficient with limited training data. To this end, we introduce a two-stage Priors-to-Geometry Deterministic (PGD) schedule: the prior stage uses Spectral-Gated Distillation (SGD) to transfer low-frequency real priors while leaving high-frequency details unconstrained, and the geometry stage applies Spectral-Gated Consistency (SGC) to enforce high-frequency fidelity while refining with synthetic ground truth. The two stages share weights and are executed with a high-to-low timestep schedule. Extensive experimental results confirm that Iris achieves significant improvements in MDE performance with strong in-the-wild generalization.