Dark3R:在黑暗中学习运动恢复结构
Dark3R: Learning Structure from Motion in the Dark
将大规模3D基础模型通过教师-学生蒸馏适配到极低光照原始图像,实现鲁棒的特征匹配与相机姿态估计,可直接用于暗光下的运动恢复结构和新视角合成。
原文依据 adapt large-scale 3D foundation models to extreme low-light conditions through a teacher-student distillation process
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论我们提出 Dark3R,一个直接在信噪比(SNR)低于 -4 dB 的原始图像上运行的运动恢复结构框架——这一信噪比区间是传统基于特征和基于学习的方法失效的区域。我们的关键见解是通过教师-学生蒸馏过程将大规模 3D 基础模型适配到极端低光照条件下,从而在低光照下实现鲁棒的特征匹配和相机姿态估计。Dark3R 不需要 3D 监督;它仅使用噪声-干净原始图像对进行训练,这些图像对可以直接拍摄,或通过对曝光良好的原始图像应用简单的泊松-高斯噪声模型合成。为了训练和评估我们的方法,我们引入了一个新的曝光包围数据集,包含 42,000 张多视角原始图像及真实 3D 标注,并证明 Dark3R 在低信噪比区间实现了最优的运动恢复结构。此外,我们使用 Dark3R 预测的姿态和由粗到细的辐射场优化过程,展示了黑暗中最优的新视角合成。
英文原文摘要 1094 字符
We introduce Dark3R, a framework for structure from motion in the dark that operates directly on raw images with signal-to-noise ratios (SNRs) below -4 dB--a regime where conventional feature- and learning-based methods break down. Our key insight is to adapt large-scale 3D foundation models to extreme low-light conditions through a teacher-student distillation process, enabling robust feature matching and camera pose estimation in low light. Dark3R requires no 3D supervision; it is trained solely on noisy--clean raw image pairs, which can be either captured directly or synthesized using a simple Poisson-Gaussian noise model applied to well-exposed raw images.To train and evaluate our approach, we introduce a new, exposure-bracketed dataset that includes 42,000 multi-view raw images with ground-truth 3D annotations, and we demonstrate that Dark3R achieves state-of-the-art structure from motion in the low-SNR regime. Further, we demonstrate state-of-the-art novel view synthesis in the dark using Dark3R's predicted poses and a coarse-to-fine radiance field optimization procedure.