运动中的光照:时空 HDR 光照估计
Lighting in Motion: Spatiotemporal HDR Lighting Estimation
用扩散先验从单目视频/图像序列回归时空变化的光探针(镜面球+漫反射球多曝光),再经可微渲染合成 HDRI 环境图;其「深度不足以做空间条件、需显式相对位置几何条件」的结论对做光照估计、逆渲染与场景重光照的视觉研究者可直接复用。
原文依据 we demonstrate that depth alone is insufficient and we introduce a new geometric condition to provide the relative position of the scene to the target 3D position
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论本文提出运动中的光照(LiMo),一种基于扩散的时空光照估计方法。LiMo 同时面向真实的高频细节预测与准确的照度估计。为兼顾二者,我们提出依据目标在输入中的三维位置,生成一组不同曝光下的镜面球与漫反射球。借助扩散先验,我们在一个大规模定制的室内外场景数据集上微调了强大的现有扩散模型,该数据集配有时空光探针。为实现准确的空间条件化,我们证明仅凭深度并不充分,并引入一种新的几何条件,用以提供场景相对于目标三维位置的相对位置。最后,我们利用可微渲染将不同曝光下的漫反射与镜面预测合成为单一 HDRI 图。我们对该方法及其设计选择进行了充分评估,确立 LiMo 在空间控制与预测精度两方面均达到最优。
英文原文摘要 1008 字符
We present Lighting in Motion (LiMo), a diffusion-based approach to spatiotemporal lighting estimation. LiMo targets both realistic high-frequency detail prediction and accurate illuminance estimation. To account for both, we propose generating a set of mirrored and diffuse spheres at different exposures, based on their 3D positions in the input. Making use of diffusion priors, we fine-tune powerful existing diffusion models on a large-scale customized dataset of indoor and outdoor scenes, paired with spatiotemporal light probes. For accurate spatial conditioning, we demonstrate that depth alone is insufficient and we introduce a new geometric condition to provide the relative position of the scene to the target 3D position. Finally, we combine diffuse and mirror predictions at different exposures into a single HDRI map leveraging differentiable rendering.We thoroughly evaluate our method and design choices to establish LiMo as state-of-the-art for both spatial control and prediction accuracy.