LumiMotion:利用场景动态改进高斯重光照
LumiMotion: Improving Gaussian Relighting with Scene Dynamics
把场景中的运动区域当作逆渲染的监督信号:同一表面在不同光照下的观测有助于解耦反照率与光照,这一思路可直接迁移到动态场景的重建、重光照与材质编辑;同时其静态/动态分离的形变约束与配套的合成基准,为评估动态环境下的逆渲染提供了可复用的设定。
原文依据 Motion reveals the same sur- faces under varying lighting conditions, providing stronger cues for disentangling material and illumination.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论在三维重建中,逆渲染问题,即恢复场景光照与材质属性,是基础性问题。现有的基于高斯泼溅的方法主要面向静态场景,且往往假设简化或适中的光照,以避免阴影与表面外观相互纠缠。这限制了它们在真实条件下将光照效果与材质属性准确分离的能力。我们通过利用动态元素——场景中发生运动的区域——作为逆渲染的监督信号来突破这一限制。运动使同一表面在不同光照条件下被观测到,从而为解耦材质与光照提供了更强的线索。实验结果支持这一论点:相对于次优基线,我们将反照率估计的感知相似度提升了 23%,将场景重光照提升了 15%。为此,我们提出 LumiMotion,这是首个利用动态信息进行逆渲染、并可在任意动态场景中运行的高斯方法。我们的方法学习一种动态二维高斯泼溅表示,并采用一组新颖的约束,促使场景的动态区域发生形变,同时保持静态区域稳定。正如我们所展示的,这种分离对于反照率的正确优化至关重要。最后,我们发布了一个新的合成基准,包含四种光照条件下的五个场景,每个场景均有静态与动态两个版本,首次使得在动态环境和具有挑战性的光照下对逆渲染方法进行系统评测成为可能。
英文原文摘要 1646 字符
In 3D reconstruction, the problem of inverse rendering, namely recovering the illumination of the scene and the material properties, is fundamental. Existing Gaussian Splatting-based methods primarily target static scenes and often assume simplified or moderate lighting to avoid entan- gling shadows with surface appearance. This limits their ability to accurately separate lighting effects from mate- rial properties, particularly in real-world conditions. We address this limitation by leveraging dynamic elements-- regions of the scene that undergo motion--as a supervisory signal for inverse rendering. Motion reveals the same sur- faces under varying lighting conditions, providing stronger cues for disentangling material and illumination. This the- sis is supported by our experimental results which show we improve LPIPS by 23% for albedo estimation and by 15% for scene relighting relative to next-best baseline. To this end, we introduce LumiMotion, the first Gaussian-based approach that leverages dynamics for inverse rendering and operates in arbitrary dynamic scenes. Our method learns a dynamic 2D Gaussian Splatting representation that em- ploys a set of novel constraints which encourage the dy- namic regions of the scene to deform, while keeping static regions stable. As we demonstrate, this separation is crucial for correct optimization of the albedo. Finally, we release a new synthetic benchmark comprising five scenes under four lighting conditions, each in both static and dynamic variants, for the first time enabling systematic evaluation of inverse rendering methods in dynamic environments and challenging lighting.