让雪落下!基于物理引导分数蒸馏为 3D 高斯场景添加动态天气效果
Let it Snow! Animating 3D Gaussian Scenes with Dynamic Weather Effects via Physics-Guided Score Distillation
把物理仿真当作运动先验去引导 Video-SDS,为静态 3DGS 场景注入物理合理的多粒子动态(雪、雨、雾、沙尘),这套「物理引导 + 分数蒸馏联合优化运动与外观」的框架可直接迁移到 3D 场景动态化、流体/粒子特效生成等任务。
原文依据 physics simulation guides Score Distillation to jointly refine the motion prior for photorealism while simultaneously optimizing appearance
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论3D 高斯泼溅近来使静态三维场景的快速、照片级真实重建成为可能。然而,对此类场景的动态编辑仍是一项重大挑战。我们提出一个新框架——物理引导分数蒸馏,用以解决一个根本性冲突:物理仿真提供了强运动先验,但不足以达到照片级真实感;而仅依赖基于视频的分数蒸馏采样(SDS)又无法为复杂的多粒子场景生成连贯运动。我们通过一个统一的优化框架来解决该问题:由物理仿真引导分数蒸馏,在优化外观的同时联合精炼运动先验以实现照片级真实感。具体而言,我们学习一个预测粒子运动与外观的神经动态模型,并通过一个结合 Video-SDS(用于照片级真实感)与我们的物理引导先验的联合损失进行端到端优化。这既允许照片级真实的细化,又确保动态保持合理。我们的框架支持场景级的动态天气效果,包括降雪、降雨、雾和沙尘暴,且运动在物理上合理。实验表明,我们的物理引导方法显著优于基线,消融实验证实这种联合精炼对于生成连贯、高保真的动态至关重要。
英文原文摘要 1337 字符
3D Gaussian Splatting has recently enabled fast and photorealistic reconstruction of static 3D scenes. However, dynamic editing of such scenes remains a significant challenge. We introduce a novel framework, Physics-Guided Score Distillation, to address a fundamental conflict: physics simulation provides a strong motion prior that is insufficient for photorealism , while video-based Score Distillation Sampling (SDS) alone cannot generate coherent motion for complex, multi-particle scenarios. We resolve this through a unified optimization framework where physics simulation guides Score Distillation to jointly refine the motion prior for photorealism while simultaneously optimizing appearance. Specifically, we learn a neural dynamics model that predicts particle motion and appearance, optimized end-to-end via a combined loss integrating Video-SDS for photorealism with our physics-guidance prior. This allows for photorealistic refinements while ensuring the dynamics remain plausible. Our framework enables scene-wide dynamic weather effects, including snowfall, rainfall, fog, and sandstorms, with physically plausible motion. Experiments demonstrate our physics-guided approach significantly outperforms baselines, with ablations confirming this joint refinement is essential for generating coherent, high-fidelity dynamics.