PhysGM:面向前馈 4D 合成的大型物理高斯模型
PhysGM: Large Physical Gaussian Model for Feed-Forward 4D Synthesis
把单图前馈预测 3D 高斯与物理参数(质量、杨氏模量等)耦合进同一网络,并用 DPO 替代 SDS 优化来对齐物理合理性,为动态 3DGS 生成提供了一条免逐场景优化、可即时仿真的路线。
原文依据 a feed-forward framework that jointly predicts 3D Gaussian representation and physical properties from a single image
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论尽管基于物理的三维运动合成已取得进展,现有方法仍存在关键局限:依赖由稠密多视图图像重建、需逐场景耗时优化得到的 3D 高斯泼溅(3DGS);物理集成要么依赖不灵活的人工指定属性,要么依赖视频模型通过分数蒸馏采样(SDS)给出的不稳定且优化开销大的引导;以及将预构建的 3DGS 与物理模块简单拼接,忽略了外观中蕴含的物理信息,导致性能欠佳。为解决这些问题,我们提出 PhysGM,一个前馈框架,可从单张图像联合预测三维高斯表示与物理属性,从而实现即时仿真与高保真 4D 渲染。与缓慢且与外观无关的优化方法不同,我们首先预训练一个物理感知重建模型,直接推断高斯参数与物理参数。我们进一步用直接偏好优化(DPO)精调该模型,使仿真结果与物理上合理的参考视频对齐,从而避免高成本的 SDS 优化。针对该任务缺乏配套数据集的问题,我们提出 PhysAssets,一个包含 5 万余个三维资产的数据集,标注了物理属性及对应的参考视频。实验表明,PhysGM 能在一分钟内从单张图像生成高保真 4D 仿真,相较先前工作实现显著加速,同时给出逼真的渲染结果。
英文原文摘要 1507 字符
Despite advances in physics-based 3D motion synthesis, current methods face key limitations: reliance on pre-reconstructed 3D Gaussian Splatting (3DGS) built from dense multi-view images with time-consuming per-scene optimization; physics integration via either inflexible, hand-specified attributes or unstable, optimization-heavy guidance from video models using Score Distillation Sampling (SDS); and naive concatenation of prebuilt 3DGS with physics modules, which ignores physical information embedded in appearance and yields suboptimal performance. To address these issues, we propose PhysGM, a feed-forward framework that jointly predicts 3D Gaussian representation and physical properties from a single image, enabling immediate simulation and high-fidelity 4D rendering. Unlike slow appearance-agnostic optimization methods, we first pre-train a physics-aware reconstruction model that directly infers both Gaussian and physical parameters. We further refine the model with Direct Preference Optimization (DPO), aligning simulations with the physically plausible reference videos and avoiding the high-cost SDS optimization. To address the absence of a supporting dataset for this task, we propose PhysAssets, a dataset of 50K+ 3D assets annotated with physical properties and corresponding reference videos. Experiments show that PhysGM produces high-fidelity 4D simulations from a single image in one minute, achieving a significant speedup over prior work while delivering realistic renderings.