视频生成模型的推理时物理对齐:基于潜变量世界模型
Inference-time Physics Alignment of Video Generative Models with Latent World Models
把潜变量世界模型(VJEPA-2)当作奖励,在推理阶段搜索并引导多条去噪轨迹,为视频生成提供了一种无需重训练、可扩展测试时计算的物理合理性对齐手段,可直接迁移到其他生成模型的后处理与采样策略中。
原文依据 leverage the strong physics prior of a latent world model (here, VJEPA-2) as a reward to search and steer multiple candidate denoising trajectories
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论最优的视频生成模型能产生颇具前景的视觉内容,却常常违背基本物理规律,限制了其可用性。部分研究将这一缺陷归因于预训练阶段物理理解的不足,而我们发现,物理合理性的欠缺同样源于次优的推理策略。为此,我们提出 WMReward,将提升视频生成的物理合理性视为一个推理时对齐问题。具体而言,我们利用潜变量世界模型(此处为 VJEPA-2)强大的物理先验作为奖励,搜索并引导多条候选去噪轨迹,从而通过扩展测试时计算来获得更好的生成性能。在实验中,我们的方法在图像条件、多帧条件与文本条件生成设定下均显著提升了物理合理性,并得到人类偏好研究的验证。值得注意的是,在 ICCV 2025 Perception Test PhysicsIQ 挑战赛中,我们取得 62.64% 的最终得分,获得第一名,并超越此前最优结果 7.42%。我们的工作表明,利用潜变量世界模型提升视频生成的物理合理性是可行的,且不局限于这一具体实现或参数化形式。
英文原文摘要 1263 字符
State-of-the-art video generative models produce promising visual content yet often violate basic physics principles, limiting their utility. While some attribute this deficiency to insufficient physics understanding from pre-training, we find that the shortfall in physics plausibility also stems from suboptimal inference strategies. We therefore introduce WMReward and treat improving physics plausibility of video generation as an inference-time alignment problem. In particular, we leverage the strong physics prior of a latent world model (here, VJEPA-2) as a reward to search and steer multiple candidate denoising trajectories, enabling scaling test-time compute for better generation performance. Empirically, our approach substantially improves physics plausibility across image-conditioned, multiframe-conditioned, and text-conditioned generation settings, with validation from human preference study. Notably, in the ICCV 2025 Perception Test PhysicsIQ Challenge, we achieve a final score of 62.64%, winning first place and outperforming the previous state of the art by 7.42%. Our work demonstrates the viability of using latent world models to improve physics plausibility of video generation, beyond this specific instantiation or parameterization.