生成模型与扩散 · CVPR2026 ·2026.06.03

PhyCo:学习可控物理先验用于生成式运动

PhyCo: Learning Controllable Physical Priors for Generative Motion

Sriram Narayanan、Ziyu Jiang、Srinivasa Narasimhan、Manmohan Chandraker

与你有关的部分

用像素对齐的物理属性图(摩擦、弹性、形变、力)作为 ControlNet 条件,把物理参数变成视频扩散模型的可控输入;再用微调后的 VLM 对生成视频做物理问答式打分作为可微奖励,这套「属性图条件 + VLM 奖励」范式可直接迁移到其他需要物理可信度的视频生成任务。

原文依据 physics-supervised fine-tuning of a pretrained diffusion model using a ControlNet conditioned on pixel-aligned physical property maps
可迁移到 可控视频生成物理一致性视频生成视频生成评测

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

现代视频扩散模型在表观合成方面表现出色,但在物理一致性上仍存在困难:物体漂移、碰撞缺乏真实的回弹、材料响应很少与其底层属性相符。我们提出 PhyCo,一个将连续、可解释且具有物理依据的控制引入视频生成的框架。我们的方法整合了三个关键组件:(i) 一个大规模数据集,包含超过 10 万段逼真的仿真视频,其中摩擦、恢复系数、形变和力在不同场景中被系统性地变化;(ii) 对预训练扩散模型进行物理监督微调,使用以像素对齐的物理属性图为条件的 ControlNet;(iii) VLM 引导的奖励优化,其中微调后的视觉语言模型通过有针对性的物理查询评估生成的视频,并提供可微反馈。这一组合使生成模型能够通过改变物理属性来产生物理一致且可控的输出——在推理时无需任何仿真器或几何重建。在 Physics-IQ 基准上,PhyCo 相比强基线显著提升了物理真实感,人类研究也证实了对物理属性更清晰、更忠实的控制。我们的结果展示了一条可扩展的路径,通向能够泛化到合成训练环境之外的物理一致、可控的生成式视频模型。

英文原文摘要 1443 字符

Modern video diffusion models excel at appearance synthesis but still struggle with physical consistency: objects drift, collisions lack realistic rebound, and material responses seldom match their underlying properties. We present PhyCo, a framework that introduces continuous, interpretable, and physically grounded control into video generation. Our approach integrates three key components: (i) a large-scale dataset of over 100K photorealistic simulation videos where friction, restitution, deformation, and force are systematically varied across diverse scenarios; (ii) physics-supervised fine-tuning of a pretrained diffusion model using a ControlNet conditioned on pixel-aligned physical property maps; and (iii) VLM-guided reward optimization, where a fine-tuned vision-language model evaluates generated videos with targeted physics queries and provides differentiable feedback. This combination enables a generative model to produce physically consistent and controllable outputs through variations in physical attributes--without any simulator or geometry reconstruction at inference. On the Physics-IQ benchmark, PhyCo significantly improves physical realism over strong baselines, and human studies confirm clearer and more faithful control over physical attributes. Our results demonstrate a scalable path toward physically consistent, controllable generative video models that generalize beyond synthetic training environments.

标签

会议 CVPR2026
页码 41892-41902
Bibkey
来源 CVF Open Access