Dynamics：基于语言从视频推断刚体动力学的表示方法
Dynamics: Language-Based Representation for Inferring Rigid-Body Dynamics From Videos
将语言作为刚体动力学的统一表示，利用光流作为语义无关输入增强泛化，并通过测试时采样和进化搜索提升物理参数推断精度，为视觉物理推理提供了新范式。
原文依据 We introduce Dynamics, a vision-language framework that uses language as a unified representation of rigid-body dynamics.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论从单目视频中推断刚体物理状态和属性是迈向基于物理的感知与仿真的基础步骤。现有方法假设特定的底层物理系统、物体类型和相机姿态，无法泛化到复杂的现实世界场景。我们提出 Dynamics，一个视觉-语言框架，使用语言作为刚体动力学的统一表示。Dynamics 不直接预测参数，而是以结构化文本格式生成场景配置用于物理仿真。我们通过整合自然语言运动推理并利用光流作为语义无关输入来增强模型的泛化能力。在 CLEVRER 数据集上，Dynamics 达到 0.30 的分割 IoU，比领先的视觉语言模型（InternVL3-8B、Qwen2.5-VL-7B 和 Claude-4-Sonnet）提升 7 倍。此外，测试时采样和进化搜索分别将分割 IoU 进一步提升 27% 和 120%。最后，我们展示了在包含 235 个真实世界刚体视频的新数据集上的强迁移能力，突显了语言驱动的物理推断在连接感知与仿真方面的潜力。
英文原文摘要 1187 字符
Inferring rigid-body physical states and properties from monocular videos is a fundamental step toward physics-based perception and simulation. Existing approaches assume specific underlying physical systems, object types, and camera poses, which are unable to generalize to complex real-world settings. We introduce Dynamics, a vision-language framework that uses language as a unified representation of rigid-body dynamics. Instead of directly predicting parameters, Dynamics generates scene configurations in a structured text format for physics simulation. We enhance the model's generalization by integrating natural language motion reasoning and leveraging optical flow as a semantic-agnostic input. On the CLEVRER dataset, Dynamics achieves a segmentation IoU of 0.30, a 7ximprovement over leading VLMs (InternVL3-8B, Qwen2.5-VL-7B and Claude-4-Sonnet). Further, test-time sampling and evolutionary search further boost performance by 27% and 120% in segmentation IoU, respectively. Finally, we demonstrate strong transfer to a new dataset of 235 real-world rigid-body videos, highlighting the potential of language-driven physics inference for bridging perception and simulation.