以物理可控世界模型理解物理对象
Physical Object Understanding with a Physically Controllable World Model
把视频世界模型当作可查询的概率推断器：通过对未来状态采样并分析运动相关性来无监督地分割出物体及其可动部件，为视觉研究者提供了一条不依赖标注的物体发现与3D操控思路。
原文依据 by analyzing motion correlations across these futures, we extract objects and articulated object subparts
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉智能的一个核心挑战是从原始视频中学习场景的物理结构：区域如何构成物体，以及支配其相互作用的规律。解决这些任务需要能够从部分观测中推断世界分布状态的世界模型——而当前架构并不具备这种能力。我们引入一类新的概率世界模型，支持在给定任意其他变量的条件下估计任意视觉变量（如外观与动力学）的概率。在此，我们发现这类模型可以通过自回归序列建模高效训练，由此得到的世界模型会涌现出丰富的物体理解能力。首先，我们证明该模型能够通过序列推断生成多个合理的未来世界状态，从而捕捉支配物体运动的物理规律。随后，通过分析这些未来状态之间的运动相关性，我们提取出物体及其可动子部件。在发现这些物体之后，我们展示该世界模型能够在三维空间中操控它们。最后，我们演示如何从世界模型中计算物体之间的物理关系，从而支持诸如 Visual Jenga 之类的应用。
英文原文摘要 1250 字符
A central challenge in visual intelligence is learning the physical structure of scenes from raw videos: how regions form objects and the laws that govern their interactions. Solving these tasks requires world models capable of inferring distributional states of the world from partial observations -- capabilities that current architectures do not provide. We introduce a new class of probabilistic world models that support estimation of the probability of any visual variable, such as appearance and dynamics, conditioned on any other variables. Here, we identify that these models can be trained efficiently with autoregressive sequence modeling, yielding world models from which rich object understanding emerges. First, we demonstrate that our model captures the physical laws governing how objects move by generating multiple plausible future states of the world through sequential inference. Then, by analyzing motion correlations across these futures, we extract objects and articulated object subparts. Having discovered these objects, we show that our world model can manipulate them in 3D. Finally, we demonstrate how physical relationships between objects can be computed from the world model, enabling applications such as Visual Jenga.