Pixel2Phys:从视觉动力学中蒸馏支配性物理定律
Pixel2Phys: Distilling Governing Laws from Visual Dynamics
把 MLLM 作为假设生成与验证的推理引擎,从原始视频像素中迭代提炼低维物理变量与可解释控制方程;对做视频理解与物理常识建模的研究者,这提供了一条「视觉观测→符号化动力学」的可复用管线,也可作为评估视频模型是否真正理解动力学的探针。
原文依据 we propose Pixel2Phys, a collaborative multi-agent framework adaptable to any Multimodal Large Language Model (MLLM)
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论直接从高维视觉数据中发现物理定律是人类长期的追求,但对机器而言仍是艰巨挑战,也是科学智能的一项根本目标。该任务本身困难,是因为物理知识是低维且结构化的,而原始视频观测是高维且冗余的,大多数像素几乎不携带或完全不携带物理含义。从此类噪声数据中提取简洁、与物理相关的变量仍是关键障碍。为此,我们提出 Pixel2Phys,一个可适配任意多模态大语言模型(MLLM)的协作式多智能体框架。它通过结构化工作流模拟人类的科学推理,经由迭代的假设生成、验证与精炼来提取形式化的物理知识。通过在高维数据上反复构造并精炼候选方程,它识别出最能刻画潜在物理演化的最简洁表示。这种自动化探索模仿了人类科学家的迭代工作流,使 AI 能够直接从原始观测中揭示可解释的支配方程。在多种模拟与真实物理视频上,Pixel2Phys 均能发现准确、可解释的支配方程,并在基线迅速发散的情况下保持稳定的长期外推。
英文原文摘要 1397 字符
Discovering physical laws directly from high-dimensional visual data is a long-standing human pursuit but remains a formidable challenge for machines, representing a fundamental goal of scientific intelligence. This task is inherently difficult because physical knowledge is low-dimensional and structured, whereas raw video observations are high-dimensional and redundant, with most pixels carrying little or no physical meaning. Extracting concise, physically relevant variables from such noisy data remains a key obstacle. To address this, we propose Pixel2Phys, a collaborative multi-agent framework adaptable to any Multimodal Large Language Model (MLLM). It emulates human scientific reasoning by employing a structured workflow to extract formalized physical knowledge through iterative hypothesis generation, validation, and refinement. By repeatedly formulating, and refining candidate equations on high-dimensional data, it identifies the most concise representations that best capture the underlying physical evolution. This automated exploration mimics the iterative workflow of human scientists, enabling AI to reveal interpretable governing equations directly from raw observations. Across diverse simulated and real-world physics videos, Pixel2Phys discovers accurate, interpretable governing equations and maintaining stable long-term extrapolation where baselines rapidly diverge.