SIMPACT：利用视觉语言模型的仿真赋能动作规划
SIMPACT: Simulation-Enabled Action Planning using Vision-Language Models
把可微/高效的物理仿真器当作 VLM 的『世界模型』在测试时闭环调用：从单张 RGB-D 快速构建仿真场景，让 VLM 提出动作、观察 rollout 再修正推理，无需额外训练。做具身操作或 VLM 物理常识的研究者可直接复用这一『仿真在环』的推理范式，替代纯文本 CoT 或昂贵的真机试错。
原文依据 a test-time, SIMulation-enabled ACTion Planning framework that equips VLMs with physical reasoning through simulation-in-the-loop world modeling, without requiring any additional training
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉语言模型（VLM）展现出卓越的常识与语义推理能力，但它们缺乏对物理动力学的具身理解。这一局限源于 VLM 是在静态的互联网规模视觉-语言数据上训练的，这些数据不包含因果交互或动作条件引起的变化。因此，利用 VLM 完成需要物理理解、推理及相应动作规划的细粒度机器人操作任务仍然具有挑战性。为克服这一困难，我们提出 SIMPACT，一个测试时的、仿真赋能的动作规划框架，它通过仿真在环的世界建模为 VLM 赋予物理推理能力，且无需任何额外训练。从单次 RGB-D 观测出发，SIMPACT 高效地构建物理仿真，使 VLM 能够提出有依据的动作、观察仿真推演，并迭代地修正其推理。通过将语言推理与物理预测相结合，我们的仿真赋能 VLM 能够以物理具身的方式理解接触动力学与动作结果。我们的方法在五项具有挑战性的真实世界刚体与可变形物体操作任务上取得了最优性能，这些任务需要细粒度的物理推理，且优于现有的通用机器人操作模型。我们的结果表明，在测试时通过高效仿真将物理理解嵌入 VLM 推理，为通向可泛化的具身智能提供了一条有前景的路径。
英文原文摘要 1523 字符
Vision-Language Models (VLMs) exhibit remarkable common-sense and semantic reasoning capabilities. However, they lack a grounded understanding of physical dynamics. This limitation arises from training VLMs on static internet-scale visual-language data that contain no causal interactions or action-conditioned changes. Consequently, it remains challenging to leverage VLMs for fine-grained robotic manipulation tasks that require physical understanding, reasoning, and corresponding action planning. To overcome this, we present SIMPACT, a test-time, SIMulation-enabled ACTion Planning framework that equips VLMs with physical reasoning through simulation-in-the-loop world modeling, without requiring any additional training. From a single RGB-D observation, SIMPACT efficiently constructs physics simulations, enabling the VLM to propose informed actions, observe simulated rollouts, and iteratively refine its reasoning. By integrating language reasoning with physics prediction, our simulation-enabled VLM can understand contact dynamics and action outcomes in a physically grounded way. Our method demonstrates state-of-the-art performance on five challenging, real-world rigid-body and deformable manipulation tasks that require fine-grained physical reasoning, outperforming existing general-purpose robotic manipulation models. Our results demonstrate that embedding physics understanding via efficient simulation into VLM reasoning at test time offers a promising path towards generalizable embodied intelligence.