生成模型与扩散 · CVPR2026 ·2026.06.03

MoReGen:面向基于代码的文本到视频合成的多智能体运动推理引擎

MoReGen: Multi-Agent Motion-Reasoning Engine for Code-based Text-to-Video Synthesis

Xiangyu Bai、He Liang、Bishoy Galoaa、Utsav Nandi、Shayda Moezzi、Yuhang He 等 7 人

与你有关的部分

把物理仿真器与渲染器接入多智能体LLM流水线,在代码域生成可复现视频,并用物体轨迹对应作为直接评测指标;对做视频生成的人,这提供了一条绕开像素级扩散、直接保证牛顿运动一致性的可复用路线与配套基准。

原文依据 we propose object-trajectory correspondence as a direct evaluation metric
可迁移到 text-to-video generationphysical plausibility evaluationtrajectory prediction

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

尽管文本到视频(T2V)生成在照片级真实感方面已取得显著进展,但生成忠实遵循物理原理、与意图对齐的视频仍是一项核心挑战。本文系统研究牛顿运动可控的文本到视频生成与评测,强调物理精度与运动连贯性。我们提出 MoReGen,一个运动感知、物理落地的 T2V 框架,它集成多智能体大语言模型(LLM)、物理仿真器与渲染器,在代码域中从文本提示生成可复现、物理准确的视频。为定量评估物理有效性,我们提出物体-轨迹对应关系作为直接评测指标,并给出 MoReSet——一个包含 1,275 段人工标注视频的基准,覆盖九类牛顿现象,并附带场景描述、时空关系与真值轨迹。利用 MoReSet,我们对现有 T2V 模型开展实验,通过我们的 MoRe 指标与现有基于物理的评测器评估其物理有效性。结果表明,最优模型难以维持物理有效性,而 MoReGen 为物理连贯的视频合成确立了一个有原则的方向。

英文原文摘要 1254 字符

While text-to-video (T2V) generation has achieved remarkable progress in photorealism, generating intent-aligned videos that faithfully obey physics principles remains a core challenge. In this work, we systematically study Newtonian motion-controlled text-to-video generation and evaluation, emphasizing physical precision and motion coherence. We introduce MoReGen, a motion-aware, physics-grounded T2V framework that integrates multi-agent large language models (LLMs), physics simulators, and renderers to generate reproducible, physically accurate videos from text prompts in the code domain. To quantitatively assess physical validity, we propose object-trajectory correspondence as a direct evaluation metric and present MoReSet, a benchmark of 1,275 human-annotated videos spanning nine classes of Newtonian phenomena with scene descriptions, spatiotemporal relations, and ground-truth trajectories. Using MoReSet, we conduct experiments on existing T2V models, evaluating their physical validity through both our MoRe metrics and existing physics-based evaluators. Our results reveal that state-of-the-art models struggle to maintain physical validity, while MoReGen establishes a principled direction toward physically coherent video synthesis.

标签

会议 CVPR2026
页码 7632-7642
Bibkey
来源 CVF Open Access