物理仿真器在环的视频生成
Physical Simulator In-the-Loop Video Generation
将物理仿真器嵌入视频扩散采样回路，用仿真轨迹约束生成时空运动，并用测试时纹理一致性优化（TTCO）基于像素对应调整文本与特征嵌入，为提升生成视频的物理合理性提供了可复用范式。
原文依据 integrates a physical simulator into the video diffusion process
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近期基于扩散的视频生成取得了显著的视觉真实感，但仍难以遵循重力、惯性和碰撞等基本物理规律。生成的物体往往在帧间运动不一致、呈现不合理的动力学，或违反物理约束，限制了 AI 生成视频的真实性与可靠性。为弥补这一差距，我们提出物理仿真器在环的视频生成（PSIVG），一种将物理仿真器集成到视频扩散过程中的新框架。PSIVG 从预训练扩散模型生成的模板视频出发，重建 4D 场景与前景物体网格，将其初始化于物理仿真器中，并生成物理一致的轨迹。随后利用这些仿真轨迹引导视频生成器产生时空上物理连贯的运动。为进一步提升物体运动过程中的纹理一致性，我们提出测试时纹理一致性优化（TTCO）技术，基于仿真器提供的像素对应关系自适应调整文本与特征嵌入。大量实验表明，PSIVG 生成的视频在保持视觉质量与多样性的同时，更好地遵循真实世界物理规律。项目主页：https://vcai.mpi-inf.mpg.de/projects/PSIVG
英文原文摘要 1338 字符
Recent advances in diffusion-based video generation have achieved remarkable visual realism but still struggle to obey basic physical laws such as gravity, inertia, and collision. Generated objects often move inconsistently across frames, exhibit implausible dynamics, or violate physical constraints, limiting the realism and reliability of AI-generated videos. We address this gap by introducing Physical Simulator In-the-loop Video Generation (PSIVG), a novel framework that integrates a physical simulator into the video diffusion process. Starting from a template video generated by a pre-trained diffusion model, PSIVG reconstructs the 4D scene and foreground object meshes, initializes them within a physical simulator, and generates physically consistent trajectories. These simulated trajectories are then used to guide the video generator toward spatio-temporally physically coherent motion. To further improve texture consistency during object movement, we propose a Test-Time Texture Consistency Optimization (TTCO) technique that adapts text and feature embeddings based on pixel correspondences from the simulator. Comprehensive experiments demonstrate that PSIVG produces videos that better adhere to real-world physics while preserving visual quality and diversity. Project Page: https://vcai.mpi-inf.mpg.de/projects/PSIVG