VerseCrafter：具备4D几何控制的动态真实视频世界模型
VerseCrafter: Dynamic Realistic Video World Model with 4D Geometric Control
用静态背景点云加逐物体3D高斯轨迹构成的4D几何控制表示，渲染成控制图去条件化预训练视频扩散模型，为视觉研究者提供了一种类别无关、可精确控制相机与多物体运动的视频生成条件化范式。
原文依据 We render 4D Geometric Control into 4D control maps for a pretrained video diffusion model
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视频世界模型旨在模拟动态的真实世界环境，然而现有方法难以对相机与多物体运动提供统一且精确的控制，因为视频本身是在投影后的二维图像平面中捕捉动态的。为弥合这一差距，我们提出VerseCrafter，一个几何驱动的视频世界模型，它从统一的4D几何世界状态生成动态、真实的视频。我们的方法以新颖的4D几何控制表示为核⼼，该表示将世界状态编码为静态背景点云与逐物体的3D高斯轨迹。这一表示捕捉每个物体的运动路径及其随时间的概率性3D占用，为刚性边界框与参数化模型提供了一种灵活且类别无关的替代方案。我们将4D几何控制渲染为4D控制图，用于预训练的视频扩散模型，从而实现高保真、视角一致且忠实遵循指定动态的视频生成。为支持大规模训练，我们开发了一个自动数据引擎，并构建了VerseControl4D，这是一个包含3.5万个训练样本的真实世界数据集，其提示与渲染的4D控制图均为自动导出。大量实验表明，VerseCrafter在视觉质量以及对相机与多物体运动的控制精度上均优于先前方法。
英文原文摘要 1347 字符
Video world models aim to simulate dynamic, real-world environments, yet existing methods struggle to provide unified and precise control over camera and multi-object motion, as videos inherently capture dynamics in the projected 2D image plane. To bridge this gap, we introduce VerseCrafter, a geometry-driven video world model that generates dynamic, realistic videos from a unified 4D geometric world state. Our approach is centered on a novel 4D Geometric Control representation, which encodes the world state as a static background point cloud and per-object 3D Gaussian trajectories. This representation captures each object's motion path and probabilistic 3D occupancy over time, providing a flexible, category-agnostic alternative to rigid bounding boxes and parametric models. We render 4D Geometric Control into 4D control maps for a pretrained video diffusion model, enabling high-fidelity, view-consistent video generation that faithfully follows the specified dynamics. To enable training at scale, we develop an automatic data engine and construct VerseControl4D, a real-world dataset of 35K training samples with automatically derived prompts and rendered 4D control maps. Extensive experiments show that VerseCrafter achieves superior visual quality and more accurate control over camera and multi-object motion than prior methods.