通过零样本相机控制驾驭视频模型实现 3D 与 4D 生成
Taming Video Models for 3D and 4D Generation via Zero-Shot Camera Control
提出一种免训练、模型无关的推理时相机轨迹控制方法：在去噪过程中用步内精修循环注入运动引导，并用光流分析定位潜空间中与运动相关的通道，从而解耦运动与外观；可直接复用于视频生成、3D/4D 生成以及视频编辑、稳定化等需要精确相机控制的下游任务。
原文依据 an optical flow-based analysis identifies and isolates motion-related channels within the latent space
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视频扩散模型具有丰富的世界先验，但其在空间任务中的应用受限于控制能力差、时空结果不一致以及场景与相机动态相互纠缠。现有方法（如逐任务微调或后处理变形）往往引入视觉伪影、难以泛化或带来高昂的计算开销。我们提出 WorldForge，一个新颖的免训练框架，完全在推理阶段运行以解决上述问题。该方法由三个协同组件构成。首先，步内精修循环在去噪过程中注入细粒度的运动引导，迭代地校正输出，以确保严格遵循目标相机路径。其次，基于光流的分析识别并分离潜变量空间中与运动相关的通道，使框架能够选择性地施加引导，从而将运动与外观解耦并保持视觉保真度。第三，双路径引导策略通过将受引导的生成与未受引导的参考去噪路径进行比较，自适应地校正漂移，有效消除由结构输入错位造成的伪影。这些组件共同注入了精确的、与轨迹对齐的控制，且无需重新训练模型，实现了准确的运动引导与照片级真实感的合成。作为一种即插即用、模型无关的解决方案，WorldForge 展现出高度的通用泛化能力。除稳健的零样本 3D/4D 生成外，它还能轻松赋能十余种多样的下游应用，无缝支持视频编辑、稳定化和虚拟试穿等任务。大量实验证实，其在轨迹遵循度和感知质量上达到最优性能，优于依赖训练的基线和仅推理的基线。
英文原文摘要 1805 字符
Video diffusion models have rich world priors, but their use in spatial tasks is limited by poor control, spatial-temporal inconsistent results, and entangled scene-camera dynamics. Current approaches, such as per-task fine-tuning or post-process warping, often introduce visual artifacts, fail to generalize, or incur high computational costs. We introduce WorldForge, a novel, training-free framework that operates purely at inference time to resolve these issues. Our method comprises three synergistic components. First, an intra-step refinement loop injects fine-grained motion guidance during the denoising process, iteratively correcting the output to ensure strict adherence to the target camera path. Second, an optical flow-based analysis identifies and isolates motion-related channels within the latent space. This allows our framework to selectively apply guidance, thereby decoupling motion from appearance and preserving visual fidelity. Third, a dual-path guidance strategy adaptively corrects for drift by comparing the guided generation against an unguided, reference denoising path, effectively neutralizing artifacts caused by misaligned structural inputs. Together, these components inject precise, trajectory-aligned control without model retraining, achieving accurate motion guidance and photorealistic synthesis. As a plug-and-play, model-agnostic solution, WorldForge demonstrates highly versatile generalizability. Beyond robust zero-shot 3D/4D generation, it readily empowers over a dozen diverse downstream applications, seamlessly enabling tasks like video editing, stabilization, and virtual try-on. Extensive experiments confirm state-of-the-art performance in trajectory adherence and perceptual quality, outperforming both training-dependent and inference-only baselines.