ReDirector:用旋转相机编码生成任意长度视频重拍
ReDirector: Creating Any-Length Video Retakes with Rotary Camera Encoding
把相机位姿编码为 RoPE 相位偏移(RoCE),使视频扩散模型能泛化到训练分布外的相机轨迹与任意长度;做相机可控视频生成/重拍时可直接复用这一条件注入方式,替代外挂相机编码器。
原文依据 we introduce Rotary Camera Encoding (RoCE), a camera-conditioned RoPE phase shift that captures and integrates multi-view relationships
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论我们提出 ReDirector,一种面向动态拍摄的可变长度视频的相机可控视频重拍生成方法。具体而言,我们纠正了此前工作中对 RoPE 的一种常见误用,使输入视频与目标重拍在时空位置上对齐。此外,我们引入旋转相机编码(RoCE),这是一种以相机为条件的 RoPE 相位偏移,能够捕捉并整合输入视频与目标视频内部及二者之间的多视角关系。通过将相机条件融入 RoPE,我们的方法可泛化到分布外的相机轨迹与视频长度,并带来动态物体定位与静态背景保持方面的提升。大量实验进一步表明,该方法在多种轨迹与长度下于相机可控性、几何一致性和视频质量上均有显著提升。
英文原文摘要 859 字符
We present ReDirector, a novel camera-controlled video retake generation method for dynamically captured variable-length videos. In particular, we rectify a common misuse of RoPE in previous works by aligning the spatiotemporal positions of the input video and the target retake. Moreover, we introduce Rotary Camera Encoding (RoCE), a camera-conditioned RoPE phase shift that captures and integrates multi-view relationships within and across the input and target videos. By integrating camera conditions into RoPE, our method generalizes to out-of-distribution camera trajectories and video lengths, yielding improved dynamic object localization and static background preservation. Extensive experiments further demonstrate significant improvements in camera controllability, geometric consistency, and video quality across various trajectories and lengths.