FlowDirector:面向精确文本到视频编辑的免训练流引导
FlowDirector: Training-Free Flow Steering for Precise Text-to-Video Editing
提出免反演的视频编辑范式:直接在数据空间用 ODE 沿视频固有的时空流形演化,并用方向感知流校正、运动-外观解耦能量项、差分平均引导抑制伪影,可复用于任何基于扩散/流的视频编辑与可控生成管线,绕开反演误差。
原文依据 we introduce FlowDirector, a novel training-free and inversion-free video editing framework
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论文本驱动的视频编辑旨在依据自然语言指令修改视频内容。近期免训练方法虽借助预训练扩散模型,但往往依赖反演-编辑范式。该范式先将视频映射到潜空间再编辑。然而反演过程并非完全精确,常损害外观保真度与运动一致性。为此,我们提出 FlowDirector,一种新颖的免训练且免反演的视频编辑框架。该框架将编辑过程建模为数据空间中的直接演化,利用常微分方程(ODE)引导视频沿其固有的时空流形平滑过渡,从而避免不精确的反演步骤。在此基础上,我们引入三种分别针对外观、运动与稳定性的流校正策略:1)方向感知流校正放大与源方向相反的分量并去除无关项,打破保守流线,实现更强的结构与纹理变化。2)运动-外观解耦在每个时间步将运动一致性作为能量项优化,显著提升一致性与运动迁移。3)差分平均引导策略利用多个候选流之间的差异,以低代价近似低方差状态,抑制伪影并稳定轨迹。在多种编辑任务与基准上的大量实验表明,FlowDirector 在指令遵循、时序一致性与背景保持方面达到最优性能,为无需反演的连贯视频编辑建立了高效新范式。
英文原文摘要 1745 字符
Text-driven video editing aims to modify video content based on natural language instructions. While recent training-free methods have leveraged pretrained diffusion models, they often rely on an inversion-editing paradigm. This paradigm maps the video to a latent space before editing. However, the inversion process is not perfectly accurate, often compromising appearance fidelity and motion consistency. To address this, we introduce FlowDirector, a novel training-free and inversion-free video editing framework. Our framework models the editing process as a direct evolution in the data space. It guides the video to transition smoothly along its inherent spatio-temporal manifold using an ordinary differential equation (ODE), thereby avoiding the inaccurate inversion step. From this foundation, we introduce three flow correction strategies for appearance, motion, and stability: 1) Direction-aware flow correction amplifies components that oppose the source direction and removes irrelevant terms, breaking conservative streamlines and enabling stronger structural and textural changes. 2) Motion-appearance decoupling optimizes motion agreement as an energy term at each timestep, significantly improving consistency and motion transfer. 3) Differential averaging guidance strategy leverages differences among multiple candidate flows to approximate a low variance regime at low cost, suppressing artifacts and stabilizing the trajectory. Extensive experiments across various editing tasks and benchmarks demonstrate that FlowDirector achieves state-of-the-art performance in instruction following, temporal consistency, and background preservation, establishing an efficient new paradigm for coherent video editing without inversion.