PropFly：借助预训练视频扩散模型的在线监督学习传播
PropFly: Learning to Propagate via On-the-Fly Supervision from Pre-trained Video Diffusion Models
利用预训练视频扩散模型在不同 CFG 尺度下的一步干净潜变量估计，在线构造源/编辑潜变量对，从而免去成对视频编辑数据集；这一在线监督思路可直接迁移到其他需要成对数据的视频编辑或可控生成任务中。
原文依据 leverages one-step clean latent estimations from intermediate noised latents with varying Classifier-Free Guidance (CFG) scales
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论基于传播的视频编辑通过将单个已编辑帧传播到后续帧，同时保持运动与结构等原始上下文，从而实现精确的用户控制。然而，训练此类模型需要大规模成对（源视频与编辑后视频）视频数据集，其获取成本高且过程复杂。为此，我们提出 PropFly，一种基于传播的视频编辑训练流程，它依赖预训练视频扩散模型（VDM）提供的在线监督，而无需现成的或预先计算好的成对视频编辑数据集。具体而言，PropFly 利用在不同无分类器引导（CFG）尺度下从中间加噪潜变量得到的一步干净潜变量估计，在线合成多样化的“源”（低 CFG）与“编辑后”（高 CFG）潜变量对。源潜变量提供视频的结构信息，而编辑后潜变量提供用于学习传播的目标变换。该流程使附加在预训练 VDM 上的适配器能够通过引导调制流匹配（GMFM）损失学习传播编辑，该损失引导模型复现目标变换。我们的在线监督确保模型学习到时间一致且动态的变换。大量实验表明，PropFly 在多种视频编辑任务上显著优于最优方法，并能产生高质量的编辑结果。
英文原文摘要 1466 字符
Propagation-based video editing enables precise user control by propagating a single edited frame into following frames while maintaining the original context such as motion and structures.However, training such models requires large-scale, paired (source and edited) video datasets, which are costly and complex to acquire.Hence, we propose the PropFly, a training pipeline for Propagation-based video editing, relying on on-the-Fly supervision from pre-trained video diffusion models (VDMs) instead of requiring off-the-shelf or precomputed paired video editing datasets.Specifically, our PropFly leverages one-step clean latent estimations from intermediate noised latents with varying Classifier-Free Guidance (CFG) scales to synthesize diverse pairs of 'source' (low-CFG) and 'edited' (high-CFG) latents on-the-fly. The source latent serves as structural information of the video, while the edited latent provides the target transformation for learning propagation.Our pipeline enables an additional adapter attached to the pre-trained VDM to learn to propagate edits via Guidance-Modulated Flow Matching (GMFM) loss, which guides the model to replicate the target transformation.Our on-the-fly supervision ensures the model to learn temporally consistent and dynamic transformations.Extensive experiments demonstrate that our PropFly significantly outperforms the state-of-the-art methods on various video editing tasks, producing high-quality editing results.