DynamicsBoost:通过无标注续写偏好优化实现动态合理的视频生成
DynamicsBoost: Dynamic Plausible Video Generation via Annotation-Free Continuation Preference Optimization
用视频续写自动构造偏好对(参考帧越多、生成内容越少、保真度越高),把 DPO 从昂贵的人工/VLM 标注中解放出来;做视频生成偏好对齐或奖励建模的可以直接复用这套数据构造与 Asymmetrical DPO 损失设计。
原文依据 we propose an annotation-free preference alignment method that constructs accurate preference pairs through video continuation
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论尽管文本到视频生成已取得显著进展,现有模型仍存在动态不真实、时序不一致以及语义对齐不稳定等问题。现有的偏好对齐方法依赖昂贵且往往含糊的人工或基于 VLM 的视频偏好标注,这已成为扩展数据规模的主要瓶颈。为应对这一挑战,我们提出一种无标注的偏好对齐方法,通过视频续写构造准确的偏好对。我们将预训练视频生成模型扩展为续写模型,并在保持视频总长度不变的前提下,使用不同数量的参考帧执行续写。由于生成片段的质量劣于真实帧,且以更多参考帧为条件的定长续写包含更少的生成内容,因此其保真度高于参考帧较少的续写,从而自然地形成偏好顺序。我们进一步提出非对称 DPO(Asymmetrical DPO),在除共享前缀条件帧之外的所有续写区域上计算偏好损失,并按这些区域的长度进行归一化,防止虚假的偏好信号泄漏到条件部分。在多个基准上的实验表明,与现有基于 DPO 的方法相比,我们的方法在动态真实性、时序连贯性和语义对齐方面均有显著提升,同时完全消除了对人工偏好标注或辅助奖励模型的需求。
英文原文摘要 1490 字符
Despite significant progress in text-to-video generation, current models still suffer from unrealistic dynamics, temporal inconsistency, and unstable semantic alignment. Existing preference alignment approaches rely on costly and often ambiguous human or VLM-based video preference annotation, which has become a major bottleneck for scaling data. To address this challenge, we propose an annotation-free preference alignment method that constructs accurate preference pairs through video continuation.We extend a pretrained video generation model into a continuation model and apply continuation with different amounts of reference frames while keeping the total video length fixed. As generated segments are inferior to ground-truth frames and and fixed-length continuations conditioned on more reference frames contain less generated content, they exhibit higher fidelity than those with fewer references, naturally inducing a preference order.We further introduce Asymmetrical DPO, which computes preference loss on all continuation regions except the shared prefix conditioning frames and normalizes it by their length, preventing spurious preference signals from leaking into the conditioned portion.Experiments across multiple benchmarks show that our method delivers significant improvements in dynamics realism, temporal coherence, and semantic alignment over existing DPO-based approaches, while fully eliminating the need for human preference labeling or auxiliary reward models.