LoL:比更长更长,将视频生成扩展至小时级
LoL: Longer than Longer, Scaling Video Generation to Hour
针对自回归长视频生成中注意力汇聚帧导致的场景重置问题,提出多头 RoPE 抖动这一免训练手段,打破头间注意力同质化;做流式/无限长视频生成或复用 RoPE 的位置编码方案时可直接借鉴。
原文依据 we propose a lightweight, training-free approach that effectively suppresses this behavior by introducing multi-head RoPE jitter
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近期长视频生成研究已从双向模型转向自回归模型,但这类方法普遍存在误差累积与长期连贯性丧失的问题。尽管已有工作引入注意力汇聚帧(attention sink frame)来缓解这种性能衰减,但它们往往会引发一种我们称为「汇聚坍缩」(sink-collapse)的关键失效模式:生成内容反复回退到汇聚帧,导致场景突然重置与循环运动模式。我们的分析表明,汇聚坍缩源于旋转位置编码(RoPE)的周期性结构与当前生成模型中普遍采用的多头注意力机制之间的内在冲突。为解决该问题,我们提出一种轻量、免训练的方法,通过引入多头 RoPE 抖动来打破头间注意力同质化并缓解长时程坍缩,从而有效抑制该行为。大量实验表明,我们的方法在保持生成质量的同时成功缓解了汇聚坍缩。据我们所知,本工作首次实现了实时、流式且无限长度的视频生成,且质量衰减极小。作为这种鲁棒性的例证,我们生成了长达 12 小时的连续视频,据我们所知,这是流式视频生成中公开演示的最长结果之一。
英文原文摘要 1401 字符
Recent research in long-form video generation has shifted from bidirectional to autoregressive models, yet these methods commonly suffer from error accumulation and a loss of long-term coherence. While attention sink frames have been introduced to mitigate this performance decay, they often induce a critical failure mode we term sink-collapse: the generated content repeatedly reverts to the sink frame, resulting in abrupt scene resets and cyclic motion patterns. Our analysis reveals that sink-collapse originates from an inherent conflict between the periodic structure of Rotary Position Embedding (RoPE) and the multi-head attention mechanisms prevalent in current generative models. To address it, we propose a lightweight, training-free approach that effectively suppresses this behavior by introducing multi-head RoPE jitter that breaks inter-head attention homogenization and mitigates long-horizon collapse. Extensive experiments show that our method successfully alleviates sink-collapse while preserving generation quality. To the best of our knowledge, this work achieves the first demonstration of real-time, streaming, and infinite-length video generation with little quality decay. As an illustration of this robustness, we generate continuous videos up to 12 hours in length, which, to our knowledge, is among the longest publicly demonstrated results in streaming video generation.