LinVideo：面向高效视频生成中 O(n) 注意力的后训练框架
LinVideo: A Post-Training Framework towards O(n) Attention in Efficient Video Generation
把视频扩散模型中部分自注意力模块替换为线性注意力，并用二分类器自动选择可替换层、用任意时刻分布匹配目标恢复性能，可直接迁移到任何 DiT 类视频/图像生成模型的推理加速上。
原文依据 replaces a target number of self-attention modules with linear attention while preserving performance
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视频扩散模型（DMs）已能实现高质量视频合成，但受自注意力机制本身特性的影响，其计算开销随序列长度呈二次增长。尽管线性注意力提供了更高效的替代方案，但完全替换二次注意力需要代价高昂的预训练。这主要是因为线性注意力缺乏足够的表达能力，难以应对视频生成中固有的复杂时空动态。本文提出 LinVideo，一种高效且无需数据的后训练框架，它在保持性能的同时，将目标数量的自注意力模块替换为线性注意力。首先，我们观察到不同层的可替换性存在显著差异。我们没有采用人工或启发式选择，而是将层选择建模为二分类问题，并提出选择性迁移，以最小性能影响自动且渐进地将层转换为线性注意力。此外，为克服现有目标在优化这一具有挑战性的迁移过程时的低效甚至无效问题，我们引入任意时刻分布匹配（ADM）目标，使采样轨迹上任意时间步的样本分布相互对齐。该目标极为高效，并能恢复模型性能。大量实验表明，LinVideo 在保持生成质量的同时实现了 1.43–1.71 倍的加速，而 4 步蒸馏模型进一步将延迟降低 15.9–20.9 倍，视觉质量仅有轻微下降。
英文原文摘要 1566 字符
Video diffusion models (DMs) have enabled high-quality video synthesis, but their computation costs scale quadratically with sequence length due to the nature of self-attention. While linear attention offers a more efficient alternative, fully replacing quadratic attention demands costly pretraining. This is largely because linear attention lacks sufficient expressiveness and struggles with the complex spatiotemporal dynamics inherent to video generation. In this paper, we present LinVideo, an efficient data-free post-training framework that replaces a target number of self-attention modules with linear attention while preserving performance. First, we observe a significant disparity in the replaceability of different layers. Instead of manual or heuristic choices, we frame layer selection as a binary classification problem and propose a selective transfer, which automatically and progressively converts layers to linear attention with minimal performance impact. Additionally, to overcome the ineffectiveness and even inefficiency of existing objectives in optimizing this challenge transfer process, we introduce an anytime distribution matching (ADM) objective that aligns the distributions of samples across any timestep along the sampling trajectory. This objective is highly efficient and recovers model performance. Extensive experiments show that LinVideo achieves a 1.43\text -1.71x speedup while preserving generation quality, and the 4-step distilled models further reduce latency by 15.9\text -20.9x with only a minor drop in visual quality.