基于层自适应分布外校正的免训练长视频生成
Free-Lunch Long Video Generation via Layer-Adaptive O.O.D Correction
针对预训练短片段视频扩散模型在长视频推理时的分布外问题，提出免训练的分层自适应校正框架，包含视频相对位置重编码和分层稀疏注意力，可直接用于提升现有视频扩散模型的长视频生成质量。
原文依据 we propose FreeLOC, a novel training-free, layer-adaptive framework that introduces two core techniques
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论使用预训练视频扩散模型生成长期视频（这些模型通常在短片段上训练）是一项重大挑战。直接将这类模型用于长视频推理往往会导致视觉质量显著下降。本文指出，该问题主要源于两个分布外（O.O.D）问题：帧级相对位置分布外和上下文长度分布外。为解决这些挑战，我们提出FreeLOC，一种新颖的免训练、层自适应框架，引入两项核心技术：针对帧级相对位置分布外的基于视频的相对位置重编码（VRPR），这是一种多粒度策略，分层重新编码时序相对位置以对齐模型预训练分布；以及针对上下文长度分布外的分层稀疏注意力（TSA），通过在不同时间尺度上构建注意力密度来同时保留局部细节和长程依赖。关键的是，我们引入了一种层自适应探测机制，用于识别每个Transformer层对这些分布外问题的敏感度，从而能够选择性地高效应用我们的方法。大量实验表明，我们的方法显著优于现有免训练方法，在时序一致性和视觉质量方面均达到最优结果。代码见 https://github.com/Westlake-AGI-Lab/FreeLOC。
英文原文摘要 1438 字符
Generating long videos using pre-trained video diffusion models, which are typically trained on short clips, presents a significant challenge. Directly applying these models for long-video inference often leads to a notable degradation in visual quality. This paper identifies that this issue primarily stems from two out-of-distribution (O.O.D) problems: frame-level relative position O.O.D and contextlength O.O.D. To address these challenges, we propose FreeLOC, a novel training-free, layer-adaptive framework that introduces two core techniques: Video-based Relative Position Re-encoding (VRPR) for frame-level relative position O.O.D, a multi-granularity strategy that hierarchically re-encodes temporal relative positions to align with the model's pre-trained distribution, and Tiered Sparse Attention (TSA) for context-length O.O.D, which preserves both local detail and long-range dependencies by structuring attention density across different temporal scales. Crucially, we introduce a layer-adaptive probing mechanism that identifies the sensitivity of each transformer layer to these O.O.D issues, allowing for the selective and efficient application of our methods. Extensive experiments demonstrate that our approach significantly outperforms existing training-free methods, achieving state-of-the-art results in both temporal consistency and visual quality. Code is available at https://github.com/Westlake-AGI-Lab/FreeLOC.