面向视频帧插值的整体建模：自回归扩散 Transformer
Towards Holistic Modeling for Video Frame Interpolation with Auto-regressive Diffusion Transformers
把视频帧插值从「三元组」式局部处理改为整段视频的自回归扩散 Transformer 建模，配合 skip-concatenate 采样抑制误差累积、稀疏局部注意力与瓦片式 VAE 编码支持 4K 任意分辨率推理，可直接迁移到长视频生成与视频复原任务的长时序一致性建模。
原文依据 we introduce a novel skip-concatenate sampling strategy that effectively maintains temporal stability
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论现有的视频帧插值（VFI）方法通常采用以帧为中心的做法，将视频作为独立的短片段（例如三元组）处理，这会导致时序不一致与运动伪影。为克服这一问题，我们提出一种整体化、以视频为中心的范式，名为面向视频帧插值的局部扩散强制（LDF-VFI）。该框架构建于自回归扩散 Transformer 之上，对整个视频序列建模以确保长程时序连贯性。为缓解自回归生成固有的误差累积，我们引入一种新颖的跳连拼接采样策略，有效维持时序稳定性。此外，LDF-VFI 结合了稀疏局部注意力与瓦片式变分自编码器编码，这一组合不仅能够高效处理长序列，还使其在推理时可泛化到任意空间分辨率（例如 4K）而无需重新训练。增强的条件变分自编码器解码器利用输入视频的多尺度特征，进一步提升了重建保真度。在实验中，LDF-VFI 在具有挑战性的 VFI 基准上取得了最优性能，展现出更优的逐帧质量与时序一致性，尤其是在大运动场景中。
英文原文摘要 1271 字符
Existing video frame interpolation (VFI) methods often adopt a frame-centric approach, processing videos as independent short segments (e.g., triplets), which leads to temporal inconsistencies and motion artifacts. To overcome this, we propose a holistic, video-centric paradigm named Local Diffusion Forcing for Video Frame Interpolation (LDF-VFI). Our framework is built upon an auto-regressive diffusion transformer that models the entire video sequence to ensure long-range temporal coherence. To mitigate error accumulation inherent in auto-regressive generation, we introduce a novel skip-concatenate sampling strategy that effectively maintains temporal stability. Furthermore, LDF-VFI incorporates sparse, local attention and tiled VAE encoding, a combination that not only enables efficient processing of long sequences but also allows generalization to arbitrary spatial resolutions (e.g., 4K) at inference without retraining. An enhanced conditional VAE decoder, which leverages multi-scale features from the input video, further improves reconstruction fidelity. Empirically, LDF-VFI achieves state-of-the-art performance on challenging VFI benchmarks, demonstrating superior per-frame quality and temporal consistency, especially in scenes with large motion.