STCDiT：时空一致的扩散 Transformer 用于高质量视频超分辨率
STCDiT: Spatio-Temporally Consistent Diffusion Transformer for High-Quality Video Super-Resolution
利用预训练视频扩散模型做视频超分时，通过运动感知的 VAE 分段重建和锚帧潜变量引导来提升结构保真度与时序稳定性，这套思路可直接迁移到其他基于扩散模型的视频复原任务（如去噪、去模糊）。
原文依据 We present STCDiT, a video super-resolution framework built upon a pre-trained video diffusion model
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论我们提出 STCDiT，一个基于预训练视频扩散模型的视频超分辨率框架，旨在从退化输入中恢复结构忠实且时序稳定的视频，即使在复杂相机运动下也能有效工作。主要挑战在于重建过程中保持时序稳定性，以及在生成过程中保持结构保真度。为解决这些挑战，我们首先开发了一种运动感知的 VAE 重建方法，该方法执行分段重建，每个片段剪辑表现出均匀的运动特性，从而有效处理具有复杂相机运动的视频。此外，我们观察到每个剪辑中由 VAE 编码器提取的第一帧潜变量（称为锚帧潜变量）不受时序压缩影响，并且比后续帧潜变量保留了更丰富的空间结构信息。我们进一步开发了一种锚帧引导方法，利用锚帧的结构信息来约束生成过程并提高视频特征的结构保真度。结合这两种设计，使得视频扩散模型能够实现高质量的视频超分辨率。大量实验表明，STCDiT 在结构保真度和时序一致性方面优于最优方法。
英文原文摘要 1309 字符
We present STCDiT, a video super-resolution framework built upon a pre-trained video diffusion model, aiming to restore structurally faithful and temporally stable videos from degraded inputs, even under complex camera motions. The main challenges lie in maintaining temporal stability during reconstruction and preserving structural fidelity during generation. To address these challenges, we first develop a motion-aware VAE reconstruction method that performs segment-wise reconstruction, with each segment clip exhibiting uniform motion characteristic, thereby effectively handling videos with complex camera motions. Moreover, we observe that the first-frame latent extracted by the VAE encoder in each clip, termed the anchor-frame latent, remains unaffected by temporal compression and retains richer spatial structural information than subsequent frame latents. We further develop an anchor-frame guidance approach that leverages structural information from anchor frames to constrain the generation process and improve structural fidelity of video features. Coupling these two designs enables the video diffusion model to achieve high-quality video super-resolution. Extensive experiments show that STCDiT outperforms state-of-the-art methods in terms of structural fidelity and temporal consistency.