FlashDecoder：基于 Transformer 的实时潜变量到像素流式解码器
FlashDecoder: Real-Time Latent-to-Pixel Streaming Decoder with Transformers
把视频 VAE 的解码器从 3D 卷积换成带滚动 KV 缓存的纯 Transformer，实现定长时序窗口下的逐帧流式解码，可直接替换现有潜视频扩散模型的解码环节，在 1080p 长视频上同时降低延迟与显存。
原文依据 a fast, memory-efficient pure-Transformer video decoder that decodes latents to pixels frame by frame
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论实时视频生成既需要快速去噪，也同样需要快速解码，然而当前的潜视频扩散模型依赖三维卷积解码器，在高分辨率或长视频场景下速度慢且显存开销大。我们提出 FlashDecoder，一种快速、显存高效的纯 Transformer 视频解码器，逐帧将潜变量解码为像素。每一步中，当前帧仅通过滚动 KV 缓存关注固定大小的历史帧窗口。固定的时序窗口使解码速度与显存占用不随视频长度增长，从而实现恒定延迟的流式处理。由于帧按顺序处理，时序因果性无需显式注意力掩码即可得到保证，使得训练可在高达 1080p 的分辨率下进行，并达到与卷积解码器相当的重建质量。在 Wan2.1 与 Wan2.2 潜空间上，FlashDecoder 的重建质量与各自的卷积解码器相当（例如 1080p 下峰值信噪比为 41.55 对 41.49 dB），同时在单块 H100 GPU 上解码速度快 3.6 至 4.7 倍，显存占用最多降低 11 倍。结合面向架构的推理优化，加速比可进一步扩大至 12 倍。
英文原文摘要 1128 字符
Real-time video generation demands fast decoding as much as fast denoising, yet current latent video diffusion models rely on 3D convolutional decoders that are slow and memory-intensive at high resolutions or for long video. We introduce FlashDecoder, a fast, memory-efficient pure-Transformer video decoder that decodes latents to pixels frame by frame. At each step, the current frame attends only to a fixed-size window of past frames through a rolling KV cache. The fixed temporal window keeps decoding fast and memory bounded regardless of video length, enabling constant-latency streaming. Because frames are processed sequentially, temporal causality is enforced without explicit attention masks, enabling training at resolutions up to 1080p and matching the reconstruction quality of convolutional decoders. On the Wan2.1 and Wan2.2 latent spaces, FlashDecoder matches each convolutional decoder in reconstruction quality (e.g., 41.55 vs. 41.49 dB PSNR at 1080p) while decoding 3.6x-4.7x faster with up to 11x less memory on a single H100 GPU. With architecture-aware inference optimizations, the speedup widens to 12x.