ResCa：面向扩散 Transformer 加速的残差缓存
ResCa: Residual Caching for Diffusion Transformers Acceleration
把「代理去噪」思路用于扩散 Transformer 的推理加速：在每个轨迹聚类内只对一个代理词元做真实去噪，再用其多阶残差模拟其余词元的去噪，从而在保持自洽且更新的去噪轨迹下大幅削减 GFLOPs，可直接套用到 DiT/FLUX/视频生成模型的采样环节。
原文依据 perform true denoising on only one proxy token within each trajectory-based cluster
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论扩散 Transformer 在高品质图像与视频生成上取得了显著进展，但其计算开销仍是重大挑战。现有的基于词元削减的加速技术，如缓存与合并，试图从时序与空间两个角度降低这一开销，但往往因引入未更新或非自去噪的方向而损害生成质量。本文提出残差缓存（ResCa），一个新颖的免训练框架，引入代理去噪视角以克服上述局限。ResCa 在保持去噪轨迹既自洽又更新的同时实现加速。其核心思想是：在每个基于轨迹的聚类中仅对一个代理词元执行真实去噪，并利用其计算出的多阶残差指导其余所有词元的模拟去噪。ResCa 可无缝集成到多种扩散模型中，包括 DiT、FLUX 与 HunyuanVideo。大量定量与定性实验证明了本方法的有效性，在 FLUX 上实现了最高 5.5 倍的 GFLOPs 加速，同时保持近乎无损的生成质量。项目主页：https://fanghaipeng.github.io/ResCa。
英文原文摘要 1251 字符
Diffusion transformers have achieved remarkable progress in high-quality image and video generation, but their computational overhead remains a significant challenge. Existing token reduction-based acceleration techniques, such as caching and merging, attempt to reduce this cost from both temporal and spatial perspectives, but often compromise generation quality by introducing non-updated or non-self denosing directions. In this paper, we propose Residual Caching (ResCa), a novel, training-free framework that introduces a proxy denoising perspective to overcome these limitations. ResCa achieves acceleration while maintaining a denoising trajectory that is both self and updated. The core idea is to perform true denoising on only one proxy token within each trajectory-based cluster, and use its computed multi-order residuals to guide the simulated denoising of all other tokens. ResCa can be seamlessly integrated into various diffusion models, including DiT, FLUX, and HunyuanVideo. Extensive quantitative and qualitative experiments demonstrate the effectiveness of our method, achieving up to a 5.5 times acceleration in GFLOPs while maintaining near-lossless generation quality on FLUX. Project page: https://fanghaipeng.github.io/ResCa.