SenCache:通过敏感性感知缓存加速扩散模型推理
SenCache: Accelerating Diffusion Model Inference via Sensitivity-Aware Caching
提出基于模型输出敏感性的动态缓存策略,可无缝集成到现有视频扩散模型(如 Wan 2.1、CogVideoX)中,在相近计算预算下提升生成质量,为免训练加速提供理论指导。
原文依据 we formalize the caching error through an analysis of the model output sensitivity to perturbations in the denoising inputs
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论扩散模型实现了最优的视频生成质量,但由于大量顺序去噪步骤,其推理仍然昂贵。这推动了加速扩散推理的研究方向不断发展。在免训练加速方法中,缓存通过跨时间步重用先前计算的模型输出来减少计算。现有缓存方法依赖启发式准则来选择缓存/重用时间步,并需要大量调参。我们通过一个原则性的敏感性感知缓存框架来解决这一局限。具体而言,我们通过分析模型输出对去噪输入(即噪声潜变量和时间步)扰动的敏感性来形式化缓存误差,并表明这种敏感性是缓存误差的关键预测因子。基于此分析,我们提出敏感性感知缓存(SenCache),一种动态缓存策略,能够以每个样本为基础自适应地选择缓存时间步。我们的框架为自适应缓存提供了理论基础,解释了先前的经验启发式方法为何部分有效,并将其扩展为动态的、样本特定的方法。在 Wan 2.1、CogVideoX 和 LTX-Video 上的实验表明,在相似的计算预算下,SenCache 比现有缓存方法实现了更好的视觉质量。
英文原文摘要 1339 字符
Diffusion models achieve state-of-the-art video generation quality, but their inference remains expensive due to the large number of sequential denoising steps. This has motivated a growing line of research on accelerating diffusion inference. Among training-free acceleration methods, caching reduces computation by reusing previously computed model outputs across timesteps. Existing caching methods rely on heuristic criteria to choose cache/reuse timesteps and require extensive tuning. We address this limitation with a principled sensitivity-aware caching framework. Specifically, we formalize the caching error through an analysis of the model output sensitivity to perturbations in the denoising inputs, i.e., the noisy latent and the timestep, and show that this sensitivity is a key predictor of caching error. Based on this analysis, we propose Sensitivity-Aware Caching (SenCache), a dynamic caching policy that adaptively selects caching timesteps on a per-sample basis. Our framework provides a theoretical basis for adaptive caching, explains why prior empirical heuristics can be partially effective, and extends them to a dynamic, sample-specific approach. Experiments on Wan 2.1, CogVideoX, and LTX-Video show that SenCache achieves better visual quality than existing caching methods under similar computational budgets.