自适应谱特征预测用于扩散采样加速
Adaptive Spectral Feature Forecasting for Diffusion Sampling Acceleration
把去噪器潜变量特征视为随时间变化的函数，用切比雪夫多项式加岭回归做全局长程预测，可在不训练的前提下替换现有特征缓存复用策略，直接用于 DiT 类图像/视频扩散模型的少步采样加速。
原文依据 we view the latent features of the denoiser as functions over time and approximate them with Chebyshev polynomials
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论扩散模型已成为高保真图像与视频生成的主流工具，但由于扩散 Transformer 需要大量迭代前向，其推理速度成为关键瓶颈。为减少繁重计算，近期工作采用特征缓存与复用方案，通过在选定的扩散步跳过网络评估、复用前几步的缓存特征来加速。然而，其初步设计仅依赖局部近似，导致在大跨度跳步时误差迅速累积，在高加速比下采样质量下降。在本工作中，我们提出谱扩散特征预测器（Spectrum），一种免训练方法，可实现全局、长程的特征复用并将误差严格受控。具体而言，我们将去噪器的潜变量特征视为随时间变化的函数，并用切比雪夫多项式对其进行近似。我们通过岭回归拟合每个基函数的系数，进而利用该系数预测未来多个扩散步的特征。我们从理论上揭示，该方法具有更优的长时程行为，且其误差界不随步长增大而累积。在多种最优图像与视频扩散模型上的大量实验一致验证了本方法的优越性。值得注意的是，我们在 FLUX.1 上实现了最高 4.79 倍加速，在 Wan2.1-14B 上实现 4.67 倍加速，同时相比基线保持了显著更高的样本质量。
英文原文摘要 1500 字符
Diffusion models have become the dominant tool for high-fidelity image and video generation, yet are critically bottlenecked by their inference speed due to the numerous iterative passes of Diffusion Transformers. To reduce the exhaustive compute, recent works resort to the feature caching and reusing scheme that skips network evaluations at selected diffusion steps by using cached features in previous steps. However, their preliminary design solely relies on local approximation, causing errors to grow rapidly with large skips and leading to degraded sample quality at high speedups. In this work, we propose spectral diffusion feature forecaster (Spectrum), a training-free approach that enables global, long-range feature reuse with tightly controlled error. In particular, we view the latent features of the denoiser as functions over time and approximate them with Chebyshev polynomials. Specifically, we fit the coefficient for each basis via ridge regression, which is then leveraged to forecast features at multiple future diffusion steps. We theoretically reveal that our approach admits more favorable long-horizon behavior and yields an error bound that does not compound with the step size. Extensive experiments on various state-of-the-art image and video diffusion models consistently verify the superiority of our approach. Notably, we achieve up to 4.79x speedup on FLUX.1 and 4.67x speedup on Wan2.1-14B, while maintaining much higher sample quality compared with the baselines.