预测主成分、稳定残差：面向扩散 Transformer 的子空间感知特征缓存
Forecast the Principal, Stabilize the Residual: Subspace-Aware Feature Caching for Diffusion Transformers
对做扩散 Transformer 采样加速的人，可直接把 SVD 分解 + 主/残差子空间差异化预测（主成分 EMA 预测、残差直接复用）作为免训练缓存策略，替换现有均匀缓存，并可与蒸馏、量化、稀疏注意力叠加。
原文依据 decomposes diffusion features via Singular Value Decomposition (SVD), applies exponential moving average (EMA) prediction to the dominant low-rank components
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论扩散 Transformer（DiT）模型在图像与视频生成上取得了前所未有的质量，但其迭代采样过程在计算上仍然代价高昂。为加速推理，特征缓存方法通过跨时间步复用或预测中间表示而兴起。然而，现有缓存方法对所有特征分量一视同仁。我们揭示出 DiT 特征空间包含性质迥异的主子空间与残差子空间，二者具有不同的时序行为：主子空间演化平滑且可预测，而残差子空间表现出剧烈、低能量的振荡，难以被准确预测。基于这一发现，我们提出 SVD-Cache，一种子空间感知的缓存框架，它通过奇异值分解（SVD）分解扩散特征，对占主导的低秩分量施加指数滑动平均（EMA）预测，并直接复用残差子空间。大量实验表明，SVD-Cache 在多种模型与方法上实现近乎无损的加速，包括在 FLUX 与 HunyuanVideo 上取得 5.55 倍加速，并与蒸馏、量化、稀疏注意力等模型加速技术兼容。我们的代码已发布于 https://github.com/BlackMaple1203/SVDCache。
英文原文摘要 1343 字符
Diffusion Transformer (DiT) models have achieved unprecedented quality in image and video generation, yet their iterative sampling process remains computationally prohibitive. To accelerate inference, feature caching methods have emerged by reusing or forecasting intermediate representations across timesteps. However, existing caching approaches treat all feature components uniformly. We reveal that DiT feature spaces contain distinct principal and residual subspaces with divergent temporal behavior: the principal subspace evolves smoothly and predictably, while the residual subspace exhibits volatile, low-energy oscillations that resist accurate prediction. Building on this insight, we propose SVD-Cache, a subspace-aware caching framework that decomposes diffusion features via Singular Value Decomposition (SVD), applies exponential moving average (EMA) prediction to the dominant low-rank components, and directly reuses the residual subspace. Extensive experiments demonstrate that SVD-Cache achieves near-lossless across diverse models and methods, including 5.55xspeedup on FLUX and HunyuanVideo, and compatibility with model acceleration techniques including distillation, quantization and sparse attention. Our code is available at \href https://github.com/BlackMaple1203/SVDCache https://github.com/BlackMaple1203/SVDCache .