从草图到湿壁画:渐进分辨率的 Efficient 扩散 Transformer
From Sketch to Fresco: Efficient Diffusion Transformer with Progressive Resolution
把「低分辨率起草 + 高分辨率精修」做成跨阶段对齐同一目标的免训练采样框架,可直接套到 FLUX、HunyuanVideo 等 DiT 上,并与蒸馏、量化、特征缓存正交叠加,是降低迭代采样成本的可复用加速手段。
原文依据 Fresco achieves near-lossless acceleration across diverse domains and models, including 10 times speedup on FLUX
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论扩散 Transformer 取得了令人印象深刻的生成质量,但由于迭代采样而仍然计算开销高昂。近来,动态分辨率采样作为一种有前景的加速技术出现,它通过降低早期采样步骤的分辨率来实现加速。然而,现有方法在每次分辨率切换时依赖启发式的重加噪,所注入的噪声破坏了跨阶段一致性,并迫使模型重新学习全局结构。此外,这些方法不加区分地一次性上采样整个潜空间,而不检查哪些区域实际已收敛,从而导致误差累积和可见伪影。因此,我们提出 Fresco,一个动态分辨率框架,它通过渐进式上采样在各阶段之间统一重加噪与全局结构,既保留了低分辨率起草的效率,又保留了高分辨率精修的保真度,且所有阶段都对齐到同一最终目标。Fresco 在不同领域和模型上实现了近乎无损的加速,包括在 FLUX 上实现 10 倍加速、在 HunyuanVideo 上实现 5 倍加速,同时与蒸馏、量化和特征缓存保持正交,在与蒸馏模型结合时达到 22 倍加速。
英文原文摘要 1236 字符
Diffusion Transformers achieve impressive generative quality but remain computationally expensive due to iterative sampling. Recently, dynamic resolution sampling has emerged as a promising acceleration technique by reducing the resolution of early sampling steps. However, existing methods rely on heuristic re-noising at every resolution transition, injecting noise that breaks cross-stage consistency and forces the model to relearn global structure. In addition, these methods indiscriminately upsample the entire latent space at once without checking which regions have actually converged, causing accumulated errors, and visible artifacts. Therefore, we propose Fresco, a dynamic resolution framework that unifies re-noise and global structure across stages with progressive upsampling, preserving both the efficiency of low-resolution drafting and the fidelity of high-resolution refinement, with all stages aligned toward the same final target. Fresco achieves near-lossless acceleration across diverse domains and models, including 10 times speedup on FLUX, and 5 times on HunyuanVideo, while remaining orthogonal to distillation, quantization and feature caching, reaching 22 times speedup when combined with distilled models.