面向扩散 Transformer 的区域自适应采样
Region-Adaptive Sampling for Diffusion Transformers
对做扩散 Transformer 推理加速的人，这篇给出了一个可直接套用的免训练策略：用上一步输出判定 DiT 关注的语义区域，只对这些区域对应的词元做前向更新，其余词元复用缓存噪声，从而在 Stable Diffusion 3、Lumina-Next-T2I 上取得 2 倍以上加速；该「按区域分配更新比例」的思路可迁移到视频 DiT 与可控生成管线中。
原文依据 RAS updates only focused regions while reusing cached noise for others, with focus determined from the previous step's output
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论扩散模型已成为跨领域生成任务的最优方法，但其对顺序前向传播的依赖限制了实时性能。此前的加速方法主要着眼于减少采样步数或复用中间结果。利用扩散 Transformer 处理可变词元数量的灵活性，我们提出 RAS，一种免训练的采样策略，依据模型关注度动态地为图像不同区域分配不同的更新比例。我们的关键观察是：在每一步中，DiT 会集中于语义上有意义的区域，且这些区域在连续步骤之间表现出很强的连续性。基于此，RAS 仅更新被关注的区域，而对其他区域复用缓存的噪声，关注区域由上一步的输出确定。在 Stable Diffusion 3 与 Lumina-Next-T2I 上评估，RAS 分别取得最高 2.36 倍与 2.51 倍的加速，且质量损失极小。这表明了迈向更高效的扩散 Transformer 以实现实时生成的一个切实步骤。
英文原文摘要 1040 字符
Diffusion models (DMs) have become the state-of-the-art for generative tasks across domains, but their reliance on sequential forward passes limits real-time performance. Prior acceleration methods mainly reduce sampling steps or reuse intermediate results. Leveraging the flexibility of Diffusion Transformers (DiTs) to handle variable token counts, we propose RAS, a training-free sampling strategy that dynamically assigns different update ratios to image regions based on model focus. Our key observation is that at each step, DiTs concentrate on semantically meaningful areas, and these regions exhibit strong continuity across consecutive steps. Exploiting this, RAS updates only focused regions while reusing cached noise for others, with focus determined from the previous step's output. Evaluated on Stable Diffusion 3 and Lumina-Next-T2I, RAS achieves up to 2.36x and 2.51x speedups, respectively, with minimal quality loss. This demonstrates a practical step toward more efficient diffusion transformers for real-time generation.