DDT：解耦扩散 Transformer
DDT: Decoupled Diffusion Transformer
把扩散 Transformer 的编码器与速度解码器解耦，让编码器专注低频语义、解码器专注高频细节，可直接迁移到任何 DiT 类骨干（如 DiT、SiT、UViT）以加速收敛；其相邻去噪步共享自条件的统计动态规划策略，也为少步推理与缓存复用提供了可复用的调度思路。
原文依据 a decoupled design of a dedicated condition encoder for semantic extraction alongside a specialized velocity decoder
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论扩散 Transformer 已展现出卓越的生成质量，但需要更长的训练迭代和大量的推理步数。在每一次去噪步骤中，扩散 Transformer 将含噪输入编码以提取低频语义成分，随后用相同的模块解码高频成分。这一方案造成了内在的优化困境：编码低频语义必然要求削减高频成分，从而在语义编码与高频解码之间形成张力。为解决这一挑战，我们提出一种新的解耦扩散 Transformer（DDT），其解耦设计包含一个专门用于语义提取的条件编码器以及一个专门的速度解码器。实验表明，随着模型规模增大，更充分的编码器会带来性能提升。在 ImageNet 256×256 上，我们的 DDT-XL/2 取得了 1.31 弗雷歇初始距离的新最优性能（训练收敛速度相比此前的扩散 Transformer 快近 4 倍）。在 ImageNet 512×512 上，我们的 DDT-XL/2 取得了 1.28 的新最优弗雷歇初始距离。此外，作为一个有益的副产品，我们的解耦架构通过允许相邻去噪步之间共享自条件而提升了推理速度。为将性能损失降到最低，我们提出一种新的统计动态规划方法来识别最优的共享策略。
英文原文摘要 1455 字符
Diffusion transformers have demonstrated remarkable generation quality, albeit requiring longer training iterations and numerous inference steps. In each denoising step, diffusion transformers encode the noisy inputs to extract the lower-frequency semantic component and then decode the higher frequency with identical modules. This scheme creates an inherent optimization dilemma: encoding low-frequency semantics necessitates reducing high-frequency components, creating tension between semantic encoding and high-frequency decoding. To resolve this challenge, we propose a new \color ddtD ecoupled \color ddtD iffusion \color ddtT ransformer(\color ddtDDT ), with a decoupled design of a dedicated condition encoder for semantic extraction alongside a specialized velocity decoder. Our experiments reveal that a more substantial encoder yields performance improvements as model size increases. For ImageNet 256x256, Our DDT-XL/2 achieves a new state-of-the-art performance of 1.31 FID (nearly 4xfaster training convergence compared to previous diffusion transformers). For ImageNet 512x512, Our DDT-XL/2 achieves a new state-of-the-art FID of 1.28. Additionally, as a beneficial by-product, our decoupled architecture enhances inference speed by enabling the sharing self-condition between adjacent denoising steps. To minimize performance degradation, we propose a novel statistical dynamic programming approach to identify optimal sharing strategies.