以扩散 Transformer 自身的内部动态引导其生成
Guiding a Diffusion Transformer with the Internal Dynamics of Itself
提出一种无需额外训练或采样步数的引导方式：训练时对中间层加辅助监督，采样时对中间层与深层输出做外推，可直接替换或叠加 CFG 用于 DiT 类扩散模型，在低训练轮次下显著降低 FID。
原文依据 we proposed a simple yet effective strategy Internal Guidance (IG), which introduces an auxiliary supervision on the intermediate layer during training process
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论扩散模型具有强大的能力，能够刻画完整的（条件）数据分布。然而，由于缺乏足够的训练和数据来学习覆盖低概率区域，模型会因未能生成这些区域对应的高质量图像而受到惩罚。为获得更好的生成质量，无分类器引导（CFG）等引导策略可以在采样阶段将样本引导至高概率区域。但标准 CFG 往往导致样本过度简化或失真。另一方面，用模型自身的劣化版本引导扩散模型这一替代路线，受限于精心设计的退化策略、额外训练以及额外的采样步数。本文提出一种简单而有效的策略——内部引导（IG）：在训练过程中对中间层引入辅助监督，并在采样过程中对中间层与深层的输出进行外推以获得生成结果。这一简单策略在多种基线上均显著提升了训练效率与生成质量。在 ImageNet 256x256 上，SiT-XL/2+IG 在 80 与 800 个 epoch 时分别取得 FID=5.31 与 FID=1.75。更令人印象深刻的是，LightningDiT-XL/1+IG 取得 FID=1.34，在这些方法之间拉开了较大差距。与 CFG 结合后，LightningDiT-XL/1+IG 取得当前最优的 FID=1.19。
英文原文摘要 1457 字符
The diffusion model presents a powerful ability to capture the entire (conditional) data distribution. However, due to the lack of sufficient training and data to learn to cover low-probability areas, the model will be penalized for failing to generate high-quality images corresponding to these areas. To achieve better generation quality, guidance strategies such as classifier free guidance (CFG) can guide the samples to the high-probability areas during the sampling stage. However, the standard CFG often leads to over-simplified or distorted samples. On the other hand, the alternative line of guiding diffusion model with its bad version is limited by carefully designed degradation strategies, extra training and additional sampling steps. In this paper, we proposed a simple yet effective strategy Internal Guidance (IG), which introduces an auxiliary supervision on the intermediate layer during training process and extrapolates the intermediate and deep layer's outputs to obtain generative results during sampling process. This simple strategy yields significant improvements in both training efficiency and generation quality on various baselines. On ImageNet 256x256, SiT-XL/2+IG achieves FID=5.31 and FID=1.75 at 80 and 800 epochs. More impressively, LightningDiT-XL/1+IG achieves FID=1.34 which achieves a large margin between all of these methods. Combined with CFG, LightningDiT-XL/1+IG achieves the current state-of-the-art FID of 1.19.