分阶段 DMD：基于子区间内分数匹配的少步分布匹配蒸馏
Phased DMD: Few-step Distribution Matching Distillation via Score Matching within Subintervals
把 SNR 区间切分并逐段做分布匹配与分数匹配蒸馏，配合 MoE 提升少步生成器容量，可直接用于把大规模文生图/文生视频扩散模型压缩成少步采样器，同时缓解随机梯度截断带来的多样性下降与运动动态变慢问题。
原文依据 we propose Phased DMD, a multi-step distillation framework that bridges the idea of phase-wise distillation with Mixture-of-Experts (MoE)
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论分布匹配蒸馏（DMD）将基于分数的生成模型蒸馏为高效的单步生成器，且不要求与教师模型的采样轨迹一一对应。然而，单步蒸馏模型容量有限，损害了生成多样性，并在复杂生成任务（例如文生视频任务中生成复杂的物体运动）上导致性能下降。直接将 DMD 扩展为多步蒸馏会增加内存占用与计算深度，从而带来不稳定性和效率下降。尽管已有工作提出随机梯度截断作为潜在解决方案，但我们观察到，这会显著降低文生图生成中的多样性，并减缓视频生成中的运动动态，使性能降至单步模型的水平。为应对这些局限，我们提出分阶段 DMD，这是一种多步蒸馏框架，将分阶段蒸馏的思想与混合专家（MoE）相结合，在降低学习难度的同时增强模型容量。分阶段 DMD 包含两个关键思想：渐进式分布匹配与子区间内的分数匹配。首先，我们的模型将信噪比（SNR）范围划分为若干子区间，逐步将模型精化到更高的 SNR 水平，以更好地捕捉复杂分布。其次，为确保每个子区间内的训练准确，我们为目标推导了严格的数学表述。我们通过蒸馏最优的图像与视频生成模型（包括 Qwen-Image-20B 和 Wan2.2-28B）来验证分阶段 DMD。实验表明，分阶段 DMD 增强了运动动态，提升了视频生成的视觉保真度，并提高了图像生成的输出多样性。我们的代码与模型已发布于 https://x-niper.github.io/projects/Phased-DMD/。
英文原文摘要 1839 字符
Distribution Matching Distillation (DMD) distills score-based generative models into efficient one-step generators, without requiring a one-to-one correspondence with the sampling trajectories of their teachers. Yet, the limited capacity of one-step distilled models compromises generative diversity and degrades performance in complex generative tasks, e.g., generating intricate object motions in text-to-video task. Directly extending DMD to multi-step distillation increases memory usage and computational depth, leading to instability and reduced efficiency. While prior works propose stochastic gradient truncation as a potential solution, we observe that it substantially reduces the generative diversity in text-to-image generation and slows motion dynamics in video generation, reducing performance to the level of one-step models. To address these limitations, we propose Phased DMD, a multi-step distillation framework that bridges the idea of phase-wise distillation with Mixture-of-Experts (MoE), reducing learning difficulty while enhancing model capacity. Phased DMD incorporates two key ideas: progressive distribution matching and score matching within subintervals. First, our model divides the SNR range into subintervals, progressively refining the model to higher SNR levels, to better capture complex distributions. Next, to ensure accurate training within each subinterval, we derive rigorous mathematical formulations for the objective. We validate Phased DMD by distilling state-of-the-art image and video generation models, including Qwen-Image-20B and Wan2.2-28B. Experiments demonstrate that Phased DMD enhances motion dynamics, improves visual fidelity in video generation, and increases output diversity in image generation. Our code and models are available at https://x-niper.github.io/projects/Phased-DMD/.