生成模型与扩散 · CVPR2026 ·2026.06.03

Flash-DMD:以高效蒸馏与联合强化学习实现高保真少步图像生成

Flash-DMD: Towards High-Fidelity Few-Step Image Generation with Efficient Distillation and Joint Reinforcement Learning

Guanjie Chen、Shirui Huang、Yifu Sun、Kai Liu、Jianchen Zhu、Xiaoye Qu 等 8 人

与你有关的部分

把持续进行的时间步蒸馏损失当作正则项与 RL 目标联合训练,可稳定少步扩散模型的偏好微调并抑制奖励黑客;该联合训练范式可直接迁移到任何需要人类偏好对齐的少步图像生成器。

原文依据 the stable, well-defined loss from the ongoing distillation acts as a powerful regularizer, effectively stabilizing the RL training process
可迁移到 少步图像生成扩散模型蒸馏文生图偏好对齐

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

扩散模型已成为一类领先的生成模型,但其迭代采样过程仍然计算代价高昂。时间步蒸馏是加速生成的一种有前景的技术,但通常需要大量训练,并会导致图像质量下降。此外,使用强化学习(RL)针对特定目标(如美学吸引力或用户偏好)对这些蒸馏模型进行微调,以极不稳定著称,且容易陷入奖励黑客。在本工作中,我们提出 Flash-DMD,一个能够实现蒸馏快速收敛并结合 RL 联合精炼的新框架。具体而言,我们首先提出一种高效的时间步感知蒸馏策略,显著降低训练成本并提升真实感,其训练成本仅为 DMD2 的 2.1% 却优于后者。其次,我们引入一种联合训练方案,在继续同时进行时间步蒸馏训练的同时,以 RL 目标对模型进行微调。我们证明,持续进行的蒸馏所提供的稳定、良定义的损失可充当强大的正则项,有效稳定 RL 训练过程并防止策略崩溃。在基于分数的模型与流匹配模型上的大量实验表明,我们提出的 Flash-DMD 不仅收敛显著更快,而且在少步采样情形下达到最优的生成质量,在视觉质量、人类偏好和文本-图像对齐指标上均优于现有方法。我们的工作为训练高效、高保真且稳定的生成模型提供了一种有效范式。代码见补充材料。

英文原文摘要 1671 字符

Diffusion Models have emerged as a leading class of generative models, yet their iterative sampling process remains computationally expensive. Timestep distillation is a promising technique to accelerate generation, but it often requires extensive training and leads to image quality degradation. Furthermore, fine-tuning these distilled models for specific objectives, such as aesthetic appeal or user preference, using Reinforcement Learning (RL) is notoriously unstable and easily falls into reward hacking. In this work, we introduce Flash-DMD, a novel framework that enables fast convergence with distillation and joint RL-based refinement.Specifically, we first propose an efficient timestep-aware distillation strategy that significantly reduces training cost with enhanced realism, outperforming DMD2 with only 2.1% its training cost. Second, we introduce a joint training scheme where the model is fine-tuned with an RL objective while the timestep distillation training continues simultaneously. We demonstrate that the stable, well-defined loss from the ongoing distillation acts as a powerful regularizer, effectively stabilizing the RL training process and preventing policy collapse. Extensive experiments on score-based and flow matching models show that our proposed Flash-DMD not only converges significantly faster but also achieves state-of-the-art generation quality in the few-step sampling regime, outperforming existing methods in visual quality, human preference, and text-image alignment metrics. Our work presents an effective paradigm for training efficient, high-fidelity, and stable generative models. Codes are attached in the supplementary.

标签

会议 CVPR2026
页码 6010-6020
Bibkey
来源 CVF Open Access