LIFT 与 PLACE：一个简单、稳定且有效的轻量扩散模型知识蒸馏框架
LIFT and PLACE: A Simple, Stable, and Effective Knowledge Distillation Framework for Lightweight Diffusion Models
把教师扩散模型复杂的去噪轨迹拆成「粗对齐 + 精修」两阶段，并用按误差分组的分段局部自适应系数做空间非均匀引导，可直接用于把大 U-Net/DiT 扩散模型蒸馏到极小参数量学生模型（1.3M 参数仍收敛到 FID 15.73），是少步/轻量扩散部署的可复用训练配方。
原文依据 under extreme compression with a 1.3M-parameter student (only 1.6% of the teacher), conventional KD fails to provide sufficient guidance
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论我们证明，在扩散模型的知识蒸馏中，教师网络因其容量显著更大而具有高度复杂的去噪过程，这给学生模型忠实模仿带来了重大挑战。为解决该问题，我们提出一个由基于线性拟合的蒸馏（LIFT）与分段局部自适应系数估计（PLACE）组成的由粗到细蒸馏框架。首先，LIFT 将目标分解为「粗」对齐与「细」精修，学生模型先在粗对齐上训练，再进入困难精修阶段。其次，PLACE 扩展 LIFT 以处理空间非均匀误差，它将输出按误差划分为若干组，从而提供局部自适应的引导。实验表明，LIFT 与 PLACE 在扩散空间（图像/潜空间）、骨干网络（U-Net/DiT）、任务（无条件/条件）、数据集上均有效，甚至可扩展到 MMDiT（SD3）等基于流的模型。此外，在仅有 130 万参数学生模型（仅为教师的 1.6%）的极端压缩下，传统知识蒸馏无法提供足够的引导以稳定训练，弗雷歇初始距离常退化到 50-200 以上，而我们的方法仍能稳定收敛并达到 15.73 的弗雷歇初始距离。
英文原文摘要 1262 字符
We demonstrate that in knowledge distillation for diffusion models, the teacher network's highly complex denoising process--stemming from its substantially larger capacity--poses a significant challenge for the student model to faithfully mimic. To address this problem, we propose a coarse-to-fine distillation framework with LInear FiTting-based distillation (LIFT) and Piecewise Local Adaptive Coefficient Estimation (PLACE). First, LIFT decomposes the objective into a "coarse" alignment and a "fine" refinement. The student is then trained on coarse alignment before proceeding to hard refinement. Second, PLACE extends LIFT to address spatially non-uniform errors by partitioning outputs into error-based groups, providing locally adaptive guidance. Our experiments show that LIFT and PLACE is effective across diffusion spaces (image/latent), backbones (U-Net/DiT), tasks (unconditional/conditional), datasets, and even extends to flow-based models such as MMDiT (SD3). Furthermore, under extreme compression with a 1.3M-parameter student (only 1.6% of the teacher), conventional KD fails to provide sufficient guidance for stable training, with FID scores often degrading to 50-200+, but our method remains stably convergent and achieves an FID of 15.73.