WaDi：权重方向感知蒸馏用于一步图像合成
WaDi: Weight Direction-aware Distillation for One-step Image Synthesis
把蒸馏中权重变化拆成方向与范数两部分，并用可学习低秩旋转矩阵建模方向变化，这一思路可直接迁移到任意扩散/流匹配模型的一步或少步蒸馏，也可作为参数高效微调适配器复用。
原文依据 changes in weight direction significantly exceed those in weight norm, highlighting it as the key factor during distillation
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论尽管 Stable Diffusion（SD）等扩散模型在图像生成上表现优异，但其推理速度缓慢限制了实际部署。近期工作通过将多步扩散蒸馏为一步生成器来加速推理。为更好地理解蒸馏机制，我们分析了一步学生模型与其多步教师模型之间的 U-Net/DiT 权重变化。分析表明，权重方向的变化显著大于权重范数的变化，凸显其是蒸馏过程中的关键因素。受此启发，我们提出权重方向的低秩旋转（LoRaD），一种为一步扩散蒸馏量身定制的参数高效适配器。LoRaD 通过可学习的低秩旋转矩阵来建模这些结构化的方向变化。我们进一步将 LoRaD 集成到变分分数蒸馏（VSD）中，得到权重方向感知蒸馏（WaDi）——一种新的一步蒸馏框架。WaDi 在 COCO 2014 和 COCO 2017 上取得了最优的弗雷歇初始距离分数，同时仅使用 U-Net/DiT 约 10% 的可训练参数。此外，蒸馏得到的一步模型展现出很强的通用性与可扩展性，能很好地泛化到可控生成、关系反演和高分辨率合成等多种下游任务。
英文原文摘要 1367 字符
Despite the impressive performance of diffusion models such as Stable Diffusion (SD) in image generation, their slow inference limits practical deployment. Recent works accelerate inference by distilling multi-step diffusion into one-step generators. To better understand the distillation mechanism, we analyze U-Net/DiT weight changes between one-step students and their multi-step teacher counterparts. Our analysis reveals that changes in weight direction significantly exceed those in weight norm, highlighting it as the key factor during distillation. Motivated by this insight, we propose the Low-rank Rotation of weight Direction (LoRaD), a parameter-efficient adapter tailored to one-step diffusion distillation. LoRaD is designed to model these structured directional changes using learnable low-rank rotation matrices. We further integrate LoRaD into Variational Score Distillation (VSD), resulting in Weight Direction-aware Distillation (WaDi)--a novel one-step distillation framework. WaDi achieves state-of-the-art FID scores on COCO 2014 and COCO 2017 while using only approximately 10% of the trainable parameters of the U-Net/DiT. Furthermore, the distilled one-step model demonstrates strong versatility and scalability, generalizing well to various downstream tasks such as controllable generation, relation inversion, and high-resolution synthesis.