2ndMatch：通过二阶雅可比匹配微调剪枝后的扩散模型
2ndMatch: Finetuning Pruned Diffusion Models via Second-Order Jacobian Matching
把稠密教师模型对扰动的敏感度（J^⊤J）作为迁移信号来微调剪枝后的扩散模型，是一种与架构无关的通用微调手段，可直接套用到 U-Net 或 Transformer 扩散模型的压缩流程中，也可迁移到其他需要从教师模型迁移二阶信息的蒸馏场景。
原文依据 a general-purpose finetuning framework that introduces a 2nd-order Jacobian (J^ \top J) Matching loss
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论扩散模型在计算机视觉的各类生成任务中取得了显著性能，但其高昂的计算开销仍是部署的主要障碍。模型剪枝为降低推理开销、获得轻量模型提供了有前景的途径。然而，剪枝会因容量减少而导致质量下降。现有剪枝方法的一个关键局限在于，剪枝后的模型仍使用与稠密模型相同的目标（去噪分数匹配）进行微调。由于微调期间稠密模型是可获取的，因此有必要采用更有效的方式实现从稠密模型到剪枝模型的知识迁移。受此启发，我们提出 2ndMatch（2ndM），一个通用的微调框架，其引入受有限时间李雅普诺夫指数启发的二阶雅可比（J^⊤J）匹配损失。2ndM 通过可扩展的随机投影，教会剪枝模型模仿稠密教师模型的敏感度，即如何随时间对微小扰动做出响应。该框架与架构无关，适用于基于 U-Net 和基于 Transformer 的扩散模型。在 CIFAR-10、CelebA、LSUN、ImageNet 和 MSCOCO 上的实验表明，2ndM 缩小了剪枝模型与稠密模型之间的性能差距，显著提升了输出质量。
英文原文摘要 1287 字符
Diffusion models achieve remarkable performance across diverse generative tasks in computer vision, but their high computational cost remains a major barrier to deployment. Model pruning offers a promising way to reduce inference cost and enable lightweight models. However, pruning leads to quality drop due to reduced capacity. A key limitation of existing pruning approaches is that pruned models are finetuned using the same objective as the dense model (denoising score matching). Since the dense model is accessible during finetuning, it warrants a more effective approach for knowledge transfer from the dense to the pruned model. Motivated by this, we propose 2ndMatch(2ndM), a general-purpose finetuning framework that introduces a 2nd-order Jacobian (J^ \top J) Matching loss inspired by Finite-Time Lyapunov Exponents. 2ndM teaches the pruned model to mimic the sensitivity of the dense teacher, i.e., how to respond to small perturbations over time, through scalable random projections. The framework is architecture-agnostic and applies to both U-Net- and Transformer-based diffusion models. Experiments on CIFAR-10, CelebA, LSUN, ImageNet, and MSCOCO demonstrate that 2ndM reduces the performance gap between pruned and dense models, substantially improving output quality.