平滑分数函数以增强扩散模型的泛化能力
Smoothing the Score Function to Enhance Generalization in Diffusion Models
为扩散模型提供了一套解释记忆化（memorization）的理论框架，并给出两种可直接插入训练流程的改进手段——噪声去条件化（Noise Unconditioning）与温度平滑（Temperature Smoothing），可用于缓解采样崩溃、提升生成泛化性，对做扩散模型训练与采样的人可直接复用。
原文依据 the empirical score function is a weighted sum of the score functions of Gaussian distributions, in which the weights are sharp softmax functions
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论扩散模型取得了显著的生成质量，却面临一个被称为记忆化的根本性挑战，即生成的样本可能完全复制训练样本。我们构建了一个理论框架来解释这一现象，证明经验分数函数（对应于经验分布的分数函数）是高斯分布分数函数的加权和，其中权重为尖锐的 softmax 函数。这种结构使得单个训练样本主导分数函数，从而导致采样崩溃。在实践中，用神经网络近似经验分数函数可以部分缓解该问题并提升泛化能力。我们的理论框架解释了原因：在训练中，神经网络学习到该加权和的一个更平滑的近似，使采样过程受局部流形而非单个点的影响。基于这一洞见，我们提出两种新方法以进一步增强泛化能力：（1）噪声去条件化使每个训练样本能够自适应地确定其分数函数权重，以增强更多训练样本的作用，从而防止单点主导并缓解崩溃；（2）温度平滑引入一个显式参数来控制平滑程度。通过提高 softmax 权重中的温度，我们自然地降低了任一单个训练样本的主导性并缓解记忆化。在多个数据集上的实验验证了我们的理论分析，并证明了所提方法在保持高生成质量的同时提升泛化能力的有效性。
英文原文摘要 1678 字符
Diffusion models achieve remarkable generation quality, yet face a fundamental challenge known as memorization, where generated samples can replicate training samples exactly. We develop a theoretical framework to explain this phenomenon by showing that the empirical score function (the score function corresponding to the empirical distribution) is a weighted sum of the score functions of Gaussian distributions, in which the weights are sharp softmax functions. This structure causes individual training samples to dominate the score function, resulting in sampling collapse. In practice, approximating the empirical score function with a neural network can partially alleviate this issue and improve generalization. Our theoretical framework explains why: In training, the neural network learns a smoother approximation of the weighted sum, allowing the sampling process to be influenced by local manifolds rather than single points. Leveraging this insight, we propose two novel methods to further enhance generalization: (1) Noise Unconditioning enables each training sample to adaptively determine its score function weight to increase the effect of more training samples, thereby preventing single-point dominance and mitigating collapse. (2) Temperature Smoothing introduces an explicit parameter to control the smoothness. By increasing the temperature in the softmax weights, we naturally reduce the dominance of any single training sample and mitigate memorization. Experiments across multiple datasets validate our theoretical analysis and demonstrate the effectiveness of the proposed methods in improving generalization while maintaining high generation quality.