权重的生成式建模：泛化还是记忆？
Generative Modeling of Weights: Generalization or Memorization?
对做生成模型的人是一个警示性评测框架：它给出「复制/插值 vs 真正新样本」的判别思路（与训练 checkpoint 的距离、与加噪/权重集成的基线对比），可直接迁移到图像生成中检测扩散模型是否只是在记忆训练集。
原文依据 we find that these methods synthesize weights largely by memorization: they produce either replicas, or at best simple interpolations, of the training checkpoints
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论生成模型在图像与视频生成上的成功，近来促使人们探索用其合成有效的神经网络权重。这类方法将训练好的神经网络检查点作为训练数据，目标是在推理时生成高性能的神经网络权重。在本工作中，我们考察了该新兴领域中四个具有代表性且广为人知的方法生成新颖模型权重（即不同于训练中所见检查点的权重）的能力。与先前工作的主张相反，我们发现这些方法在很大程度上是通过记忆来合成权重的：它们产生的要么是训练检查点的复制品，要么至多是对其的简单插值。在获得既不同又高性能的模型方面，当前方法未能超越简单基线，例如向权重添加噪声或进行简单的权重集成。我们进一步的结果表明，这种记忆可能源于数据有限、模型过参数化，以及对权重数据特有的结构先验利用不足。我们的发现凸显了在新领域中更审慎地设计与评估生成模型的必要性。
英文原文摘要 1188 字符
Generative models, with their success in image and video generation, have recently been explored for synthesizing effective neural network weights. These approaches take trained neural network checkpoints as training data, and aim to generate high-performing neural network weights during inference. In this work, we examine four representative, well-known methods in this emerging area on their ability to generate novel model weights, i.e., weights that are different from the checkpoints seen during training. Contrary to claims in prior work, we find that these methods synthesize weights largely by memorization: they produce either replicas, or at best simple interpolations, of the training checkpoints. Current methods fail to outperform simple baselines, such as adding noise to the weights or taking a simple weight ensemble, in obtaining different and simultaneously high-performing models. Our further results suggest that the memorization potentially resulted from limited data, overparameterized models, and the underuse of structural priors specific to weight data. Our findings highlight the need for more careful design and evaluation of generative models in new domains.