可学习性引导的扩散用于数据集蒸馏
Learnability-Guided Diffusion for Dataset Distillation
提出可学习性引导的扩散生成框架，通过课程式增量合成数据减少冗余，可复用于任何需要高效训练数据合成的视觉任务，如图像分类、目标检测的数据压缩。
原文依据 We propose learnability-driven dataset distillation, which constructs synthetic datasets incrementally through successive stages.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论在大规模数据集上训练机器学习模型既昂贵又耗时。数据集蒸馏通过创建一个小的合成数据集来达到与完整数据集相同的性能，从而解决这一问题。近期方法使用扩散模型生成蒸馏数据，或通过促进多样性，或通过匹配训练梯度。然而，现有方法产生冗余的训练信号，即样本传达重叠的信息。经验上，蒸馏数据集的不相交子集捕获了80-90%的重叠信号。这种冗余源于优化视觉多样性或平均训练动态而未考虑样本间的相似性，导致数据集中多个样本共享相似信息而非互补知识。我们提出可学习性驱动的数据集蒸馏，通过连续阶段增量构建合成数据集。从小集合开始，我们训练一个模型，并基于可学习性分数生成新样本，该分数识别当前模型能学习的内容，从而创建自适应课程。我们引入可学习性引导扩散（LGD），它平衡当前模型的训练效用与参考模型下的有效性，以生成课程对齐的样本。我们的方法将冗余减少39.1%，促进训练阶段间的专业化，并在ImageNet-1K（60.1%）、ImageNette（87.2%）和ImageWoof（72.9%）上取得最优结果。
英文原文摘要 1472 字符
Training machine learning models on massive datasets is expensive and time-consuming. Dataset distillation addresses this by creating a small synthetic dataset that achieves the same performance as the full dataset. Recent methods use diffusion models to generate distilled data, either by promoting diversity or matching training gradients. However, existing approaches produce redundant training signals, where samples convey overlapping information. Empirically, disjoint subsets of distilled datasets capture 80-90% overlapping signals. This redundancy stems from optimizing visual diversity or average training dynamics without accounting for similarity across samples, leading to datasets where multiple samples share similar information rather than complementary knowledge. We propose learnability-driven dataset distillation, which constructs synthetic datasets incrementally through successive stages. Starting from a small set, we train a model and generate new samples guided by learnability scores that identify what the current model can learn from, creating an adaptive curriculum. We introduce Learnability-Guided Diffusion (LGD), which balances training utility for the current model with validity under a reference model to generate curriculum-aligned samples. Our approach reduces redundancy by 39.1%, promotes specialization across training stages, and achieves state-of-the-art results on ImageNet-1K (60.1%), ImageNette (87.2%), and ImageWoof (72.9%).