基于影响匹配的数据集蒸馏
Dataset Distillation by Influence Matching
用可微的样本级影响估计器替代逐步梯度/轨迹匹配来学习合成数据，视觉研究者可将其作为数据压缩与高效预训练的数据构造手段，并可直接迁移到图文检索等跨模态蒸馏场景。
原文依据 we introduce a fully differentiable, sample-level influence estimator that quantifies parameter shifts from adding or removing data
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论本文从以结果为中心的视角重新审视数据集蒸馏。影响匹配（Inf-Match）不再对齐过程代理（逐步梯度或训练轨迹），而是对齐训练的最终结果：它学习一个紧凑的合成集，使其对收敛参数的影响与完整数据集的影响相匹配。具体而言，我们引入一种完全可微的样本级影响估计器，用于量化增加或删除数据所带来的参数变化，无需耗时的逆 Hessian 乘积，也不依赖凸性假设。该估计器通过展开优化动态并应用一阶泰勒近似，以线性时间运行。随后，我们通过最小化合成集与真实数据集之间影响的失配来学习合成集，从而实现结果对齐而非启发式的过程模仿。Inf-Match 在标准分类基准上取得了最佳准确率。例如，在 Tiny-ImageNet（IPC=10）上，Inf-Match 达到 31.5%，相较 NCFM 提升 4.7%。除分类之外，Inf-Match 还可扩展到 Flickr30K 上的视觉语言蒸馏，优于强过程匹配基线。例如，在 200 至 1000 个合成样本的设置下，我们的方法在图像/文本检索任务上取得了领先的平均表现，比 NCFM 高 2.5%。代码将通过 https://github.com/hrtan/infmatch 发布。
英文原文摘要 1386 字符
We revisit dataset distillation from an outcome-centric perspective. Rather than aligning process surrogates (per-step gradients or training trajectories), Influence Matching (Inf-Match) aligns the final outcome of training: it learns a compact synthetic set whose effect on the converged parameters matches that of the full dataset. Concretely, we introduce a fully differentiable, sample-level influence estimator that quantifies parameter shifts from adding or removing data, without time-consuming inverse-Hessian products or convexity assumptions. The estimator runs in linear time by unrolling the optimization dynamics and applying a first-order Taylor approximation. We then learn the synthetic set by minimizing the mismatch between its influence and that of the real dataset, yielding outcome alignment rather than heuristic process imitation. Inf-Match delivers the best accuracy across standard classification benchmarks. For instance, on Tiny-ImageNet (IPC=10), Inf-Match attains 31.5%, a +4.7% improvement over NCFM. Beyond classification, Inf-Match scales to vision-language distillation on Flickr30K, outperforming strong process-matching baselines. For instance, with 200 to 1000 synthetic samples, our method achieved a leading impressive average on image/text retrieval tasks, higher than NCFM by 2.5%. The code will be released via https://github.com/hrtan/infmatch.