在极小预算下加速扩散模型训练：基于数据集蒸馏的视角
Accelerating Diffusion Model Training under Minimal Budgets: A Condensation-Based Perspective
把数据集蒸馏引入扩散模型训练：用扩散难度分数加区间采样挑出极小子集，再用语义/视觉表征增强条件信号，可在 0.8% ImageNet 上把 SiT-XL/2 训练步数压到 40k，为低预算下的扩散训练提供可直接复用的数据侧加速方案。
原文依据 D2C attains a FID of 4.3 in just 40k steps using only 0.8% of the training images
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论扩散模型在广泛的生成任务上取得了显著性能，但从零开始训练它们以资源密集著称，通常需要数百万张训练图像和大量 GPU 天。受这一瓶颈的数据中心视角启发，我们采用基于蒸馏的视角：给定一个大型训练集，目标是构造一个小得多的蒸馏数据集，使其仍能在极少数据和计算预算下支撑训练出强扩散模型。为落实这一视角，我们提出扩散数据集蒸馏（D2C），一个包含选择与附加两个阶段的框架。在选择阶段，结合扩散难度分数与区间采样，从原始数据中识别出一个紧凑且信息量大的训练子集。在该子集基础上，附加阶段通过为每张选中的图像增强丰富的语义与视觉表征，进一步强化条件信号。据我们所知，D2C 是首个系统研究扩散模型数据集蒸馏的框架，而此前的蒸馏方法主要面向判别式架构。在占 ImageNet 0.8% 到 8% 的数据预算、多种模型架构与图像分辨率上的大量实验表明，D2C 在保持高生成质量的同时大幅加速了扩散模型训练。在 ImageNet 256x256 上使用 SiT-XL/2，D2C 仅用 0.8% 的训练图像、在 40k 步内即达到 4.3 的弗雷歇初始距离，相比原始 SiT-XL/2 和 SiT-XL/2 + REPA 分别约快 233 倍和 100 倍。代码：https://github.com/xie-lab-ml/Diffusion-Dataset-Condensation。
英文原文摘要 1699 字符
Diffusion models have achieved remarkable performance on a wide range of generative tasks, yet training them from scratch is notoriously resource-intensive, typically requiring millions of training images and many GPU days. Motivated by a data-centric view of this bottleneck, we adopt a condensation-based perspective: given a large training set, the goal is to construct a much smaller condensed dataset that still supports training strong diffusion models under minimal data and compute budgets. To operationalize this perspective, we introduce Diffusion Dataset Condensation (D2C), a two-phase framework comprising Select and Attach. In the Select phase, a diffusion difficulty score combined with interval sampling is used to identify a compact, informative training subset from the original data. Building on this subset, the Attach phase further strengthens the conditional signals by augmenting each selected image with rich semantic and visual representations. To our knowledge, D2C is the first framework that systematically investigates dataset condensation for diffusion models, whereas prior condensation methods have mainly targeted discriminative architectures. Extensive experiments across data budgets from 0.8% to 8% of ImageNet, model architectures, and image resolutions demonstrate that D2C dramatically accelerates diffusion model training while preserving high generative quality. On ImageNet 256x256 with SiT-XL/2, D2C attains a FID of 4.3 in just 40k steps using only 0.8% of the training images, corresponding to about 233x and 100x faster training than vanilla SiT-XL/2 and SiT-XL/2 + REPA, respectively. Code: https://github.com/xie-lab-ml/Diffusion-Dataset-Condensation.