ManifoldGD：面向扩散式数据集蒸馏的免训练层次流形引导
ManifoldGD: Training-Free Hierarchical Manifold Guidance for Diffusion-Based Dataset Distillation
把每个去噪步的引导向量投影到由 IPC 质心邻域估计的局部切空间上，是一种可复用的免训练采样约束手段，可直接迁移到任何需要保持语义一致性的扩散引导或条件生成流程中。
原文依据 we project the mode-alignment vector onto the local tangent space of the estimated latent manifold
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近年来，大规模数据集既阻碍了高效的模型训练，又包含冗余概念。数据集蒸馏旨在合成紧凑数据集，在保留大规模训练集知识的同时大幅降低存储与计算开销。扩散模型的最新进展使得借助预训练生成先验实现免训练蒸馏成为可能，但现有引导策略仍然有限。当前基于得分的方法要么进行无引导去噪，要么依赖朝向实例原型质心（IPC 质心）的简单模态引导，而这些质心往往较为粗糙且次优。我们提出流形引导蒸馏（ManifoldGD），一个免训练的扩散框架，在每一个去噪时间步融入流形一致的引导。该方法采用通过对变分自编码器潜变量特征进行层次化分裂聚类计算得到的 IPC，从而获得一个多尺度的 IPC 核心集，既捕捉粗粒度语义模态，也捕捉类内细粒度变化。利用所提取 IPC 质心的局部邻域，我们为每个扩散去噪时间步构建潜变量流形。在每个去噪步，我们将模态对齐向量投影到所估计潜变量流形的局部切空间上，从而约束生成轨迹保持流形忠实性，同时维持语义一致性。该形式化方法在无需任何模型重训练的情况下提升了代表性、多样性与图像保真度。实证结果表明，在 FID、真实与合成数据集嵌入之间的 l_2 距离以及分类准确率上，该方法相较现有免训练与基于训练的基线均取得一致增益，确立了 ManifoldGD 作为首个几何感知的免训练数据蒸馏框架的地位。
英文原文摘要 1806 字符
In recent times, large datasets hinder efficient model training while also containing redundant concepts. Dataset distillation aims to synthesize compact datasets that preserve the knowledge of large-scale training sets while drastically reducing storage and computation. Recent advances in diffusion models have enabled training-free distillation by leveraging pre-trained generative priors; however, existing guidance strategies remain limited. Current score-based methods either perform unguided denoising or rely on simple mode-based guidance toward instance prototype centroids (IPC centroids), which often are rudimentary and suboptimal. We propose Manifold-Guided Distillation (ManifoldGD), a training-free diffusion-based framework that integrates manifold consistent guidance at every denoising timestep. Our method employs IPCs computed via a hierarchical, divisive clustering of VAE latent features--yielding a multi-scale coreset of IPCs that captures both coarse semantic modes and fine intra-class variability. Using a local neighborhood of the extracted IPC centroids, we create the latent manifold for each diffusion denoising timestep. At each denoising step, we project the mode-alignment vector onto the local tangent space of the estimated latent manifold, thus constraining the generation trajectory to remain manifold-faithful while preserving semantic consistency. This formulation improves representativeness, diversity, and image fidelity without requiring any model retraining. Empirical results demonstrate consistent gains over existing training-free and training-based baselines in terms of FID, l_2 distance among real and synthetic dataset embeddings, and classification accuracy, establishing ManifoldGD as the first geometry-aware training-free data distillation framework.