无需数据的流映射蒸馏
Flow Map Distillation Without Data
把流映射蒸馏的数据依赖替换为仅从先验分布采样,并显式校正学生沿采样路径的累积误差,可直接用于把任意预训练扩散/流模型压到 1 步采样而不需要外部数据集。
原文依据 we explore a data-free alternative that samples only from the prior distribution
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论最优的流模型能够取得出色的生成质量,但采样过程缓慢且需要迭代。为加速这一过程,可以从预训练教师模型中蒸馏出流映射,而该过程通常需要从外部数据集中采样。我们认为,这种对数据的依赖引入了教师-数据不匹配的根本风险,因为静态数据集可能无法完整、甚至可能错误地刻画教师模型全部的生成能力。这促使我们思考:成功的流映射蒸馏是否真的必须依赖数据。在本工作中,我们探索了一种无数据的替代方案,即仅从先验分布中采样——该分布由构造方式保证是教师模型必然遵循的分布,从而完全规避了不匹配风险。为证明这一思路的实际可行性,我们提出了一个系统性框架,它学习预测教师的采样路径,同时主动校正自身的累积误差以保证高保真度。我们的方法超越了所有基于数据的同类方法,并以显著优势确立了新的最优性能。具体而言,从 SiT-XL/2+REPA 蒸馏时,我们的方法在 ImageNet 256x256 上取得 1.45 的弗雷歇初始距离,在 ImageNet 512x512 上取得 1.49,且均只需 1 步采样。我们希望本工作能为加速生成模型建立更稳健的范式,并推动无数据流映射蒸馏的更广泛采用。
英文原文摘要 1465 字符
State-of-the-art flow models achieve remarkable quality but require slow, iterative sampling. To accelerate this, flow maps can be distilled from pre-trained teachers, a procedure that conventionally requires sampling from an external dataset. We argue that this data-dependency introduces a fundamental risk of Teacher-Data Mismatch, as a static dataset may provide an incomplete or even misaligned representation of the teacher's full generative capabilities. This leads us to question whether this reliance on data is truly necessary for successful flow map distillation. In this work, we explore a data-free alternative that samples only from the prior distribution--a distribution the teacher is guaranteed to follow by construction, thereby circumventing the mismatch risk entirely. To demonstrate the practical viability of this philosophy, we introduce a principled framework that learns to predict the teacher's sampling path while actively correcting for its own compounding errors to ensure high fidelity. Our approach surpasses all data-based counterparts and establishes a new state-of-the-art by a significant margin. Specifically, distilling from SiT-XL/2+REPA, our method reaches an impressive FID of 1.45 on ImageNet 256x256, and 1.49 on ImageNet 512x512, both with only 1 sampling step. We hope our work establishes a more robust paradigm for accelerating generative models and motivates the broader adoption of flow map distillation without data.