通过判别性文本表征将一步图像生成从类别标签扩展到文本
Extending One-Step Image Generation from Class Labels to Text via Discriminative Text Representation
把 MeanFlow 这类一步/少步生成框架的条件从类别标签扩展到文本时,关键不是换更强的文本编码器,而是要求文本特征具备足够高的判别性;做少步蒸馏或一步采样时可直接借鉴这一「条件表征判别性」诊断与适配思路。
原文依据 the text feature representations are required to possess sufficiently high discriminability
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论少步生成一直是一个长期目标,近期以 MeanFlow 为代表的一步生成方法取得了显著成果。现有关于 MeanFlow 的研究主要集中于类别到图像的生成。然而,一个直观但尚未探索的方向是将条件从固定的类别标签扩展为灵活的文本输入,从而实现更丰富的内容创作。与有限的类别标签相比,文本条件对模型的理解能力提出了更大挑战,需要将强大的文本编码器有效集成到 MeanFlow 框架中。令人意外的是,尽管引入文本条件看似直接,我们发现采用常规训练策略集成强大的基于 LLM 的文本编码器会得到不理想的效果。为揭示其根本原因,我们进行了细致分析并发现:由于 MeanFlow 生成中的细化步数极为有限(例如仅一步),文本特征表征需要具备足够高的判别性。这也解释了为何离散且易于区分的类别特征在 MeanFlow 框架中表现良好。基于这些洞见,我们采用了一个经证实具备所需语义性质的强大基于 LLM 的文本编码器,并将 MeanFlow 生成过程适配到该框架,首次实现了高效的文本条件合成。此外,我们在广泛使用的扩散模型上验证了该方法,展现出显著的生成性能提升。我们希望本工作能为未来文本条件 MeanFlow 生成的研究提供通用且实用的参考。代码见 https://github.com/AMAP-ML/EMF。
英文原文摘要 1783 字符
Few-step generation has been a long-standing goal, with recent one-step generation methods exemplified by MeanFlow achieving remarkable results. Existing research on MeanFlow primarily focuses on class-to-image generation. However, an intuitive yet unexplored direction is to extend the condition from fixed class labels to flexible text inputs, enabling richer content creation. Compared to the limited class labels, text conditions pose greater challenges to the model's understanding capability, necessitating the effective integration of powerful text encoders into the MeanFlow framework. Surprisingly, although incorporating text conditions appears straightforward, we find that integrating powerful LLM-based text encoders using conventional training strategies results in unsatisfactory performance. To uncover the underlying cause, we conduct detailed analyses and reveal that, due to the extremely limited number of refinement steps in the MeanFlow generation, such as only one step, the text feature representations are required to possess sufficiently high discriminability. This also explains why discrete and easily distinguishable class features perform well within the MeanFlow framework. Guided by these insights, we leverage a powerful LLM-based text encoder validated to possess the required semantic properties and adapt the MeanFlow generation process to this framework, resulting in efficient text-conditioned synthesis for the first time. Furthermore, we validate our approach on the widely used diffusion model, demonstrating significant generation performance improvements. We hope this work provides a general and practical reference for future research on text-conditioned MeanFlow generation. The code is available at \supp https://github.com/AMAP-ML/EMF.