去噪，快与慢：面向图像生成的难度感知自适应采样
Denoising, Fast and Slow: Difficulty-Aware Adaptive Sampling for Image Generation
提出图像块级时间步采样与难度感知自适应计算分配，可复用于扩散/流匹配模型的训练与推理加速，并可与表示对齐、引导方法正交结合。
原文依据 we introduce a timestep sampler that explicitly controls the maximum patch-level information available during training
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论基于扩散和流的模型通常在整个空间上均匀分配计算量，以相同的时间步和函数评估次数更新所有图像块。这种做法虽然方便，却忽略了自然图像的异质性：有些区域容易去噪，而另一些区域则需要更多细化或额外上下文。受此启发，我们探索了图像合成中的图像块级噪声尺度。我们发现，简单地在图像令牌间改变时间步效果不佳，因为这会使模型暴露于推理时不会出现的、信息量过大的训练状态。因此，我们引入了一种时间步采样器，显式控制训练期间可用的最大图像块级信息量，并表明从全局时间步转向图像块级时间步已经能在标准基线之上改进图像生成。通过进一步为模型配备轻量级的逐图像块难度头，我们实现了自适应采样器，能够在最需要的地方动态分配计算量。结合在空间和扩散时间上变化的噪声水平，我们得到了 Patch Forcing (PF) 框架，该框架让较容易的区域先行推进，从而为较难区域提供上下文。PF 在类别条件 ImageNet 上取得了优异结果，与表示对齐和引导方法保持正交，并可扩展到文本到图像合成。我们的结果表明，图像块级去噪调度为自适应图像生成提供了有前景的基础。
英文原文摘要 1472 字符
Diffusion- and flow-based models usually allocate compute uniformly across space, updating all patches with the same timestep and number of function evaluations. While convenient, this ignores the heterogeneity of natural images: some regions are easy to denoise, whereas others benefit from more refinement or additional context. Motivated by this, we explore patch-level noise scales for image synthesis. We find that naively varying timesteps across image tokens performs poorly, as it exposes the model to overly informative training states that do not occur at inference. We therefore introduce a timestep sampler that explicitly controls the maximum patch-level information available during training, and show that moving from global to patch-level timesteps already improves image generation over standard baselines. By further augmenting the model with a lightweight per-patch difficulty head, we enable adaptive samplers that allocate compute dynamically where it is most needed. Combined with noise levels varying over both space and diffusion time, this yields Patch Forcing (PF), a framework that advances easier regions earlier so they can provide context for harder ones. PF achieves superior results on class-conditional ImageNet, remains orthogonal to representation alignment and guidance methods, and scales to text-to-image synthesis. Our results suggest that patch-level denoising schedules provide a promising foundation for adaptive image generation.