DDiT：面向高效扩散 Transformer 的动态图像块调度
DDiT: Dynamic Patch Scheduling for Efficient Diffusion Transformers
把 DiT 固定的图像块划分换成随去噪时间步与内容复杂度变化的动态图像块尺寸，是一种无需训练、即插即用的推理加速手段，可直接套用到 FLUX、Wan 等基于 Transformer 的扩散生成模型上，在几乎不损失生成质量的前提下减少 token 数量。
原文依据 We propose dynamic tokenization, an efficient test-time strategy that varies patch sizes based on content complexity and the denoising timestep.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论扩散 Transformer（DiT）在图像与视频生成上取得了最优性能，但其成功以沉重的计算开销为代价。这种低效在很大程度上源于固定的 token 化过程：在整个去噪阶段始终使用恒定尺寸的图像块，而不考虑内容的复杂度。我们提出动态 token 化，一种高效的测试时策略，根据内容复杂度和去噪时间步改变图像块尺寸。我们的关键洞见是：早期时间步通常需要较粗的图像块来建模全局结构，而后期迭代则需要更细（尺寸更小）的图像块来精修局部细节。在推理阶段，我们的方法在图像与视频生成的各个去噪步骤之间动态重新分配图像块尺寸，在保持感知生成质量的同时大幅降低成本。大量实验证明了该方法的有效性：在 FLUX-1.Dev 和 Wan 2.1 上分别实现了最高 3.52 倍和 3.2 倍的加速，且不损害生成质量与提示词遵循度。
英文原文摘要 1084 字符
Diffusion Transformers (DiTs) have achieved state-of-the-art performance in image and video generation, but their success comes at the cost of heavy computation. This inefficiency is largely due to the fixed tokenization process, which uses constant-sized patches throughout the entire denoising phase, regardless of the content's complexity. We propose dynamic tokenization, an efficient test-time strategy that varies patch sizes based on content complexity and the denoising timestep. Our key insight is that early timesteps usually require coarser patches to model global structure, while later iterations demand finer (smaller-sized) patches to refine local details. During inference, our method dynamically reallocates patch sizes across denoising steps for image and video generation and substantially reduces cost while preserving perceptual generation quality. Extensive experiments demonstrate the effectiveness of our approach: it achieves up to 3.52xand 3.2xspeedup on FLUX-1.Dev and Wan 2.1, respectively, without compromising the generation quality and prompt adherence.