面向高效视频扩散的内容感知动态图像块划分
Content-Aware Dynamic Patchification for Efficient Video Diffusion
把「固定图像块划分」换成由轻量 router 从 3D VAE latent 预测每个时空区域的 patch 大小，并配可学习的 patchify/unpatchify 层，可直接插到现有 DiT 骨干上；做视频/图像扩散加速时，这是一个无需改架构就能减少词元数的即插即用方案，其注意力引导的显著性对齐损失也可迁移到其他词元压缩方法。
原文依据 a lightweight router predicts patch sizes directly from the latents encoded by 3D Variational Autoencoder (VAE)
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论扩散 Transformer（DiT）在视频生成上表现强劲，但由于稠密的时空词元化，其计算开销高得难以承受。现有工作大多依赖均匀图像块划分，即以固定图像块大小对不重叠的时空区域进行词元化，而不考虑底层内容。这种与内容无关的词元化会造成大量冗余计算，在视觉简单或静止的区域尤为明显。为在保持视频生成质量的同时解决这一低效问题，我们提出 DynaPatch，一个细粒度的动态图像块划分框架，它根据内容复杂度为每个时空区域自适应地选择图像块大小。一个轻量路由器直接从 3D 变分自编码器（VAE）编码得到的潜变量预测图像块大小，并通过扩散损失、注意力引导的显著性对齐损失以及词元预算正则项与扩散模型联合优化。可学习的 patchify/unpatchify 层可与标准 DiT 骨干无缝集成，从而在不改动架构的情况下实现灵活的词元化。实验表明，DynaPatch 能有效减少冗余计算，同时保留细节，在质量损失极小的情况下实现 1.3–1.8 倍加速。在 VBench 上，DynaPatch 在词元减少 30% 时取得 83.42 的总分，显著优于先前的图像块划分与词元剪枝方法。这些结果表明，内容感知的图像块划分为高效且可扩展的视频扩散提供了一个有效方向。项目主页：https://shengli99.github.io/DynaPatch/。
英文原文摘要 1621 字符
Diffusion Transformers (DiTs) achieve strong video generation performance but suffer from prohibitive computation cost due to dense spatiotemporal tokenization. Most existing works rely on uniform patchification, tokenizing non-overlapping spatiotemporal with a fixed patch size regardless of the underlying content. This content-agnostic tokenization results in substantial redundant computation, especially in visually simple or static areas. To address this inefficiency while preserving the video generation quality, we propose DynaPatch, a fine-grained dynamic patchification framework that adaptively selects patch sizes for each spatiotemporal region based on content complexity. A lightweight router predicts patch sizes directly from the latents encoded by 3D Variational Autoencoder (VAE), and is jointly optimized with the diffusion model through diffusion loss, an attention-guided saliency alignment loss, and a token-budget regularizer. Learnable patchify/unpatchify layers integrate seamlessly with standard DiT backbones, allowing flexible tokenization without architectural changes. Experiments demonstrate that DynaPatch can effectively reduce redundant computations while preserving fine details, achieving 1.3-1.8x acceleration with minimal quality degradation. On VBench, DynaPatch attains a Total Score of 83.42 at 30% token reduction, significantly outperforming prior patchification and token pruning approaches. These results indicate that content-aware patchification offers an effective direction for efficient and scalable video diffusion. Project page: https://shengli99.github.io/DynaPatch/.