DPAR：面向高效自回归视觉生成的动态图像块划分
DPAR: Dynamic Patchification for Efficient Autoregressive Visual Generation
用轻量无监督自回归模型的下一词元预测熵作为信息量准则，动态合并词元为大图像块，可直接迁移到任何 decoder-only 视觉自回归生成器中以降低注意力计算并加速收敛。
原文依据 next-token prediction entropy from a lightweight and unsupervised autoregressive model provides a reliable criterion for merging tokens into larger patches
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论仅解码器自回归图像生成通常依赖固定长度的词元化方案，其词元数量随分辨率呈二次增长，大幅增加了注意力的计算与内存需求。本文提出 DPAR，一种新颖的仅解码器自回归模型，它将图像词元动态聚合为数量可变的图像块以实现高效图像生成。本工作是首个证明来自轻量且无监督自回归模型的下一词元预测熵能够作为可靠准则、依据信息量将词元合并为更大图像块的研究。DPAR 对标准解码器架构仅做最小改动，确保与多模态生成框架兼容，并将更多计算分配给高信息量图像区域的生成。进一步地，我们证明使用动态尺寸图像块进行训练所得到的表征对图像块边界具有鲁棒性，使 DPAR 在推理时能够扩展到更大的图像块尺寸。在 ImageNet 256 与 384 生成分辨率上，DPAR 分别将词元数量减少 1.81 倍与 2.06 倍，从而使训练成本的浮点运算量最多降低 40.4%。此外，相较基线模型，本方法展现出更快的收敛速度，并将弗雷歇初始距离最多改善 29.6%。
英文原文摘要 1301 字符
Decoder-only autoregressive image generation typically relies on fixed-length tokenization schemes whose token counts grow quadratically with resolution, substantially increasing the computational and memory demands of attention. We present DPAR, a novel decoder-only autoregressive model that dynamically aggregates image tokens into a variable number of patches for efficient image generation. Our work is the first to demonstrate that next-token prediction entropy from a lightweight and unsupervised autoregressive model provides a reliable criterion for merging tokens into larger patches based on information content. DPAR makes minimal modifications to the standard decoder architecture, ensuring compatibility with multimodal generation frameworks and allocating more compute to the generation of high-information image regions. Further, we demonstrate that training with dynamically sized patches yields representations that are robust to patch boundaries, allowing DPAR to scale to larger patch sizes at inference. DPAR reduces token count by 1.81x and 2.06x on ImageNet 256 and 384 generation resolution, respectively, leading to a reduction of up to 40.4% FLOPs in training costs. Further, our method exhibits faster convergence and improves FID by up to 29.6% relative to baseline models.