LazyVAR：通过尺度级词元剪枝与并行分组解码加速视觉自回归模型
LazyVAR: Accelerating Visual Autoregressive Models via Scale-wise Token Pruning and Parallel Group Decoding
对做视觉自回归生成的研究者，这篇给出了一个免训练、即插即用的加速范式：用相邻尺度聚合潜变量特征的相似度作为尺度更新指数来指导剪枝，并在大尺度上并行解码跨尺度词元，可直接套用到 VAR/Infinity 类模型的推理管线中。
原文依据 we propose LazyVAR, a training-free and plug-and-play acceleration method for VAR models
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉自回归（VAR）建模将自回归机制从下一词元预测扩展为下一尺度预测，为图像生成引入了新范式，并取得了显著性能。然而，随着尺度增大，词元数量迅速增长，在高分辨率下处理完整词元图的计算开销变得十分昂贵。此外，自回归建模固有的序列性阻碍了跨尺度的并行推理，进一步增加了延迟。为应对这些挑战，我们提出 LazyVAR，一种面向 VAR 模型的免训练、即插即用加速方法。我们的关键观察是：相邻尺度之间聚合潜变量特征的相似度随尺度索引逐步上升，在较大尺度处达到尤其高的数值。我们将该相似度视为尺度更新指数，并将其作为剪枝准则。因此，在较大尺度处可以剪除更多词元以提升效率。进一步地，我们提出并行分组解码，利用大尺度处的高相似度并行解码来自不同尺度的词元，从而进一步加速推理。实验结果表明，所提出的 LazyVAR 在性能退化可忽略的情况下，相较近期的 VAR 模型最高实现 2.94 倍加速，使 Infinity-2B 文本到图像模型能够在单张 RTX 4090 GPU 上于 0.5 秒内生成 1024x1024 分辨率的图像。
英文原文摘要 1453 字符
Visual Autoregressive (VAR) modeling introduces a new paradigm for image generation by extending autoregressive mechanisms from next-token prediction to next-scale prediction, achieving remarkable performance. However, as the number of tokens increases rapidly with scale, processing full token maps at high resolution becomes computationally expensive. In addition, the inherently sequential nature of autoregressive modeling prevents parallel inference across scales, which further increases latency.To address these challenges, we propose LazyVAR, a training-free and plug-and-play acceleration method for VAR models. Our key observation is that the similarity of aggregated latent features between adjacent scales progressively increases with the scale index, reaching particularly higher values at larger scales. We treat this similarity as a Scale-wise Update Index, which serves as the pruning criterion. Consequently, more tokens can be pruned at larger scales to improve efficiency. Furthermore, we propose Parallel Group Decoding, which leverages this high similarity at larger scales to decode tokens from different scales in parallel, further accelerating inference. Experimental results show that the proposed LazyVAR achieves up to a 2.94x speedup of recent VAR models with negligible performance degradation, allowing the Infinity-2B text-to-image model to generate 1024x1024 resolution images within 0.5 seconds on a single RTX 4090 GPU.