RADAR：VAR的VQ-VAE解码器是加速退化恢复的好学生
RADAR: VQ-VAE Decoder of VAR is a Good Student for Restoring Against Degradation by Acceleration
针对VAR中跨尺度注意力语义偏置问题，提出语义代价感知的注意力掩码与解码器端LoRA知识蒸馏恢复，为视觉自回归生成提供可复用的加速与保真平衡方案。
原文依据 we propose a two-stage acceleration framework for VAR
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉自回归建模（VAR）近期已成为一种强大的图像生成范式，在效率和质量上超越了扩散模型。然而，加速VAR中的注意力计算仍然具有挑战性，因为跨尺度的注意力模式表现出强烈且复杂的语义偏置：早期粗尺度词元主导全局结构，而细尺度词元主要细化局部细节。现有的加速方法依赖于启发式词元剪枝或固定注意力掩码，缺乏平衡加速与语义保真度的原则性方法。在本工作中，我们提出了一种用于VAR的两阶段加速框架。首先，我们引入了一种语义代价感知的掩码策略（SCA-Mask），该策略量化每个注意力块的重要性，并将掩码形状设计表述为代价约束优化问题。这使得在给定计算预算下能够进行自适应剪枝，同时保留必要的语义上下文。其次，我们提出了加速后适应（PAA），一种解码器端的微调方案，采用内部知识蒸馏从剪枝后的潜在表示中恢复图像质量。PAA不需要外部数据，并使用基于LoRA的轻量级适应，为重新训练自回归Transformer提供了一种高效替代方案。在多个VAR任务上的综合实验表明，我们的方法在视觉保真度损失可忽略的情况下实现了可观的速度提升，为快速高质量的视觉自回归生成提供了一条原则性且有效的途径。
英文原文摘要 1561 字符
Visual Autoregressive Modeling (VAR) has recently emerged as a powerful paradigm for image generation that surpasses diffusion models in efficiency and quality. However, accelerating attention computation in VAR is still challenging because attention patterns across scales exhibit strong and complex semantic biases that early coarse-scale tokens dominate global structure, while fine-scale tokens mainly refine local details. Existing acceleration methods rely on heuristic token pruning or fixed attention masks, lacking a principled way to balance acceleration and semantic fidelity. In this work, we propose a two-stage acceleration framework for VAR. First, we introduce a semantic-cost-aware masking strategy (SCA-Mask) that quantifies the importance of each attention tile and formulates mask shape design as a cost-constrained optimization problem. This enables adaptive pruning under a given compute budget while preserving essential semantic context. Second, we present Post-Acceleration Adaptation (PAA), a decoder-side fine-tuning scheme that employs internal knowledge distillation to restore image quality from pruned latents. PAA does not require external data and uses a lightweight LoRA-based adaptation, providing a highly efficient alternative to retraining the autoregressive transformer. Comprehensive experiments across multiple VAR tasks demonstrate that our method achieves decent speedup with negligible loss of visual fidelity, yielding a principled and effective pathway toward fast and high-quality visual autoregressive generation.