掩码自回归变分加速：快速推理使强化学习变得实用
Masked Auto-Regressive Variational Acceleration: Fast Inference Makes Practical Reinforcement Learning
把 MAR 的「外层自回归解掩码 + 内层扩散去噪链」两级推理蒸馏成单步生成，并用基于分数的变分目标保住样本质量；做少步扩散/自回归生成或想把 RL 后训练接到生成模型上的人可直接复用该蒸馏目标与 MARVAL-RL 流程。
原文依据 compresses the diffusion chain into a single AR generation step while preserving the flexible auto-regressive unmasking order
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论掩码自回归扩散模型（MAR）兼具扩散模型强大的建模能力与掩码自回归排序的灵活性。然而，原始 MAR 因其分层推理机制而推理缓慢：外层是自回归解掩码循环，内层是扩散去噪链。这种解耦结构不仅损害生成效率，也阻碍了 MAR 在强化学习（RL）中的实际使用，而 RL 正日益成为生成模型后训练的关键范式。为解决这一根本问题，我们提出 MARVAL（掩码自回归变分加速），一个基于蒸馏的框架，在保留灵活自回归解掩码顺序的同时，将扩散链压缩为单次自回归生成步骤。借助 MARVAL 的蒸馏不仅带来显著的推理加速，更关键的是使带可验证奖励的 RL 后训练变得可行，从而得到可扩展且更受人类偏好的快速生成模型。我们的贡献有两方面：（1）一种新的基于分数的变分目标，可将掩码自回归扩散模型蒸馏为单步生成而不牺牲样本质量；（2）通过 MARVAL-RL 为掩码自回归模型提供的高效 RL 框架。在 ImageNet 256x256 上，MARVAL-Huge 取得 2.00 的弗雷歇初始距离，相比 MAR-diffusion 加速超过 30 倍；MARVAL-RL 在带实体名称的 ImageNet 数据集上于 CLIP 与 image-reward 分数上取得一致提升。总之，MARVAL 展示了掩码自回归扩散模型蒸馏与 RL 的首条实用路径，实现了快速采样与更好的偏好对齐。项目主页：https://ai4imaginglab.github.io/MARVAL-RL-Page/
英文原文摘要 1789 字符
Masked auto-regressive diffusion models (MAR) benefit from the expressive modeling ability of diffusion models and the flexibility of masked auto-regressive ordering. However, vanilla MAR suffers from slow inference due to its hierarchical inference mechanism: an outer AR unmasking loop and an inner diffusion denoising chain. Such decoupled structure not only harm the generation efficiency but also hinder the practical use of MAR for reinforcement learning (RL), an increasingly critical paradigm for generative model post-training.To address this fundamental issue, we introduce MARVAL (Masked Auto-regressive Variational Acceleration), a distillation-based framework that compresses the diffusion chain into a single AR generation step while preserving the flexible auto-regressive unmasking order. Such a distillation with MARVAL not only yields substantial inference acceleration but, crucially, makes RL post-training with verifiable rewards practical, resulting in scalable yet human-preferred fast generative models. Our contributions are twofold: (1) a novel score-based variational objective for distilling masked auto-regressive diffusion models into a single generation step without sacrificing sample quality; and (2) an efficient RL framework for masked auto-regressive models via MARVAL-RL. On ImageNet 256x256, MARVAL-Huge achieves an FID of 2.00 with more than 30 times speedup compared with MAR-diffusion, and MARVAL-RL yields consistent improvements in CLIP and image-reward scores on ImageNet datasets with entity names. In conclusion, MARVAL demonstrates the first practical path to distillation and RL of masked auto-regressive diffusion models, enabling fast sampling and better preference alignments. Project page: https://ai4imaginglab.github.io/MARVAL-RL-Page/