MaskFocus：将策略优化聚焦于掩码图像生成的关键步骤
MaskFocus: Focusing Policy Optimization on Critical Steps for Masked Image Generation
对做掩码生成模型（MaskGIT 类）的研究者，这篇给出了把 RL 后训练落到多步迭代采样上的可行路径：用中间图像与最终图像的相似度定位关键步、只在这些步上算似然做策略优化，并用熵驱动的动态路由采样提升掩码策略探索，可直接迁移到其他迭代式生成器的后训练。
原文依据 we determine the step-level information gain by measuring the similarity between the intermediate images at each sampling step and the final generated image
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论强化学习（RL）在语言模型和自回归视觉生成模型的后训练中已展现出显著潜力，但将 RL 适配到掩码生成模型（MGM）上仍具挑战性。其核心因素在于，策略优化因其多步迭代过程而需要每一步的概率似然。这种对完整采样轨迹的依赖带来高昂的计算开销，而直接对随机步骤进行优化往往只能得到次优结果。本文提出 MaskFocus，一种新颖的 RL 框架，通过聚焦关键步骤来实现对 MGM 的有效策略优化。具体而言，我们通过度量每个采样步的中间图像与最终生成图像之间的相似度来确定步级信息增益。关键在于，我们利用该信息增益识别出最关键、最有价值的步骤，并在这些步骤上执行策略优化。此外，我们设计了基于熵的动态路由采样，以鼓励模型探索更有价值的掩码策略。在多个文本到图像基准上的大量实验验证了本方法的有效性。
英文原文摘要 1175 字符
Reinforcement learning (RL) has demonstrated significant potential for post-training language models and autoregressive visual generative models, but adapting RL to masked generative models (MGMs) remains challenging. The core factor is that policy optimization requires the probability likelihood of each step due to its multi-step iteration process. This reliance on entire sampling trajectories introduces high computational cost, whereas natively optimizing random steps often yields suboptimal results. In this paper, we present MaskFocus, a novel RL framework that achieves effective policy optimization for MGMs by focusing on critical steps. Specifically, we determine the step-level information gain by measuring the similarity between the intermediate images at each sampling step and the final generated image. Crucially, we leverage this to identify the most critical and valuable steps and execute policy optimization on them. Furthermore, we design a dynamic routing sampling based on entropy to encourage the model to explore more valuable masking strategies. Extensive experiments on multiple Text-to-Image benchmarks validate the effectiveness of our method.