课程组策略优化：自适应采样释放文生图生成潜力
Curriculum Group Policy Optimization: Adaptive Sampling for Unleashing the Potential of Text-to-Image Generation
把 GRPO 训练中的提示词采样从均匀改为按组奖励方差自适应加权，可直接迁移到扩散/流匹配模型的强化学习后训练，用来提升提示词对齐与训练效率；类别校准的比例公平思路也可用于多类别文生图数据配比。
原文依据 We use the variance of group rewards as an online proxy for prompt inconsistency.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论文生图（T2I）生成近年来取得了显著进展。与此同时，强化学习方法，尤其是基于组相对策略优化（GRPO）的方法，受到了广泛关注并被成功应用于 T2I 任务。然而，训练中常用的均匀采样策略往往忽略了样本难度与模型当前学习能力之间的匹配关系，导致训练效率低下。我们认为，提升训练效率需要持续优先采样那些与模型不断演进的能力相匹配、且仍处于可学习状态的提示词。为此，我们提出课程组策略优化（CGPO），一种自适应课程训练框架。训练过程中，每个提示词生成一组图像并由奖励模型打分。我们以组奖励的方差作为提示词不一致性的在线代理指标。方差较高说明模型已部分捕捉到提示词要求但尚未稳定掌握，这类提示词更可能提供有用的学习信号，因此我们相应提高其采样概率。此外，针对多类别数据集中的数据不平衡问题，我们设计了一种基于比例公平优化的类别校准方法，以平衡各类别之间的训练难度。在 GenEval、T2I-CompBench++ 和 DPG Bench 上的实验表明，我们的框架有效提升了生成性能。
英文原文摘要 1501 字符
Text-to-Image (T2I) generation has achieved remarkable progress in recent years. Meanwhile, reinforcement learning methods, particularly those based on Group Relative Policy Optimization (GRPO), have attracted widespread attention and been successfully applied to T2I tasks. However, the uniform sampling strategy commonly used during training often ignores the match between sample difficulty and the model's current learning capability, leading to low training efficiency. We argue that improving training efficiency requires continuously prioritizing prompts that match the model's evolving capability and remain actively learnable. To this end, we propose Curriculum Group Policy Optimization (CGPO), an adaptive curriculum training framework. During training, each prompt produces a group of images scored by a reward model. We use the variance of group rewards as an online proxy for prompt inconsistency. A higher variance suggests that the model has partially captured the prompt requirements but has not yet achieved stable mastery. Such prompts are more likely to provide useful learning signals, so we increase their sampling probabilities accordingly. Additionally, to address data imbalance in multi-category datasets, we design a category calibration method based on proportional fairness optimization, which balances training difficulty across categories. Experiments on GenEval, T2I-CompBench++, and DPG Bench demonstrate that our framework effectively improves generation performance.