POCA：面向视觉文本生成的帕累托最优课程对齐
POCA: Pareto-Optimal Curriculum Alignment for Visual Text Generation
把视觉文本生成中的文本准确率与图像美感/指令遵循的冲突建模为多目标优化，用帕累托最优集替代加权求和奖励，并配合自动难度评估的课程式提示选择；这套「帕累托奖励+课程采样」思路可直接迁移到任何需要多奖励 RL 微调的扩散生成任务（如编辑、个性化）。
原文依据 identifying the Pareto-optimal set to avoid simple scalarization
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论当前的视觉文本生成模型难以在文本准确率与整体图像一致性之间取得平衡。我们发现，追求高文本准确率会降低美学质量和指令遵循能力。尽管强化学习方法可以通过与多种奖励对齐来缓解该问题，但它们在文本生成上往往不稳定，因为现有方法通常以加权求和的方式优化多个奖励。此外，各奖励的权重难以平衡。而且，强化学习需要一组训练指令：大量提示词需要更多训练时间和计算资源，而少量提示词又会导致性能不佳。因此，如何选择提示词以实现高效训练仍是一个未解决的问题。在本研究中，我们提出帕累托最优课程对齐（POCA），一个将该问题作为多目标问题来处理的框架，其做法是：1）识别帕累托最优集以避免简单的标量化；2）设计自适应课程对齐策略，利用自动难度评估来管理多奖励数据集的学习顺序，这对于强化学习方法在有限数据环境中探索时的最优收敛至关重要。二者协同作用，POCA 在统一的奖励空间中寻找帕累托最优集，从而消除不一致的信号，在由易到难的优化地形下从不同奖励中找到最佳权衡解。实验结果表明，POCA 显著提升了 CLIP、HPS 分数和句子准确率等所有指标。
英文原文摘要 1600 字符
Current visual text generation models struggle with the trade-off between text accuracy and overall image coherence. We find that achieving high text accuracy can reduce aesthetic quality and instruction-following capability. Although reinforcement learning approaches can alleviate the problem through aligning with multiple rewards, they are often unstable for text generation, as existing approaches normally optimize multiple rewards in a weighted-sum way. In addition, it is difficult to balance the weight of each reward. Moreover, reinforcement learning requires a set of training instructions. A large number of prompts require more training time and computing resources, while a small set leads to poor performance. Hence, how to select the prompts for efficient training is an unsolved problem. In this study, we propose Pareto-Optimal Curriculum Alignment (POCA), a framework that addresses this issue as a multi-objective problem by: 1) identifying the Pareto-optimal set to avoid simple scalarization and 2) designing an adaptive curriculum alignment strategy to manage a learning sequence of a multi-reward dataset using automatic difficulty assessment, which is crucial for optimal convergence as RL methods explore in a limited data environment. In synergy, POCA finds the Pareto-optimal set in a unified reward space, which eliminates inconsistent signals to find the best trade-off solution from different rewards under an easy-to-hard optimization landscape. The experimental results show that POCA significantly improves all metrics such as CLIP, HPS scores and sentence accuracy.