用内在自置信度奖励改进文本到图像生成
Improving Text-to-Image Generation with Intrinsic Self-Confidence Rewards
用模型自身对加噪潜变量的重建误差构造内在奖励，替代外部奖励模型做强化学习后训练，可直接迁移到扩散/流匹配模型的偏好对齐与组合生成能力提升，并可与外部奖励组合以缓解奖励黑客。
原文依据 we re-noise the model's own outputs and measure how accurately it recovers the injected noise
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论文本到图像生成支撑着设计、媒体与数据增强等领域的内容创作。文本到图像生成模型的后训练是提升人类偏好对齐、事实性与美学质量的可行路径。我们提出 SOLACE（Self-Originating LAtent Confidence Estimation，自源潜变量置信度估计），这是一个后训练框架，它用内部自置信度信号替代外部奖励监督：我们对模型自身的输出重新加噪，并测量其恢复所注入噪声的准确程度，将低重建误差视为高自置信度。SOLACE 将该内在信号转换为强化学习所用的标量奖励，无需外部奖励模型、标注人员或偏好数据。通过强化高置信度的生成结果，SOLACE 在组合式生成、文本渲染与文本-图像对齐上带来一致的提升。将 SOLACE 与外部奖励结合可产生互补的改进，同时缓解奖励黑客问题。
英文原文摘要 973 字符
Text-to-image generation powers content creation across design, media, and data augmentation. Post-training of text-to-image generative models is a promising path to improve human preference alignment, factuality, and aesthetics. We introduce SOLACE (Self-Originating LAtent Confidence Estimation), a post-training framework that replaces external reward supervision with an internal self-confidence signal: we re-noise the model's own outputs and measure how accurately it recovers the injected noise, treating low reconstruction error as high self-confidence. SOLACE converts this intrinsic signal into scalar rewards for reinforcement learning, requiring no external reward models, annotators, or preference data. By reinforcing high-confidence generations, SOLACE delivers consistent gains in compositional generation, text rendering, and text-image alignment. Integrating SOLACE with external rewards yields complementary improvements while alleviating reward hacking.