学习信任什么：面向视觉生成的贝叶斯先验引导优化
Learning What to Trust: Bayesian Prior-Guided Optimization for Visual Generation
把奖励模型的不确定性显式建模为语义先验锚点，并在组间与组内两个层级调制优化信任度，可直接迁移到任何用 GRPO/RLHF 后训练的扩散或视频生成模型上，缓解文本-视觉多对多对应带来的奖励噪声过拟合。
原文依据 BPGO adaptively modulates optimization trust at two levels
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论组相对策略优化（GRPO）已成为视觉生成模型后训练中一种有效且轻量的框架。然而，其性能从根本上受限于文本-视觉对应关系的歧义性：单个提示词可以合理地描述多样的视觉输出，而单张图像或单段视频也可能支持多种同样正确的解读。这种多对多关系使奖励模型产生不确定且判别力弱的信号，导致 GRPO 未能充分利用可靠反馈，并对噪声反馈过拟合。我们提出贝叶斯先验引导优化（BPGO），这是 GRPO 的一种新颖扩展，通过语义先验锚点显式建模奖励不确定性。BPGO 在两个层级上自适应地调制优化信任度：组间贝叶斯信任分配强调与先验一致的组所产生的更新，同时降低歧义组的权重；组内先验锚定重归一化则通过放大置信偏差、压缩不确定分数来锐化样本区分度。在图像与视频生成任务上，BPGO 相比标准 GRPO 及近期变体，均能持续带来更强的语义对齐、更高的感知保真度以及更快的收敛。
英文原文摘要 1249 字符
Group Relative Policy Optimization (GRPO) has emerged as an effective and lightweight framework for post-training visual generative models. However, its performance is fundamentally limited by the ambiguity of textual-visual correspondence: a single prompt may validly describe diverse visual outputs, and a single image or video may support multiple equally correct interpretations. This many-to-many relationship leads reward models to generate uncertain and weakly discriminative signals, causing GRPO to underutilize reliable feedback and overfit noisy ones. We introduce Bayesian Prior-Guided Optimization (BPGO), a novel extension of GRPO that explicitly models reward uncertainty through a semantic prior anchor. BPGO adaptively modulates optimization trust at two levels: inter-group Bayesian trust allocation emphasizes updates from groups consistent with the prior while down-weighting ambiguous ones, and intra-group prior-anchored renormalization sharpens sample distinctions by expanding confident deviations and compressing uncertain scores.Across both image and video generation tasks, BPGO delivers consistently stronger semantic alignment, enhanced perceptual fidelity, and faster convergence than standard GRPO and recent variants.