JoPPO：基于对比联合条件概率强化学习的分层摄影评估
JoPPO: Hierarchical Photography Assessment via Contrastive Joint Conditional Probabilistic Reinforcement Learning
把 VLM 当作评分器时，用基于批内细粒度属性分数条件分布的期望胜率作为奖励，替代 GRPO 的标量奖励，可直接迁移到任何需要多属性组合排序的视觉评测任务（美学、生成质量、偏好对齐）。
原文依据 we introduce JoPPO, which compute reward based on the expected win rate of total scores derived from the conditional distribution of fine-grained attribute scores within batches
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论随着视觉语言模型（VLM）的发展，采用 VLM-as-a-Judge 进行视觉评估已成为视觉研究中被广泛采用的指标。然而，现有的 VLM-as-a-Judge 方法存在评分结果有偏、区分度低的问题，且缺乏统一的多属性组合评估能力。为解决这些局限，我们提出一种新的训练范式，称为 JoPPO（联合概率策略优化），使 VLM 能够在组合评估约束下学习排序。我们以图像美学作为试验平台对 JoPPO 进行评估，该任务需要对构图、光照、色彩和几何等多个属性有细致理解。训练分为两个阶段：（1）在由自动化数据生成流程提供的合成构图数据集上进行监督微调（SFT），以注入构图先验；（2）对比联合条件概率强化学习：在 GRPO 算法基础上，我们引入 JoPPO，其奖励基于批内细粒度属性分数条件分布所导出的总分的期望胜率来计算，有效增强了模型在组合评估中的判别能力。在标准美学基准上，我们的方法在排序一致性上取得了一致提升，展现出较强的零样本泛化能力。
英文原文摘要 1428 字符
With the advancement of Vision-Language Models (VLMs), employing VLM-as-a-Judge for visual evaluation has become a widely adopted metric in vision research. However, existing VLM-as-a-Judge approaches suffer from biased scoring outcomes with low discrimination and lack the capacity for unified multi-attribute compositional assessment. To address these limitations, we propose a novel training paradigm, termed JoPPO (**Jo**int **P**robabilistic **P**olicy **O**ptimization) that enables the VLMs to learn ranking under compositional assessment constraints. We evaluate the JoPPO on image aesthetics as a testbed, a task requiring nuanced understanding of multiple attributes including composition, lighting, color and geometry. Training follows two stages: (1) Supervised Fine-Tuning (SFT) on synthetic composition dataset provided by automated data generation pipeline to instill compositional priors; and (2) Contrastive Joint Conditional Probabilistic Reinforcement Learning: building upon the GRPO algorithm, we introduce JoPPO, which compute reward based on the expected win rate of total scores derived from the conditional distribution of fine-grained attribute scores within batches, effectively enhancing the model's discriminative ability in composite evaluation. Across standard aesthetic benchmarks, our method achieves consistent improvements in ranking consistency, demonstrating strong zero-shot generalization.