OraPO：面向数据高效与事实准确的放射学报告生成的 Oracle 教育强化学习
OraPO: Oracle-educated Reinforcement Learning for Data-efficient and Factual Radiology Report Generation
把 GRPO 探索失败的样本通过轻量 oracle 步骤转成直接偏好监督，配合基于原子临床事实蕴含检查的句级稠密奖励，为视觉语言模型在低数据预算下做单阶段 RL 训练提供了可复用范式。
原文依据 OraPO enables single-stage, RL-only training by converting failed GRPO explorations on rare or difficult studies into direct preference supervision
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论放射学报告生成（RRG）旨在从胸部 X 光图像自动生成临床忠实的报告。现有工作通常遵循规模驱动范式，即在大规模配对语料上多阶段训练并使用超大骨干网络，使流程在数据和算力上高度密集。本文提出基于 FactScore 奖励（FactS）的 Oracle 教育 GRPO（OraPO），以在预算受限条件下解决 RRG 任务。OraPO 通过一个轻量的 oracle 步骤，将 GRPO 在罕见或困难病例上失败的探索转化为直接偏好监督，从而实现单阶段、纯强化学习的训练。FactS 通过抽取原子临床事实并检查其与真实标签的蕴含关系，将学习锚定在诊断证据上，产生稠密且可解释的句级奖励。OraPO 与 FactS 共同构成一个紧凑而强大的框架，显著提升在临床困难病例上的学习效率，在 CheXpert Plus 数据集上以少 2 至 3 个数量级的训练数据、在普通硬件上使用小型基础视觉语言模型，取得了新的 SOTA 性能（F1 为 0.341）。
英文原文摘要 1128 字符
Radiology report generation (RRG) aims to automatically produce clinically faithful reports from chest X-ray images. Prevailing work typically follows a scale-driven paradigm, by multi-stage training over large paired corpora and oversized backbones, making pipelines highly data- and compute-intensive. In this paper, we propose Oracle-educated GRPO (OraPO) with a FactScore-based reward (FactS) to tackle the RRG task under constrained budgets. OraPO enables single-stage, RL-only training by converting failed GRPO explorations on rare or difficult studies into direct preference supervision via a lightweight oracle step. FactS grounds learning in diagnostic evidence by extracting atomic clinical facts and checking entailment against ground-truth labels, yielding dense, interpretable sentence-level rewards. Together, OraPO and FactS create a compact and powerful framework that significantly improves learning efficiency on clinically challenging cases, setting the new SOTA performance on the CheXpert Plus dataset (0.341 in F1) with 2-3 orders of magnitude less training data using a small base VLM on modest hardware.