MedLoc-R1：面向基于 GRPO 的医学视觉定位的性能感知课程奖励调度
MedLoc-R1: Performance-Aware Curriculum Reward Scheduling for GRPO-Based Medical Visual Grounding
把 GRPO 的固定 IoU 奖励换成随模型能力逐步收紧的课程式奖励调度，可直接迁移到任何用 RL 做框级/区域级定位的视觉语言模型训练中，缓解小目标奖励稀疏导致的策略梯度消失。
原文依据 we propose MedLoc-R1, a performance-aware reward scheduling framework that progressively tightens the reward criterion in accordance with model readiness
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论医学视觉定位是细粒度多模态推理与可解释临床决策支持的重要基础。尽管强化学习在定位任务上已有进展，但现有方法如组相对策略优化（GRPO）直接应用于医学图像时会遭遇严重的奖励稀疏问题，其主要原因是定位细小或模糊的感兴趣区域本身存在固有困难，而强化学习中固定且次优的基于 IoU 的奖励方案又进一步加剧了这一问题。这导致策略梯度消失与优化停滞，尤其在训练早期。为应对该挑战，我们提出 MedLoc-R1，一种性能感知的奖励调度框架，它依据模型就绪程度逐步收紧奖励判据。MedLoc-R1 引入滑动窗口性能跟踪器与多条件更新规则，自动将奖励调度从稠密、易于获得的信号调整为更严格的细粒度定位要求，同时保留 GRPO 的优良性质，且不引入辅助网络或额外梯度路径。在三个医学视觉定位基准上的实验表明，MedLoc-R1 在定位精度与训练稳定性上均持续优于基于 GRPO 的基线。我们的框架为高风险医学应用中基于强化学习的定位提供了一种通用、轻量且有效的解决方案。代码与检查点将在录用后发布。
英文原文摘要 1562 字符
Medical visual grounding serves as a crucial foundation for fine-grained multimodal reasoning and interpretable clinical decision support. Despite recent advances in reinforcement learning (RL) for grounding tasks, existing approaches such as Group Relative Policy Optimization (GRPO) suffer from severe reward sparsity when directly applied to medical images, primarily due to the inherent difficulty of localizing small or ambiguous regions of interest, which is further exacerbated by the rigid and suboptimal nature of fixed IoU-based reward schemes in RL. This leads to vanishing policy gradients and stagnated optimization, particularly during early training. To address this challenge, we propose MedLoc-R1, a performance-aware reward scheduling framework that progressively tightens the reward criterion in accordance with model readiness. MedLoc-R1 introduces a sliding-window performance tracker and a multi-condition update rule that automatically adjust the reward schedule from dense, easily obtainable signals to stricter, fine-grained localization requirements, while preserving the favorable properties of GRPO without introducing auxiliary networks or additional gradient paths. Experiments on three medical visual grounding benchmarks demonstrate that MedLoc-R1 consistently improves both localization accuracy and training stability over GRPO-based baselines. Our framework offers a general, lightweight, and effective solution for RL-based grounding in high-stakes medical applications. Code and checkpoints will be released after acceptance.