MedMO：面向医学图像的多模态大语言模型视觉定位与理解
MedMO: Grounding and Understanding Multimodal Large Language Model for Medical Images
MedMO 的多阶段训练配方（跨模态预训练对齐异构视觉编码器、多任务指令微调、带 GIoU 框级奖励的强化学习）为医学视觉-语言模型提供了可复用的训练范式，尤其其框级 GIoU 奖励可直接用于提升视觉定位任务的空间推理能力。
原文依据 reinforcement learning with verifiable rewards that combine factuality checks with a box-level GIoU reward to strengthen spatial grounding
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多模态大语言模型（MLLMs）已快速发展，然而其在医学领域的应用仍受限于领域覆盖、模态对齐和视觉定位推理方面的不足。本文提出 MedMO，一个基于通用 MLLM 架构、完全在大规模领域特定数据上训练的医学基础模型。MedMO 采用多阶段训练配方：(i) 跨模态预训练，将异构视觉编码器与医学语言主干对齐；(ii) 在多任务监督上进行指令微调，涵盖图像描述、视觉问答、报告生成、检索以及基于边界框的疾病定位；(iii) 使用可验证奖励的强化学习，结合事实性检查与框级 GIoU 奖励，以增强复杂临床场景中的空间视觉定位与推理能力。MedMO 在多种模态和任务上持续优于强开源医学 MLLMs。在视觉问答基准上，MedMO 相比基线平均准确率提升 +21.3%，与 SOTA 模型 Fleming-VL 的差距在 0.6% 以内。在文本问答上，相比基线提升 +7.9%，相比 Fleming-VL 提升 +15.6%。在医学报告生成中，MedMO 在语义和临床准确性上均取得显著增益。此外，它展现出强大的视觉定位能力，相比基线 IoU 提升 +40.4，相比 Fleming-VL 提升 +37.0%，凸显了其稳健的空间推理与定位性能。在放射学、眼科、病理学和急诊护理领域的评估证实了 MedMO 广泛的跨模态泛化能力和可靠的空间推理能力。
英文原文摘要 1647 字符
Multimodal large language models (MLLMs) have rapidly advanced, yet their adoption in medicine remains limited by gaps in domain coverage, modality alignment, and grounded reasoning. In this work, we introduce MedMO, a medical foundation model built upon a generalized MLLM architecture and trained exclusively on large-scale, domain-specific data. MedMO follows a multi-stage training recipe: (i) cross-modal pretraining to align heterogeneous visual encoders with a medical language backbone; (ii) instruction tuning on multi-task supervision that spans captioning, VQA, report generation, retrieval, and grounded disease localization with bounding boxes; and (iii) reinforcement learning with verifiable rewards that combine factuality checks with a box-level GIoU reward to strengthen spatial grounding and reasoning in complex clinical scenarios. MedMO consistently outperforms strong open-source medical MLLMs across multiple modalities and tasks. On VQA benchmarks, MedMO achieves an average accuracy improvement of +21.3% over the baseline and performs within 0.6% of the SOTA Fleming-VL. For text-based QA, it attains +7.9% over the baseline and +15.6% over Fleming-VL. In medical report generation, MedMO delivers significant gains in both semantic and clinical accuracy. Moreover, it exhibits strong grounding capability, achieving an IoU improvement of +40.4 over the baseline and +37.0% over Fleming-VL, underscoring its robust spatial reasoning and localization performance. Evaluations across radiology, ophthalmology, pathology, and emergency care confirm MedMO's broad cross-modality generalization and reliable spatial reasoning.