CURE:面向可靠解剖定位报告生成的课程引导多任务训练
CURE: Curriculum-guided Multi-task Training for Reliable Anatomy Grounded Report Generation
把错误感知的课程学习用在多模态指令微调上,按模型当前表现动态调整短语定位与报告生成样本的采样权重,为视觉-语言模型提升空间对齐、抑制幻觉提供了一个无需额外数据的可复用训练策略。
原文依据 dynamically adjusts sampling based on model performance emphasizing on harder samples to improve spatial and textual alignment
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论医学视觉-语言模型能够自动生成放射学报告,但在准确的视觉定位与事实一致性方面仍存在困难。现有模型常常将文本发现与视觉证据错误对齐,导致预测不可靠或定位薄弱。我们提出“CURE”,一个错误感知的课程学习框架,在无需任何额外数据的情况下提升定位与报告质量。CURE 利用公开数据集,在多模态指令模型上针对短语定位、定位式报告生成和解剖定位式报告生成进行微调。该方法根据模型表现动态调整采样,侧重较难的样本,以改善空间与文本对齐。CURE 将定位精度提升 +0.35 IoU,将报告质量提升 +0.192 CXRFEScore,并将幻觉减少 18.6%。CURE 是一个数据高效的框架,同时增强定位精度与报告可靠性。
英文原文摘要 927 字符
Medical vision-language models can automate the generation of radiology reports but struggle with accurate visual grounding and factual consistency. Existing models often misalign textual findings with visual evidence, leading to unreliable or weakly grounded predictions. We present "CURE", an error-aware curriculum learning framework that improves grounding and report quality without any additional data. CURE tunes a multimodal instructional model on phrase grounding, grounded report generation, and anatomy-grounded report generation using public datasets. The method dynamically adjusts sampling based on model performance emphasizing on harder samples to improve spatial and textual alignment. CURE improves grounding accuracy by +0.35 IoU, boosts report quality by +0.192 CXRFEScore, and reduces hallucinations by 18.6%. CURE is a data-efficient framework that enhances both grounding accuracy and report reliability.