视觉语言模型 · CVPR2026 ·2026.06.03

EMAD:面向阿尔茨海默病的以证据为中心的可定位多模态诊断

EMAD: Evidence-Centric Grounded Multimodal Diagnosis for Alzheimer's Disease

Qiuhui Chen、Xuancheng Yao、Zhenglei Zhou、Xinyue Hu、Yi Hong

与你有关的部分

把「句子→证据短语→3D MRI 解剖结构」的分层视觉定位机制与可验证奖励的强化微调结合,可复用于任何需要让生成式报告中的每条结论落到具体图像区域的可解释医学视觉语言任务;GTX-Distill 用弱监督教师向模型自生成报告的学生迁移定位行为,为降低密集标注成本提供了可借鉴范式。

原文依据 evidence-to-anatomy grounding localizes corresponding structures on 3D brain MRI
可迁移到 医学报告生成视觉定位3D 医学图像理解可解释诊断

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

医学图像分析的深度学习模型常表现为「黑箱」,很少与临床指南保持一致,也鲜有将决策与支撑证据显式关联。这在阿尔茨海默病(AD)中尤为关键,因为预测应当同时建立在解剖学与临床发现之上。我们提出 EMAD,一个视觉语言框架,用于生成结构化的 AD 诊断报告,其中每条论断都显式地定位于多模态证据。EMAD 采用分层的句子—证据—解剖(SEA)定位机制:(i) 句子到证据的定位将生成的句子关联到临床证据短语;(ii) 证据到解剖的定位在 3D 脑 MRI 上定位相应结构。为降低密集标注需求,我们提出 GTX-Distill,将定位行为从以有限监督训练的教师模型迁移到在模型生成报告上运行的学生模型。我们进一步引入 Executable-Rule GRPO,一种带可验证奖励的强化微调方案,用以强化临床一致性、方案遵循性与推理—诊断连贯性。在 AD-MultiSense 数据集上,EMAD 取得了最优的诊断准确率,并生成比现有方法更透明、解剖上更忠实的报告;我们将发布代码与定位标注,以支持可信医学视觉语言模型的未来研究。

英文原文摘要 1399 字符

Deep learning models for medical image analysis often act as "black boxes," seldom aligning with clinical guidelines or explicitly linking decisions to supporting evidence. This is especially critical in Alzheimer's disease (AD), where predictions should be grounded in both anatomical and clinical findings. We present EMAD, a vision-language framework that generates structured AD diagnostic reports with each claim explicitly grounded in multimodal evidence. EMAD uses a hierarchical Sentence-Evidence-Anatomy (SEA) Grounding mechanism: (i) sentence-to-evidence grounding links generated sentences to clinical evidence phrases, and (ii) evidence-to-anatomy grounding localizes corresponding structures on 3D brain MRI. To reduce dense annotation requirements, we propose GTX-Distill, which transfers grounding behavior from a teacher trained with limited supervision to a student operating on model-generated reports. We further introduce Executable-Rule GRPO, a reinforcement fine-tuning scheme with verifiable rewards that enforces clinical consistency, protocol adherence, and reasoning-diagnosis coherence. On the AD-MultiSense dataset, EMAD achieves state-of-the-art diagnostic accuracy and produces more transparent, anatomically faithful reports than existing methods; we will release code and grounding annotations to support future research in trustworthy medical vision-language models.

标签

会议 CVPR2026
页码 23031-23040
Bibkey
来源 CVF Open Access