视觉语言模型 · CVPR2026 ·2026.06.03

OralGPT-Omni:一种多功能的牙科多模态大语言模型

OralGPT-Omni: A Versatile Dental Multimodal Large Language Model

Jing Hao、Yuci Liang、Lizhuo Lin、Yuxuan Fan、Wenkai Zhou、Kaixin Guo 等 15 人

与你有关的部分

构建了牙科领域的思维链数据集 TRACE-CoT 和统一多模态基准 MMOral-Uni,其四阶段训练范式可迁移到其他医学影像的 VLM 微调中,为领域专用 MLLM 的可靠性评估提供了可复用的数据构造与评测思路。

原文依据 we construct TRACE-CoT, a clinically grounded chain-of-thought dataset that mirrors dental radiologists' decision-making processes
可迁移到 医学视觉问答多模态推理图像理解

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

多模态大语言模型(MLLMs)在众多医学专科中展现出巨大潜力,然而牙科领域仍探索不足,部分原因在于领域特定数据有限、牙科专家标注稀缺、模态特定建模不足以及可靠性方面的挑战。本文提出 OralGPT-Omni,这是首个专为牙科设计的 MLLM,旨在跨多种牙科成像模态和临床任务进行全面且可信的分析。为明确捕捉牙医的诊断推理过程,我们构建了 TRACE-CoT,一个基于临床的思维链数据集,其反映了牙科放射科医生的决策过程。这种推理监督与我们提出的四阶段训练范式相结合,显著增强了模型对牙科图像理解和分析的能力。同时,我们引入了 MMOral-Uni,这是首个用于牙科图像分析的统一多模态基准。它包含 2,809 个开放式问答对,涵盖五种模态和五项任务,为数字牙科中的 MLLMs 提供了迄今为止全面的评估套件。OralGPT-Omni 在 MMOral-Uni 基准上取得 51.84 的总分,在 MMOral-OPG 基准上取得 45.31 的总分,大幅超越 GPT-5 的得分。我们的工作推动了智能牙科的发展,并为牙科图像分析的未来进步铺平了道路。所有代码、基准和模型将公开提供。

英文原文摘要 1459 字符

Multimodal Large Language Models (MLLMs) have exhibited immense potential across numerous medical specialties, yet dentistry remains underexplored, in part due to limited domain-specific data, scarce dental expert annotations, insufficient modality-specific modeling, and challenges in reliability. In this paper, we present OralGPT-Omni, the first dental-specialized MLLM designed for comprehensive and trustworthy analysis across diverse dental imaging modalities and clinical tasks. To explicitly capture dentists' diagnostic reasoning, we construct TRACE-CoT, a clinically grounded chain-of-thought dataset that mirrors dental radiologists' decision-making processes. This reasoning supervision, combined with our proposed four-stage training paradigm, substantially strengthens the model's capacity for dental image understanding and analysis. In parallel, we introduce MMOral-Uni, the first unified multimodal benchmark for dental image analysis. It comprises 2,809 open-ended question-answer pairs spanning five modalities and five tasks, offering a comprehensive evaluation suite to date for MLLMs in digital dentistry. OralGPT-Omni achieves an overall score of 51.84 on the MMOral-Uni benchmark and 45.31 on the MMOral-OPG benchmark, dramatically outperforming the scores of GPT-5. Our work promotes intelligent dentistry and paves the way for future advances in dental image analysis. All code, benchmark, and models will be made publicly available.

标签

会议 CVPR2026
页码 38509-38519
Bibkey
来源 CVF Open Access