Uni-Hema:面向数字血液病理学的统一模型
Uni-Hema: Unified Model for Digital Hematopathology
Hema-Former 在层级上桥接视觉与语言表征、统一处理检测/分类/分割/形态预测/VQA 的多粒度多任务设计,可迁移到其他需要细胞级多任务推理的显微成像场景;其 46 个数据集、700K 图像与 21K QA 对的整合方式也提供了多源医学数据统一训练的参考。
原文依据 built upon Hema-Former, a multimodal module that bridges visual and linguistic representations at the hierarchy level
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论数字血液病理学需要对多种疾病类别进行细胞级分析,包括恶性疾病(如白血病)、感染性疾病(如疟疾)以及非恶性红细胞疾病(如镰状细胞病)。无论是单任务模型、视觉语言模型、面向全切片图像(WSI)优化的模型,还是单细胞血液学模型,这些方法都存在一个关键局限:它们无法在数字血液病理学的复杂场景中提供统一的多任务、多模态推理。为克服这些局限,我们提出 Uni-Hema,一个面向数字血液病理学的多任务统一模型,集成了跨多种疾病的检测、分类、分割、形态预测与推理。Uni-Hema 利用了 46 个公开数据集,涵盖超过 70 万张图像和 2.1 万组问答对,并构建于 Hema-Former 之上;Hema-Former 是一个多模态模块,在不同层级上为不同粒度下的各项任务(检测、分类、分割、形态学、掩码语言建模和视觉问答)桥接视觉与语言表征。大量实验表明,在多种血液学任务上,Uni-Hema 的性能与单任务、单数据集训练的模型相当或更优,同时在单细胞层面提供可解释、与形态学相关的洞见。我们的框架为多任务、多模态数字血液病理学建立了新的标准。代码见 https://github.com/intelligentMachines-ITU/Uni-Hema。
英文原文摘要 1570 字符
Digital hematopathology requires cell-level analysis across diverse disease categories, including malignant disorders (e.g., leukemia), infectious conditions (e.g., malaria), and non-malignant red blood cell disorders (e.g., sickle cell disease). Whether single-task, vision-language, WSI- optimized, or single-cell hematology models, these approaches share a key limitation: they cannot provide unified, multi-task, multi-modal reasoning across the complexities of digital hematopathology. To overcome these limitations, we propose Uni-Hema, a multi-task, unified model for digital hematopathology integrating detection, classification, segmentation, morphology prediction, and reasoning across multiple diseases. Uni-Hema leverages 46 publicly available datasets, encompassing over 700K images and 21K question-answer pairs, and is built upon Hema-Former, a multimodal module that bridges visual and linguistic representations at the hierarchy level for the different tasks (detection, classification, segmentation, morphology, mask language modeling, and visual question answering) at different granularities. Extensive experiments demonstrate that Uni-Hema achieves comparable or superior performance compared to training on a single task and single-dataset models, across diverse hematological tasks, while providing interpretable, morphologically relevant insights at the single-cell level. Our framework establishes a new standard for multi-task and multi-modal digital hematopathology. The code is available at https://github.com/intelligentMachines-ITU/Uni-Hema