虚拟节点引导的动态图神经网络用于缺失模态脑肿瘤分割
Virtual Nodes Guided Dynamic Graph Neural Network for Brain Tumor Segmentation with Missing Modalities
针对多模态医学影像中模态缺失问题,提出基于虚拟节点和图动态连接的统一框架,可复用于其他多模态融合任务(如RGB-D、多光谱)的缺失模态鲁棒性提升。
原文依据 we propose a graph-based one-stage framework for robust brain tumor segmentation with missing modalities
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多模态磁共振成像(MRI)对脑肿瘤分割至关重要,许多方法利用其四个关键模态来捕获互补信息,从而有效进行子区域分析。然而,实践中多个模态缺失的情况非常常见,导致现有的全模态分割方法性能严重下降。受限于结构化数据模型,近期工作通常对全模态和缺失模态场景采用多阶段训练策略,这增加了训练成本,且未能充分应对模态缺失的干扰。在本工作中,我们提出了一种基于图的一阶段框架,用于缺失模态下的鲁棒脑肿瘤分割。具体而言,我们引入了模态特定的虚拟节点,作为补充信息源以补偿缺失的模态。为增强模型对任意模态组合的鲁棒性,我们利用图网络固有的灵活性设计了一种动态连接策略。该机制根据模态可用性动态调整邻接矩阵,在保留有益信息流的同时减轻缺失模态引起的干扰效应。此外,我们通过异构权重矩阵增强图网络,提升其对多模态场景的适应性。在BRATS-2018和BRATS-2020数据集上的大量实验表明,我们的方法在几乎所有不完整模态子集上均优于最优方法。
英文原文摘要 1538 字符
Multimodal magnetic resonance imaging (MRI) is crucial for brain tumor segmentation, with many methods leveraging its four key modalities to capture complementary information for effective sub-region analysis. However, the absence of several modalities is very common in practice, leading to severe performance degradation in existing full-modality segmentation methods. Limited by the structured data model, recent works often adopt a multi-stage training strategy for full-modality and missing-modality scenarios, which increases training costs and inadequately addresses the interference of miss. In this work, we propose a graph-based one-stage framework for robust brain tumor segmentation with missing modalities. Specifically, we introduce modality-specific virtual nodes that serve as supplementary information sources to compensate for missing modalities. To enhance model robustness against arbitrary modality combinations, we leverage the inherent flexibility of graph networks to devise a dynamic connection strategy. This mechanism dynamically adjusts the adjacency matrix based on modality availability, preserving beneficial information flow while mitigating interference effects caused by missing modalities. Furthermore, we enhance the graph network through heterogeneous weight matrices, enhancing its adaptability to multimodal scenarios. Extensive experiments on the BRATS-2018 and BRATS-2020 datasets demonstrate that our method outperforms the state-of-the-art methods on almost all subsets of incomplete modalities.