Uni-Encoder 遇上多编码器：面向缺失模态脑肿瘤分割的融合前表征
Uni-Encoder Meets Multi-Encoders: Representation Before Fusion for Brain Tumor Segmentation with Missing Modalities
两阶段「先表征后融合」的异构架构值得借鉴：先用掩码图像建模预训练单一 ViT 获得对模态缺失鲁棒的全局表征，再接入模态特定的 CNN 多编码器补回高分辨率细节，这一解耦思路可迁移到其他多模态缺失输入的密集预测任务。
原文依据 Stage 1 pretrains a single ViT Uni-Encoder with masked image modeling to establish a unified representation robust to missing modalities.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多模态 MRI 为脑肿瘤分割提供互补信息，但临床扫描常常缺少一个或多个模态，导致分割性能下降。本文提出 UniME（Uni-Encoder Meets Multi-Encoders），一种面向缺失模态脑肿瘤分割的两阶段异构方法，用以调和细粒度结构捕捉、跨模态互补性建模与可用模态利用之间的权衡。其思路是通过两阶段异构架构将表征学习与分割解耦。第一阶段以掩码图像建模预训练单个 ViT Uni-Encoder，建立对模态缺失鲁棒的统一表征。第二阶段加入模态特定的 CNN 多编码器，以提取高分辨率、多尺度、细粒度的特征。我们将这些特征与全局表征融合，从而产生精确的分割结果。在 BraTS 2023 与 BraTS 2024 上的实验表明，UniME 在不完整多模态场景下优于以往方法。代码已发布于 https://github.com/Hooorace-S/UniME
英文原文摘要 1112 字符
Multimodal MRI offers complementary information for brain tumor segmentation, but clinical scans often lack one or more modalities, which degrades segmentation performance. In this paper, we propose UniME (Uni-Encoder Meets Multi-Encoders), a two-stage heterogeneous method for brain tumor segmentation with missing modalities that reconciles the trade-offs among fine-grained structure capture, cross-modal complementarity modeling, and exploitation of available modalities. The idea is to decouple representation learning from segmentation via a two-stage heterogeneous architecture. Stage 1 pretrains a single ViT Uni-Encoder with masked image modeling to establish a unified representation robust to missing modalities. Stage 2 adds modality-specific CNN Multi-Encoders to extract high-resolution, multi-scale, fine-grained features. We fuse these features with the global representation to produce precise segmentations. Experiments on BraTS 2023 and BraTS 2024 show that UniME outperforms previous methods under incomplete multi-modal scenarios. The code is available at https://github.com/Hooorace-S/UniME