VoxTell:可自由文本提示的通用三维医学图像分割
VoxTell: Free-Text Promptable Universal 3D Medical Image Segmentation
把自由文本提示(单词到完整临床句子)映射为三维掩码,并在解码器各层做多尺度视觉-语言融合,为医学体数据分割提供可复用的零样本跨模态迁移范式;其跨模态迁移与语言变体鲁棒性评测协议也值得借鉴。
原文依据 uses multi-stage vision-language fusion across decoder layers to align textual and visual features at multiple scales
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论本文提出 VoxTell,一个用于文本提示三维医学图像分割的视觉语言模型。它将从单个词到完整临床句子的自由形式描述映射为三维掩码。VoxTell 在涵盖 1000 多个解剖与病理类别的 62K 余例 CT、MRI 与 PET 体数据上训练,通过跨解码器层的多阶段视觉-语言融合,在多个尺度上对齐文本特征与视觉特征。它在未见数据集上取得了跨模态的最优零样本性能,在熟悉概念上表现优异,并能泛化到相关的未见类别。大量实验进一步表明其具有强大的跨模态迁移能力、对语言变体与临床语言的鲁棒性,以及依据真实世界文本进行准确的实例特异性分割的能力。代码见:https://github.com/MIC-DKFZ/VoxTell
英文原文摘要 841 字符
We introduce VoxTell, a vision-language model for text-prompted volumetric medical image segmentation. It maps free-form descriptions, from single words to full clinical sentences, to 3D masks. Trained on 62K+ CT, MRI, and PET volumes spanning 1K+ anatomical and pathological classes, VoxTell uses multi-stage vision-language fusion across decoder layers to align textual and visual features at multiple scales. It achieves state-of-the-art zero-shot performance across modalities on unseen datasets, excelling on familiar concepts while generalizing to related unseen classes. Extensive experiments further demonstrate strong cross-modality transfer, robustness to linguistic variations and clinical language, as well as accurate instance-specific segmentation from real-world text. Code is available at: https://github.com/MIC-DKFZ/VoxTell