MedCLIPSeg：面向数据高效与可泛化医学图像分割的概率视觉-语言适配
MedCLIPSeg: Probabilistic Vision-Language Adaptation for Data-Efficient and Generalizable Medical Image Segmentation
将CLIP的patch级嵌入通过概率跨模态注意力进行适配，并引入软patch级对比损失，为视觉研究者提供了一种在标注稀缺的医学图像上实现文本引导分割且能输出不确定性图的通用思路。
原文依据 leverages patch-level CLIP embeddings through probabilistic cross-modal attention
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论医学图像分割仍面临训练标注有限、解剖特征模糊以及域偏移等挑战。尽管CLIP等视觉-语言模型提供了强大的跨模态表示，但其在密集、文本引导的医学图像分割中的潜力仍未得到充分探索。我们提出MedCLIPSeg，一种新颖的框架，将CLIP适配用于稳健、数据高效且不确定性感知的医学图像分割。我们的方法通过概率跨模态注意力利用patch级CLIP嵌入，实现图像与文本token之间的双向交互，并显式建模预测不确定性。结合软patch级对比损失——该损失鼓励在不同文本提示下进行更细致的语义学习——MedCLIPSeg有效提升了数据效率和域泛化能力。在涵盖五种成像模态和六个器官的16个数据集上的大量实验表明，MedCLIPSeg在准确性、效率和鲁棒性上优于先前方法，同时提供可解释的不确定性图，突出分割结果的局部可靠性。这项工作展示了概率视觉-语言建模在文本驱动医学图像分割中的潜力。
英文原文摘要 1274 字符
Medical image segmentation remains challenging due to limited annotations for training, ambiguous anatomical features, and domain shifts. While vision-language models such as CLIP offer strong cross-modal representations, their potential for dense, text-guided medical image segmentation remains underexplored. We present MedCLIPSeg, a novel framework that adapts CLIP for robust, data-efficient, and uncertainty-aware medical image segmentation. Our approach leverages patch-level CLIP embeddings through probabilistic cross-modal attention, enabling bidirectional interaction between image and text tokens and explicit modeling of predictive uncertainty. Together with a soft patch-level contrastive loss that encourages more nuanced semantic learning across diverse textual prompts, MedCLIPSeg effectively improves data efficiency and domain generalizability. Extensive experiments across 16 datasets spanning five imaging modalities and six organs demonstrate that MedCLIPSeg outperforms prior methods in accuracy, efficiency, and robustness, while providing interpretable uncertainty maps that highlight local reliability of segmentation results. This work demonstrates the potential of probabilistic vision-language modeling for text-driven medical image segmentation.