Simple-ViLMedSAM：简单文本提示结合视觉语言模型用于医学图像分割
Simple-ViLMedSAM: Simple Text Prompts Meet Vision-Language Models for Medical Image Segmentation
用 CLIP 文本嵌入隐式生成位置先验图来替代 SAM 的几何提示，并通过双向交叉注意力把该先验与 SAM 像素特征对齐，这一「文本→位置图→分割」的提示替代思路可直接迁移到其他需要免手工提示的开放词表分割任务。
原文依据 an Implicit Pos-Prompter (IPP), which generates attribution maps containing implicit positional cues to replace traditional geometric prompts
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论医学图像分割面临标注数据有限、标注成本高昂以及图像异质性显著等挑战。尽管大规模视觉基础模型（如 SAM）在该领域已展现出巨大潜力，但现有基于 SAM 的方法通常依赖专家定义的几何提示或复杂的临床文本提示，这限制了它们在不同医学图像分割任务上的泛化能力。为克服这些挑战，我们提出 Simple-ViLMedSAM，一个 CLIP-SAM 集成框架，仅使用简单文本查询（即仅使用基本的解剖结构或疾病相关文本标签）即可在零样本和少样本设置下实现高精度分割。其核心是隐式位置提示器（Implicit Pos-Prompter，IPP），它生成包含隐式位置线索的归因图，以替代传统的几何提示。IPP 结合了多模态信息瓶颈和基于亲和度的细化策略，以确保来自 CLIP-SAM 交互的高质量引导。为进一步提升分割效果，我们引入双向交互解码器（Bidirectional Interaction Decoder，BID），它采用双向交叉注意力将 IPP 的位置图与 SAM 的像素级特征对齐。通过联合建模全局语义与局部细节，BID 显著提升了分割精度。在四个公开数据集上的大量实验表明，Simple-ViLMedSAM 仅使用简单文本查询，在零样本和少样本医学图像分割任务中均持续优于现有方法。代码将在论文录用后公开。
英文原文摘要 1582 字符
Medical image segmentation is challenging due to limited annotated data, high labeling costs, and substantial image heterogeneity. Although large-scale vision foundation models (e.g., SAM) have shown great potential in this field, existing SAM-based methods typically rely on expert-defined geometric prompts or complex clinical text prompts, which limits their generalizability across diverse medical image segmentation tasks. To overcome these challenges, we propose Simple-ViLMedSAM, a CLIP-SAM integration framework that enables high-accuracy segmentation in zero-shot and few-shot settings using only simple text queries, that is, using only basic anatomical or disease-related text labels. At its core is an Implicit Pos-Prompter (IPP), which generates attribution maps containing implicit positional cues to replace traditional geometric prompts. IPP incorporates a multi-modal information bottleneck and an affinity-based refinement strategy to ensure high-quality guidance from CLIP-SAM interactions. To further enhance segmentation, we introduce a Bidirectional Interaction Decoder (BID) that employs bidirectional cross-attention to align IPP's positional maps with SAM's pixel-level features. By jointly modeling global semantics and local details, BID significantly improves segmentation accuracy. Extensive experiments on four public datasets demonstrate that Simple-ViLMedSAM consistently outperforms existing methods in both zero-shot and few-shot medical image segmentation tasks, using only simple text queries. The code will be publicly available upon acceptance.