Ultrasound-CLIP：面向超声图像-文本理解的语义感知对比预训练
Ultrasound-CLIP: Semantic-Aware Contrastive Pre-training for Ultrasound Image-Text Understanding
对做医学视觉-语言预训练的人，这篇给出了一个可复用的思路：用领域诊断属性体系构造语义软标签和语义损失，替代 CLIP 的硬负样本对比，并额外引入由文本属性构建的异构图模态做结构化推理；US-365K 与 UDT 体系也可直接作为超声域 VLM 的评测与预训练资源。
原文依据 we propose Ultrasound-CLIP, a semantic-aware contrastive learning framework that introduces semantic soft labels and semantic loss to refine sample discrimination
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论超声成像凭借实时性与无辐射特性，在临床诊断中被广泛使用。然而，现有的视觉-语言预训练模型（如 CLIP）主要面向其他模态设计，难以直接应用于解剖结构异质、诊断属性多样的超声数据。为弥补这一差距，我们构建了 US-365K，一个大规模超声图像-文本数据集，包含覆盖 52 个解剖类别的 36.5 万对配对样本。我们建立了超声诊断分类体系（UDT），其中包含两个层级化知识框架：超声层级化解剖分类体系用于规范解剖组织方式，超声诊断属性框架则形式化了九个诊断维度，包括身体系统、器官、诊断、形状、边缘、回声性、内部特征、后声学现象与血管性。在这些基础之上，我们提出 Ultrasound-CLIP，一个语义感知的对比学习框架，引入语义软标签与语义损失以细化样本判别。此外，我们基于 UDAF 的文本表示构建了一种异构图模态，从而能够对病灶-属性关系进行结构化推理。采用患者级数据划分的大量实验表明，我们的方法在分类与检索基准上达到最优性能，同时在零样本、线性探测与微调任务上也展现出强泛化能力。
英文原文摘要 1574 字符
Ultrasound imaging is widely used in clinical diagnostics due to its real-time capability and radiation-free nature. However, existing vision-language pre-training models, such as CLIP, are primarily designed for other modalities, and are difficult to directly apply to ultrasound data, which exhibit heterogeneous anatomical structures and diverse diagnostic attributes. To bridge this gap, we construct US-365K, a large-scale ultrasound image-text dataset containing 365k paired samples across 52 anatomical categories. We establish Ultrasonographic Diagnostic Taxonomy (UDT) containing two hierarchical knowledge frameworks. Ultrasonographic Hierarchical Anatomical Taxonomy standardizes anatomical organization, and Ultrasonographic Diagnostic Attribute Framework formalizes nine diagnostic dimensions, including body system, organ, diagnosis, shape, margins, echogenicity, internal characteristics, posterior acoustic phenomena, and vascularity. Building upon these foundations, we propose Ultrasound-CLIP, a semantic-aware contrastive learning framework that introduces semantic soft labels and semantic loss to refine sample discrimination. Moreover, we construct a heterogeneous graph modality derived from UDAF's textual representations, enabling structured reasoning over lesion-attribute relations. Extensive experiments with patient-level data splitting demonstrate that our approach achieves state-of-the-art performance on classification and retrieval benchmarks, while also delivering strong generalization to zero-shot, linear probing, and fine-tuning tasks.