Tell2Adapt:基于视觉基础模型的无源无监督域自适应统一框架
Tell2Adapt: A Unified Framework for Source Free Unsupervised Domain Adaptation via Vision Foundation Model
利用视觉基础模型(VFM)的上下文感知提示正则化生成高质量伪标签,并通过视觉合理性精炼将预测重新锚定到目标图像的低层特征,为无源域自适应分割提供可复用的两阶段框架。
原文依据 Context-Aware Prompts Regularization (CAPR), which robustly translates varied text prompts into canonical instructions
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论无源无监督域自适应(SFUDA)对于在多样化的临床环境中部署深度学习模型至关重要。然而,现有方法通常针对低差异、特定域偏移设计,无法泛化为统一的多模态、多目标框架,这构成了实际应用的主要障碍。为解决此问题,我们提出 Tell2Adapt,一种新颖的 SFUDA 框架,利用视觉基础模型(VFM)广泛且可泛化的知识。我们的方法通过上下文感知提示正则化(CAPR)确保高保真 VFM 提示,该正则化稳健地将各种文本提示转换为规范指令。这使得能够生成高质量伪标签,从而高效地将轻量级学生模型适配到目标域。为保证临床可靠性,该框架结合了视觉合理性精炼(VPR),利用 VFM 的解剖学知识将适配模型的预测重新锚定到目标图像的低层视觉特征上,有效去除噪声和假阳性。我们进行了迄今为止最广泛的 SFUDA 评估之一,在 10 个域自适应方向和 22 个解剖目标(包括脑、心脏、息肉和腹部目标)上验证了我们的框架。结果表明,Tell2Adapt 始终优于现有方法,在医学图像分割中实现了统一 SFUDA 框架的 SOTA。代码可在 GitHub 获取。
英文原文摘要 1512 字符
Source Free Unsupervised Domain Adaptation (SFUDA) is critical for deploying deep learning models across diverse clinical settings. However, existing methods are typically designed for low-gap, specific domain shifts and cannot generalize into a unified, multi-modalities, and multi-target framework, which presents a major barrier to real-world application. To overcome this issue, we introduce Tell2Adapt, a novel SFUDA framework that harnesses the vast, generalizable knowledge of the Vision Foundation Model (VFM). Our approach ensures high-fidelity VFM prompts through Context-Aware Prompts Regularization (CAPR), which robustly translates varied text prompts into canonical instructions. This enables the generation of high-quality pseudo-labels for efficiently adapting the lightweight student model to target domain. To guarantee clinical reliability, the framework incorporates Visual Plausibility Refinement (VPR), which leverages the VFM's anatomical knowledge to re-ground the adapted model's predictions in target image's low-level visual features, effectively removing noise and false positives. We conduct one of the most extensive SFUDA evaluations to date, validating our framework across 10 domain adaptation directions and 22 anatomical targets, including brain, cardiac, polyp, and abdominal targets. Our results demonstrate that Tell2Adapt consistently outperforms existing approaches, achieving SOTA for a unified SFUDA framework in medical image segmentation. Code are avaliable at GitHub.