SOTA：面向多基础模型零样本分类的自适应最优传输
SOTA: Self-adaptive Optimal Transport for Zero-Shot Classification with Multiple Foundation Models
提出一种免训练的自适应最优传输集成框架，可即插即用地融合多个视觉或视觉语言基础模型的输出，提升零样本分类性能，为多模型集成提供新思路。
原文依据 we propose SOTA (Self-adaptive Optimal TrAnsport), a training-free ensemble framework that integrates the outputs of multiple foundation models
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论基础模型因其强大的零样本分类能力而在各领域受到广泛关注。本文的动机源于两个关键观察：（1）视觉语言模型（如CLIP）往往过度依赖类别级文本先验，难以捕捉细粒度视觉线索，而纯视觉基础模型（如DINO）能提供丰富且具判别力的视觉特征，但缺乏语义对齐；（2）由于预训练差异，不同视觉语言模型在不同数据集上的性能差异显著。为解决这些挑战，我们提出SOTA（自适应最优传输），一种免训练的集成框架，通过学习自适应传输方案来整合多个基础模型（纯视觉或视觉语言模型）的输出。值得注意的是，SOTA无需先验且能自动平衡模型贡献。在自然图像、医学病理和遥感等多个领域的广泛实验验证了SOTA的泛化性。结果一致表明，它有效利用了不同基础模型的互补优势，并相较于单个模型取得了显著提升。实现代码见：https://github.com/Afleve/self-adaptive-Optimal-Transport。
英文原文摘要 1310 字符
Foundation models have attracted widespread attention across domains due to their powerful zero-shot classification capabilities. This work is motivated by two key observations: (1) Vision-Language Models (VLMs), such as CLIP, often over-rely on class-level textual priors and struggle to capture fine-grained visual cues, whereas Vision-only Foundation Models (VFMs), such as DINO, provide rich and discriminative visual features but lack semantic alignment; (2) the performance of different VLMs varies considerably across datasets owing to differences in pre-training. To address these challenges, we propose SOTA (Self-adaptive Optimal TrAnsport), a training-free ensemble framework that integrates the outputs of multiple foundation models (VFMs or VLMs) by learning a self-adaptive transport plan. Notably, SOTA is prior-free and automatically balances model contributions. Extensive experiments across diverse domains, including natural images, medical pathology, and remote sensing, validate the generalizability of SOTA. The results consistently show that it effectively leverages the complementary strengths of different foundation models and achieves substantial improvements over individual models. The implementation code is available at: https://github.com/Afleve/self-adaptive-Optimal-Transport.