概念区域至关重要：一种基于聚类重要性的 CLIP 基准测试方法
Concept Regions Matter: Benchmarking CLIP with a New Cluster-Importance Approach
CCI 利用 CLIP 自身的图像块嵌入做聚类并掩码评估预测变化，为视觉研究者提供了一种无需额外训练即可定位模型决策依据的归因工具；结合 GroundedSAM 可自动区分前景/背景驱动预测，COVAR 基准则系统解耦前景与背景变化，可用于诊断和提升 VLM 的鲁棒性。
原文依据 We introduce Cluster-based Concept Importance (CCI), a novel interpretability method that uses CLIP's own patch embeddings to group spatial patches into semantically coherent clusters
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论对比视觉语言模型（VLM）如 CLIP 实现了强大的零样本识别，但仍易受虚假关联影响，尤其是过度依赖背景。我们提出基于聚类的重要性（CCI），一种新颖的可解释性方法，利用 CLIP 自身的图像块嵌入将空间图像块分组为语义连贯的聚类，对其进行掩码，并评估模型预测的相对变化。CCI 在忠实性基准上达到新的最优水平，大幅超越先前方法；例如，在 MS COCO 检索任务上，其删除-AUC 指标提升超过两倍。我们进一步提出，CCI 与 GroundedSAM 结合可自动将预测归类为前景或背景驱动，提供关键的诊断能力。然而，现有基准如 CounterAnimals 仅依赖准确率，并隐含地将所有性能下降归因于背景关联。我们的分析表明这一假设并不完整，因为许多错误源于视角变化、尺度偏移和细粒度物体混淆。为解耦这些效应，我们引入 COVAR，一个系统变化物体前景和背景的基准。利用 CCI 与 COVAR，我们对十八种 CLIP 变体进行了全面评估，提供了方法学进展与经验证据，为通向更鲁棒的视觉语言模型铺平道路。
英文原文摘要 1464 字符
Contrastive vision-language models (VLMs) such as CLIP achieve strong zero-shot recognition yet remain vulnerable to spurious correlations, particularly background over-reliance. We introduce Cluster-based Concept Importance (CCI), a novel interpretability method that uses CLIP's own patch embeddings to group spatial patches into semantically coherent clusters, masking them, and evaluating relative changes in model predictions. CCI sets a new state of the art on faithfulness benchmarks, surpassing prior methods by large margins; for example, it yields more than a twofold improvement on the deletion-AUC metric for MS COCO retrieval. We further propose that CCI when combined with GroundedSAM, automatically categorizes predictions as foreground or background-driven, providing a crucial diagnostic ability. Existing benchmarks such as CounterAnimals, however, rely solely on accuracy and implicitly attribute all performance degradation to background correlations. Our analysis shows this assumption to be incomplete, since many errors arise from viewpoint variation, scale shifts, and fine-grained object confusions. To disentangle these effects, we introduce COVAR, a benchmark that systematically varies object foregrounds and backgrounds. Leveraging CCI with COVAR, we conduct a comprehensive evaluation of eighteen CLIP variants, providing both methodological advances and empirical evidence that chart a path toward more robust vision-language models.