从权重到概念：基于奇异向量分解的 CLIP 无数据可解释性
From Weights to Concepts: Data-Free Interpretability of CLIP via Singular Vector Decomposition
提供一种无需数据、无需训练的权重空间分析手段：对 CLIP 视觉 Transformer 每个注意力头的 value-output 矩阵做奇异向量分解，并用稀疏匹配追踪把奇异向量解释为可读概念，可直接用于定位并放大或抑制特定语义方向，从而在不重训练的情况下编辑模型行为。
原文依据 a fully data-free, training-free framework that directly analyzes CLIP's vision transformer in weight space
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论随着视觉语言模型的大规模部署，理解其内部机制变得愈发关键。现有的可解释性方法主要依赖激活值，因而依赖数据集、易受数据偏置影响，且往往局限于粗粒度的注意力头级解释。我们提出 SITH（Semantic Inspection of Transformer Heads），一个完全无需数据、无需训练的框架，直接在权重空间中分析 CLIP 的视觉 Transformer。对于每个注意力头，我们将其 value-output 矩阵分解为奇异向量，并通过 COMP（Coherent Orthogonal Matching Pursuit）这一新算法对每个奇异向量进行解释，将其表示为人类可理解概念的稀疏且语义连贯的组合。我们表明 SITH 能够给出连贯、忠实的头内解释，并通过重建保真度与可解释性实验加以验证。这使得我们能够利用 SITH 进行精确、可解释的权重空间模型编辑，以放大或抑制特定概念，从而在不重新训练的情况下提升下游性能。此外，我们利用 SITH 研究模型适配过程，表明微调主要是在一个稳定的语义基上重新加权，而非学习全新的特征。
英文原文摘要 1226 字符
As vision-language models are deployed at scale, understanding their internal mechanisms becomes increasingly critical. Existing interpretability methods predominantly rely on activations, making them dataset-dependent, vulnerable to data bias, and often restricted to coarse head-level explanations. We introduce SITH (Semantic Inspection of Transformer Heads), a fully data-free, training-free framework that directly analyzes CLIP's vision transformer in weight space. For each attention head, we decompose its value-output matrix into singular vectors and interpret each one via COMP (Coherent Orthogonal Matching Pursuit), a new algorithm that explains them as sparse, semantically coherent combinations of human-interpretable concepts. We show that SITH yields coherent, faithful intra-head explanations, validated through reconstruction fidelity and interpretability experiments. This allows us to use SITH for precise, interpretable weight-space model edits that amplify or suppress specific concepts, improving downstream performance without retraining. Furthermore, we use SITH to study model adaptation, showing how fine-tuning primarily reweights a stable semantic basis rather than learning entirely new features.