CIGMA：视觉 Transformer 中注意力头的因果信息增益机制归因
CIGMA: Causal Information-Gain Mechanistic Attribution of Attention Heads in Vision Transformers
CIGMA 提供了一种基于信息增益的注意力头级因果归因方法，可用于诊断和修正 ViT 模型对背景虚假相关性的依赖，其双编辑（移除物体/移除上下文）的干预思路可直接迁移到其他视觉模型的可解释性分析与去偏任务中。
原文依据 CIGMA provides a mechanistic, information theoretic explanation by quantifying the importance of each attention head and determining whether it supports the main object or routes spurious background cues.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉 Transformer 往往依赖虚假的背景相关性，而非前景物体特征。现有的模型剪枝方法仅关注提升准确率，缺乏可解释性，且无法验证预测是否真正基于主要前景物体，未能对驱动虚假行为的组件提供因果验证。我们提出因果信息增益机制归因（CIGMA），一个用于解释视觉 Transformer 内部计算的通用框架。CIGMA 通过量化每个注意力头的重要性并判定其支持主要物体还是传递虚假背景线索，提供了一种机制性的信息论解释。它基于模型在两种互补编辑（移除物体区域和移除周围上下文）后的完整预测分布，计算基于 Jensen-Shannon 的信息增益，以衡量对物体与上下文的依赖程度，从而对注意力头进行排序，揭示出携带背景信号的虚假子网络和一组互补的证据对齐注意力头。在 CIFAR-10、CIFAR-100 和 Tiny-ImageNet 上，针对三种 VLM 架构（InternVL2-26B、LLaVA-1.6、LLaVA-1.5-13B）进行评估，CIGMA 相比未修改模型将准确率提升了 7.6 至 24.8 个百分点，同时将背景依赖降低了 79.5% 至 88.1%，显著优于所有基线方法，表明因果的注意力头级干预比词元剪枝或重训练方法能更有效地缓解虚假相关性。
英文原文摘要 1573 字符
Vision Transformers often rely on spurious background correlations rather than foreground object features. While prior model pruning approaches focus solely on improving accuracy, they lack interpretability and fail to verify whether predictions are actually made by focusing on the main foreground object, providing no causal validation of which components drive spurious behavior. We introduce Causal Information Gain Mechanistic Attribution (CIGMA), a general framework for explaining the internal computation of Vision Transformers. CIGMA provides a mechanistic, information theoretic explanation by quantifying the importance of each attention head and determining whether it supports the main object or routes spurious background cues. It ranks attention heads by measuring object versus context reliance with Jensen Shannon based information gain computed from the model's full predictive distributions after two complementary edits, removing the object region and removing the surrounding context, which reveals a spurious subnet that carries background signals and a complementary set of evidence aligned heads. Evaluated on CIFAR-10, CIFAR-100, and Tiny-ImageNet across three VLM architectures (InternVL2-26B, LLaVA-1.6, LLaVA-1.5-13B), CIGMA improves accuracy by 7.6 to 24.8 percentage points over unmodified models while reducing background reliance by 79.5% to 88.1%, substantially outperforming all baselines, demonstrating that causal head-level interventions enable more effective spurious correlation mitigation than token pruning or retraining approaches.