由内而外：通过内部工作机制度量视觉 Transformer 的泛化
Inside-Out: Measuring Generalization in Vision Transformers Through Inner Workings
提出用模型内部电路（因果交互）作为无标签泛化代理指标，视觉研究者可在部署前模型选择与部署后分布偏移监控中复用该思路，替代不可靠的置信度指标。
原文依据 using the inner workings of a model, i.e., circuits, as a predictive metric of generalization performance
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论可靠的泛化指标是评估机器学习模型的基础。在标注目标数据稀缺的高风险应用中，评估模型在分布偏移下的泛化性能是一项迫切需求。我们关注两个实际场景：（1）部署前，如何为无标签目标数据选择最佳模型？（2）部署后，如何在分布偏移下监控模型性能？这两种情况的核心需求都是一个可靠且无需标签的代理指标。然而，现有的代理指标（如模型置信度或准确率在线性）往往不可靠，因为它们只评估模型输出，而忽略了产生这些输出的内部机制。我们通过引入一个新视角来解决这一局限：使用模型的内部工作机制，即电路，作为泛化性能的预测指标。利用电路发现，我们提取内部表示之间的因果交互作为电路，并由此推导出针对两个实际场景的两个指标。（1）部署前，我们引入依赖深度偏差，用于度量不同模型在目标数据上的泛化能力。（2）部署后，我们提出电路偏移分数，用于预测模型在不同分布偏移下的泛化性能。在各种任务上，这两个指标与泛化性能的相关性显著提高，分别平均优于现有代理指标 13.4% 和 34.1%。我们的代码可在 https://github.com/deep-real/GenCircuit 获取。
英文原文摘要 1604 字符
Reliable generalization metrics are fundamental to the evaluation of machine learning models. Especially in high-stakes applications where labeled target data are scarce, evaluation of models' generalization performance under distribution shift is a pressing need. We focus on two practical scenarios: (1) Before deployment, how to select the best model for unlabeled target data? (2) After deployment, how to monitor model performance under distribution shift? The central need in both cases is a reliable and label-free proxy metric. Yet existing proxy metrics, such as model confidence or accuracy-on-the-line, are often unreliable as they only assess model output while ignoring the internal mechanisms that produce them. We address this limitation by introducing a new perspective: using the inner workings of a model, i.e., circuits, as a predictive metric of generalization performance. Leveraging circuit discovery, we extract the causal interactions between internal representations as a circuit, from which we derive two metrics tailored to the two practical scenarios. (1) Before deployment, we introduce Dependency Depth Bias, which measures different models' generalization capability on target data. (2) After deployment, we propose Circuit Shift Score, which predicts a model's generalization under different distribution shifts. Across various tasks, both metrics demonstrate significantly improved correlation with generalization performance, outperforming existing proxies by an average of 13.4% and 34.1%, respectively. Our code is available at https://github.com/deep-real/GenCircuit.