模型何时真正做出决策？映射预训练神经网络中的逐层决策时间线
When Do Models Actually Decide? Mapping the Layer-Wise Decision Timeline in Pretrained Neural Networks
该论文通过逐层线性探针揭示了预训练ResNet在前向传播中决策形成的层间时间线，为设计基于稳定性的早退机制提供了量化依据，并指出简单图像可能在网络早期即被决定，这对视觉模型推理加速和动态计算分配有直接启发。
原文依据 We investigate when pretrained models make up their minds during a forward pass by training linear probes at each layer of ResNet variants on ImageNet, without modifying the base model.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论神经网络常被视为整体黑箱，对所有输入统一经过所有层处理。然而，研究者直观地疑问：简单图像是否需要ResNet-50的全部50层，还是预测实际上在更早阶段就已确定？我们通过在ImageNet上为ResNet变体的每一层训练线性探针，且不修改基础模型，来研究预训练模型在前向传播中何时做出决定。我们的发现揭示了不同架构间显著的计算异质性。我们发现了明显的双峰模式，存在早期决策者和晚期决策者两个不同群体：在更深的ResNet中，33%至39%的样本在网络前三分之一内稳定，而39%至42%的样本被分配到最终30%的深度；在所有三个ResNet中，晚期比例范围为39%至54%。决策层对稳定性标准高度敏感，平均深度从2.6至4.1（k=1）增加到9.0至10.0（k=4）。线性探针准确率在最后残差阶段出现急剧跃升，ResNet-50/101达到73%至75%，ResNet-18达到65%，表明语义整合发生在较晚阶段。这些发现揭示了标准推理中的计算异质性，并阐明了基于稳定性的早退机制的前景与局限。
英文原文摘要 1292 字符
Neural networks are often treated as monolithic black boxes that process all inputs uniformly through all layers. However, researchers intuitively wonder: do simple images require all 50 layers of ResNet-50, or is the prediction effectively decided much earlier? We investigate when pretrained models make up their minds during a forward pass by training linear probes at each layer of ResNet variants on ImageNet, without modifying the base model. Our findings reveal substantial computational heterogeneity across architectures. We discover pronounced bimodal patterns with distinct populations of early and late deciders: in deeper ResNets, 33--39% of samples stabilize within the first third of the network, while 39--42% are assigned to the final 30% of depth; across all three ResNets, the late fraction ranges from 39% to 54%. The decision layer is highly sensitive to stability criteria, with mean depths increasing from 2.6--4.1 (k=1) to 9.0--10.0 (k=4). Linear probe accuracy exhibits sharp jumps in final residual stages, reaching 73--75% for ResNet-50/101 and 65% for ResNet-18, indicating that semantic consolidation occurs late. These findings expose computational heterogeneity in standard inference and clarify both the promise and the limits of stability-based early exiting.