InTrain：面向零成本神经架构搜索的内在可训练性
InTrain: Intrinsic Trainability for Zero-Cost Neural Architecture Search
对做视觉骨干网络选型的人，InTrain 提供了一个无需训练即可给候选架构打分的统一代理：用激活协方差特征谱的参与率衡量表征有效维度、用逐层梯度健康度衡量深层反传稳定性，可直接嵌入 NAS 搜索循环替代昂贵的训练式评估。
原文依据 Training-free neural architecture search promises efficient discovery of high-performance networks without costly training.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论免训练神经架构搜索有望在无需昂贵训练的情况下高效发现高性能网络。然而，现有的零成本代理依赖碎片化的启发式规则，未能回答一个根本问题：是什么使一个架构具有可训练性？本文提出内在可训练性（InTrain），一个统一的理论代理，将可训练性形式化为一种架构不变量，它由两个协同成分产生：几何容量与优化韧性。我们通过对神经信息处理的分析来操作化内在可训练性。几何容量通过激活协方差特征谱的参与率来量化，刻画表征流形的有效维度。优化韧性通过累积梯度健康度来度量，评估反向传播在网络深度上的鲁棒性。InTrain 通过一种尺度不变的乘性耦合综合这两个维度，我们验证了该耦合对于捕捉二者协同的、非可加的关系是必要的。在标准 NAS 基准与搜索空间上的大量实验表明，InTrain 取得了与最优的基于集成的代理相当的排序相关性，并优于其他单指标方法。
英文原文摘要 1304 字符
Training-free neural architecture search promises efficient discovery of high-performance networks without costly training. However, existing zero-cost proxies rely on fragmented heuristics that fail to capture the fundamental question: what makes an architecture trainable? This paper introduces Intrinsic Trainability (InTrain), a unified theoretical proxy that formalizes trainability as an architectural invariant emerging from two synergistic components: geometric capacity and optimization resilience. We operationalize intrinsic trainability through analysis of neural information processing. Geometric capacity is quantified via the participation ratio of activation covariance eigenspectrum, capturing the effective dimensionality of representation manifolds. Optimization resilience is measured through cumulative gradient health, assessing the robustness of backpropagation across network depth. InTrain synthesizes these dimensions through a scale-invariant multiplicative coupling, which we validate is essential for capturing their synergistic, non-additive relationship. Extensive experiments on standard NAS benchmarks and search spaces demonstrate that InTrain achieves ranking correlations on par with state-of-the-art ensemble-based proxies and outperforms other single-metric methods.