解锁预训练权重：面向零样本初始化的参数继承
Unlocking Pre-trained Weights: Parameter Inheritance for Zero-Shot Initialization
对视觉研究者而言，这是一种无需微调即可为任意配置的 ViT 等骨干网络生成可用初始权重的参数投影机制，可直接用于新架构的快速冷启动与预训练权重复用。
原文依据 introduces a novel parameter projection mechanism to directly inherit parameters from pre-trained models for initializing target networks of varying configurations
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论恰当的参数初始化对于降低深度神经网络的训练成本至关重要。图超网络（Graph HyperNetworks, GHN）已成为为多种架构进行初始化的一种有前景的方法，近期诸如任务感知学习基因（Task-Aware Learngene, TAL）等方法进一步尝试通过软标签监督来利用预训练模型的知识。然而，这种间接监督未能充分利用预训练权重中编码的丰富信息。我们提出参数继承超网络（Parameter InheriTance HyperNetwork, PITH），引入一种新颖的参数投影机制，直接从预训练模型中继承参数，以初始化配置各异的目标网络。我们的方法使初始化后的网络无需任何进一步训练即可在下游任务上直接取得有竞争力的性能，我们将此称为零样本初始化。大量实验证明了 PITH 的优越性：由 PITH 初始化的 ViT-Base 在 ImageNet-1K 上取得 53.35% 的零样本准确率，超过此前最优方法 6.54%，并在多个下游任务上取得一致的提升。
英文原文摘要 1131 字符
Appropriate parameter initialization is crucial for reducing the training cost of deep neural networks. Graph HyperNetworks (GHN) have emerged as a promising approach for initializing diverse architectures, with recent methods such as Task-Aware Learngene (TAL) further attempting to leverage pre-trained model knowledge via soft label supervision. However, such indirect supervision fails to fully exploit the rich information encoded in pre-trained weights. We propose **P**arameter **I**nheri**T**ance **H**yperNetwork (**PITH**), which introduces a novel parameter projection mechanism to directly inherit parameters from pre-trained models for initializing target networks of varying configurations. Our method enables initialized networks to directly achieve competitive performance on downstream tasks without any further training, which we term zero-shot initialization. Extensive experiments demonstrate the superiority of PITH: ViT-Base initialized by PITH achieves 53.35% zero-shot accuracy on ImageNet-1K, surpassing the previous state-of-the-art by 6.54%, with consistent improvements across multiple downstream tasks.