HyperNAS：通过超网络增强 NAS 预测器的架构表征
HyperNAS: Enhancing Architecture Representation for NAS Predictor via Hypernetwork
对做视觉模型选型的人，HyperNAS 提供了一种用共享超网络作为辅助任务来增强架构编码、从而在极少代理样本下预测 ViT/CNN 精度的方法，可直接用于低预算的架构筛选与搜索。
原文依据 HyperNAS strikes new state-of-the-art results, with 97.60% top-1 accuracy on CIFAR-10 and 82.4% top-1 accuracy on ImageNet, using at least 5.0xfewer samples.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论耗时的性能评估严重阻碍了神经架构搜索（NAS）的进展。为解决这一问题，神经预测器利用在代理数据集上训练得到的代理模型，从而可以直接对新架构进行性能预测。然而，这些预测器往往泛化能力较差，原因是其捕获不同架构之间复杂关系的能力有限。本文提出 HyperNAS，一种用于增强架构表征学习的新型神经预测器范式。HyperNAS 由两个主要部分组成：全局编码方案与共享超网络。全局编码方案旨在捕获全面的宏观结构信息，而共享超网络则作为辅助任务，用于加强对架构间模式的探究。为保证训练稳定性，我们进一步设计了动态自适应多任务损失，以促进在帕累托前沿上的个性化探索。在包括 ViT 在内的五个代表性搜索空间上的大量实验证明了 HyperNAS 的优势，尤其是在少样本场景下。例如，HyperNAS 取得了新的最优结果：在 CIFAR-10 上 top-1 准确率为 97.60%，在 ImageNet 上 top-1 准确率为 82.4%，且所用样本至少减少 5.0 倍。
英文原文摘要 1318 字符
Time-intensive performance evaluations significantly impede progress in Neural Architecture Search (NAS). To address this, neural predictors leverage surrogate models trained on proxy datasets, allowing for direct performance predictions for new architectures.However, these predictors often exhibit poor generalization due to their limited ability to capture intricate relationships among various architectures. In this paper, we propose HyperNAS, a novel neural predictor paradigm for enhancing architecture representation learning. HyperNAS consists of two primary components: a global encoding scheme and a shared hypernetwork. The global encoding scheme is devised to capture the comprehensive macro-structure information, while the shared hypernetwork serves as an auxiliary task to enhance the investigation of inter-architecture patterns. To ensure training stability, we further develop a dynamic adaptive multi-task loss to facilitate personalized exploration on the Pareto front. Extensive experiments across five representative search spaces, including ViTs, demonstrate the advantages of HyperNAS, particularly in few-shot scenarios. For instance, HyperNAS strikes new state-of-the-art results, with 97.60% top-1 accuracy on CIFAR-10 and 82.4% top-1 accuracy on ImageNet, using at least 5.0xfewer samples.