渐进式神经架构生成
Progressive Neural Architecture Generation
对做视觉模型设计的读者，PNAG 提供了一种用多尺度子架构量化加逐步一致性约束来直接生成有效架构的范式，可替代贝叶斯优化中的随机采样环节，把架构搜索的生成时间降低约 1300 倍，并可直接生成 Transformer 类视觉骨干。
原文依据 reducing the architecture generation time by 1300x
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论作为神经架构搜索中的代表性技术，神经架构生成旨在为给定任务直接构造高性能架构。它有望替代部分搜索策略中低效的随机探索环节，例如贝叶斯优化中的采集策略。尽管已有大量研究，当前的架构生成技术仍面临生成效率低、约束不足等问题，导致生成出无效架构。为此，我们提出渐进式神经架构生成（PNAG），通过由粗到细的演化以增量方式构造架构，从而提升生成效率，并引入逐步细化以确保所生成架构的有效性。为实现这一点，PNAG 包含两个模块：多尺度子架构量化（MSQ）与逐步一致性约束（SCC）。具体而言，MSQ 利用量化解码构造子架构并逐步扩展，从简单形态过渡到复杂形态。该操作绕过了网络推理以提升效率。作为对 MSQ 的补充，SCC 通过一种定制的正则化机制实现，在子架构生成过程中对偏差施加惩罚，引导过程朝向有效的目标架构。由此，PNAG 建立了一条清晰的生成路径，为在下游任务中生成合适的架构奠定基础。大量实验表明，PNAG 不仅能为各类下游任务生成更优的架构（平均提升 +8.43%/+5.07%），还显著提升了生成效率，将架构生成时间减少约 1300 倍。此外，PNAG 展现出很强的可扩展性，能够成功生成基于 Transformer 的架构。
英文原文摘要 1856 字符
As a representative technique in neural architecture search, neural architecture generation aims to construct high-performance architectures for a given task directly. It is poised to replace the inefficient random exploration components of some search strategies, such as the acquisition strategies in Bayesian optimization. Despite significant research, current architecture generation techniques face problems such as low generation efficiency and insufficient constraints, leading to invalidly generated architectures. To this end, we propose Progressive Neural Architecture Generation (PNAG), which constructs architectures incrementally through coarse-to-fine evolution, enhancing generation efficiency, and incorporates step-wise refinements to ensure the validity of the generated architecture. To achieve this, PNGA involves two modules, multi-scale sub-architecture quantization (MSQ) and step-wise consistency constraint (SCC). Specifically, MSQ constructs sub-architectures using quantization decoding and progressively expands them, transitioning from simple to complex forms. This operation bypasses network inference to enhance efficiency.Complementing MSQ, SCC, implemented through a tailored regularization mechanism, introduces penalties for deviations during sub-architecture generation, guiding the process towards valid target architectures. As such, PNAG establishes a clear generation path, laying the groundwork for generating suitable architectures in downstream tasks. Extensive experiments demonstrate that PNAG not only generates superior architectures for various downstream tasks (+8.43%/+5.07%, on average) but also significantly improves generation efficiency, reducing the architecture generation time by 1300x. Furthermore, PNAG demonstrates strong extensibility by successfully generating Transformer-based architectures.