NESTOR：面向大规模偏微分方程预训练的嵌套混合专家神经算子
NESTOR: A Nested MOE-based Neural Operator for Large-Scale PDE Pre-Training
嵌套 MoE 神经算子（图像级 MoE 捕捉全局依赖 + token 级 Sub-MoE 捕捉局部依赖）为视觉中的算子学习与大规模预训练提供了可复用的稀疏专家路由结构，可迁移到神经场、物理仿真等连续场建模任务。
原文依据 the image-level MoE is designed to capture global dependencies, while the token-level Sub-MoE focuses on local dependencies
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论神经算子已成为求解偏微分方程的一种高效范式，它克服了传统数值方法的局限，并显著提升了计算效率。然而，由于偏微分方程系统的多样性与复杂性，现有神经算子通常依赖单一网络架构，这限制了其充分捕捉异构特征与复杂系统依赖关系的能力。这一约束构成了基于神经算子的大规模偏微分方程预训练的瓶颈。为应对这些挑战，我们提出了一种基于嵌套混合专家（MoE）框架的大规模偏微分方程预训练神经算子。具体而言，图像级 MoE 用于捕捉全局依赖关系，而 token 级 Sub-MoE 则聚焦于局部依赖关系。我们的模型能够针对给定输入有选择地激活最合适的专家网络，从而增强泛化性与可迁移性。我们在来自多种来源的十二个偏微分方程数据集上开展大规模预训练，并成功将模型迁移至下游任务。大量实验验证了本方法的有效性。代码已公开于：https://github.com/Event-AHU/OpenFusion/tree/main/NESTOR。
英文原文摘要 1239 字符
Neural operators have emerged as an efficient paradigm for solving PDEs, overcoming the limitations of traditional numerical methods and significantly improving computational efficiency. However, due to the diversity and complexity of PDE systems, existing neural operators typically rely on a single network architecture, which limits their capacity to fully capture heterogeneous features and complex system dependencies. This constraint poses a bottleneck for large-scale PDE pre-training based on neural operators. To address these challenges, we propose a large-scale PDE pre-trained neural operator based on a nested Mixture-of-Experts (MoE) framework. In particular, the image-level MoE is designed to capture global dependencies, while the token-level Sub-MoE focuses on local dependencies. Our model can selectively activate the most suitable expert networks for a given input, thereby enhancing generalization and transferability. We conduct large-scale pre-training on twelve PDE datasets from diverse sources and successfully transfer the model to downstream tasks. Extensive experiments demonstrate the effectiveness of our approach. The code is publicly available at: https://github.com/Event-AHU/OpenFusion/tree/main/NESTOR.