面向持续学习的谱混合专家
Spectral Mixture-of-Experts for Continual Learning
把专家参数约束到互不相交的频域子空间以获得先验正交更新，并用离线专家重要性记忆约束路由漂移，这套「频域正交 + 双路由」思路可直接移植到视觉大模型的持续微调与多任务适配器合并中。
原文依据 Spectral Experts are parameterized using unique, disjoint spectral masks to confine their learnable parameters to distinct frequency subspaces
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论基于混合专家（MoE）的参数高效微调是持续学习（CL）的一种有前景的方案，但它存在两个关键失效模式：结构干扰，即专家更新之间相互干扰；以及组合遗忘，即模型的路由策略发生漂移。为解决这些问题，我们提出 Spectral MoE，一个由三个核心组件构成的、专为持续学习设计的新框架。首先，谱专家（Spectral Experts）使用互不相交的独特谱掩码进行参数化，将其可学习参数限制在互不相同的频率子空间中，从而确保先验正交的更新，防止结构干扰。其次，双路由（Dual-Router）机制将学习新任务的在线路由与归档历史专家重要性的离线记忆解耦。最后，该离线记忆支撑了动态一致性投影（Dynamic Consistency Projection），这是一种几何约束，可抑制路由漂移，并依据专家过去的贡献自适应地保护专家，从而缓解组合遗忘。在严格的跨域持续学习基准上验证，我们的框架显著优于现有方法，在新任务上展现出更优的知识保持能力与可塑性。代码见：https://github.com/ouycc/Spectral_MoE。
英文原文摘要 1243 字符
While Parameter-Efficient Fine-Tuning using Mixture-of-Experts (MoE) is a promising solution for continual learning (CL), it suffers from two critical failure modes: structural interference, where expert updates interfere, and compositional forgetting, where the model's routing policy drifts. To address these issues, we introduce Spectral MoE, a novel framework built for CL from three core components. First, Spectral Experts are parameterized using unique, disjoint spectral masks to confine their learnable parameters to distinct frequency subspaces, ensuring a priori orthogonal updates that prevent structural interference. Second, a Dual-Router mechanism decouples online routing that learns new tasks from an offline memory that archives historical expert importance. Finally, this offline memory enables a Dynamic Consistency Projection, a geometric constraint that suppresses router drift and adaptively shields experts based on their past contributions, mitigating compositional forgetting. Validated on a strict cross-domain CL benchmark, our framework significantly outperforms existing methods, demonstrating superior knowledge retention and plasticity for new tasks. Code is available at: https://github.com/ouycc/Spectral_MoE.