OneSparse：视觉模型中稀疏激活层的统一框架
OneSparse: A Unified Framework for Sparse Activation Layers in Vision Models
提出统一的稀疏激活层抽象（OneSparse）和Nexus层，将MoE与记忆模块结合，为视觉模型提供可插拔的高效模块，可直接用于替换CNN/Transformer中的FFN或卷积层以提升性能-效率前沿。
原文依据 we design the Nexus Layer, which features two key innovations: a unified routing mechanism that combines the efficiency of memory retrieval with MoE's load balancing
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论稀疏激活层，主要是混合专家（MoE）和基于记忆的模块，已成为扩展大模型的核心方法，并在视觉任务中日益受到关注。尽管概念上相似，这些范式却独立发展，阻碍了系统比较以及利用其互补优势的模块开发。为弥合这一差距，我们提出OneSparse，一个统一框架，在共同抽象下重新表述MoE和记忆模块。这使得能够进行系统比较和集成，揭示出一个连续的设计空间。在此抽象指导下，我们设计了Nexus层，具有两项关键创新：统一路由机制，结合记忆检索的效率与MoE的负载均衡，以确保稳定且可扩展的词元分配；以及自适应处理策略，其中记忆模块勾勒粗略表示，而专家模块细化关键区域。在图像分类、目标检测和语义分割上的大量实验表明，我们的Nexus层建立了新的性能-效率前沿，在卷积和Transformer架构上超越了代表性稀疏基线。这些结果验证了OneSparse框架统一和集成互补稀疏范式的力量，并强调了混合稀疏建模在视觉中的潜力。
英文原文摘要 1420 字符
Sparse activation layers, primarily Mixture-of-Experts (MoE) and memory-based modules, have become a central approach for scaling large models and are gaining traction in vision tasks. Despite conceptual similarities, these paradigms have evolved independently, hindering systematic comparison and the development of modules that exploit their complementary strengths. To bridge this gap, we propose OneSparse, a unified framework that reformulates MoE and memory modules under a common abstraction. This enables their systematic comparison and integration, revealing a continuous design space. Guided by this abstraction, we design the Nexus Layer, which features two key innovations: a unified routing mechanism that combines the efficiency of memory retrieval with MoE's load balancing to ensure stable and scalable token assignment, and an adaptive processing strategy where memory modules sketch coarse representations while expert modules refine critical regions. Extensive experiments on image classification, object detection, and semantic segmentation demonstrate that our Nexus Layer establishes a new performance--efficiency frontier, surpassing representative sparse baselines across convolutional and Transformer architectures. These results validate the power of the OneSparse framework to unify and integrate complementary sparse paradigms and underscore the potential of hybrid sparse modeling in vision.