ReMoE：面向对抗鲁棒视觉Transformer的区域混合专家
ReMoE: Region-Mixture Experts for Adversarially-Robust Vision Transformers
把 ViT 的 FFN 换成区域感知的多粒度专家层，并用注意力引导的 patch-to-region / region-to-patch 路由按空间位置激活专家，为对抗鲁棒性提供区域级归纳偏置；做 ViT 对抗防御或想把 MoE 用作结构先验的工作可直接复用该即插即用设计。
原文依据 a plug-and-play module that replaces the standard feed-forward network (FFN) with a region-aware expert layer
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉Transformer（ViT）在广泛的视觉任务上取得了最优性能，但由于缺乏显式的区域级语义建模，它们仍然极易受到对抗扰动的影响。对抗扰动通常具有局部性和空间结构性，而ViT中全局耦合的自注意力和空间均匀的前馈网络会将局部损坏传播到整幅图像，且不在语义连贯区域内强制一致性。为缓解这一失配，我们提出区域感知混合专家（Region-aware Mixture-of-Experts），即“ReMoE”，这是一个即插即用模块，用区域感知的专家层替换标准前馈网络（FFN）。具体而言，ReMoE 策略性地引入多粒度专家（即全局、中心与区域专家），并将其与一种注意力引导的路由机制相耦合，该机制在 patch-to-region（P2R）与 region-to-patch（R2P）变换上运行。该机制根据每个空间位置的注意力分布自适应地激活最相关的专家，使模型能够在保持全局一致性的同时捕获区域级语义与局部上下文，从而为对抗鲁棒的ViT表征提供更强的归纳偏置。大量实验表明，ReMoE 仅以极小的额外计算开销显著提升了ViT的对抗鲁棒性。我们的代码已公开于 https://github.com/zhongskr0114/ReMoE。
英文原文摘要 1501 字符
Vision Transformers (ViTs) achieve state-of-the-art performance on a wide range of vision tasks, yet they remain highly vulnerable to adversarial perturbations due to the lack of explicit region-level semantic modeling. Adversarial perturbations are typically local and spatially structured, whereas the globally coupled self-attention and spatially uniform feed-forward networks in ViTs propagate local corruptions across the whole image without enforcing consistency within semantically coherent regions. To mitigate this mismatch, we propose Region-aware Mixture-of-Experts, namely "ReMoE", a plug-and-play module that replaces the standard feed-forward network (FFN) with a region-aware expert layer. Specifically, our ReMoE strategically introduces multi-granularity experts (i.e., global, center, and regional) and couples them with an attention-guided routing mechanism that operates on patch-to-region (P2R) and region-to-patch (R2P) transformations. This mechanism adaptively activates the most relevant experts for each spatial location according to its attention profile, enabling the model to capture region-level semantics and local context while preserving global consistency, thereby providing a stronger inductive bias for adversarially robust ViT representations. Extensive experiments demonstrate that our ReMoE substantially improves the adversarial robustness of ViTs with only marginal additional computational cost. Our code is available at https://github.com/zhongskr0114/ReMoE.