选择即非线性：通过联合博弈决策视角桥接注意力与激活，实现可解释、判别性的视觉表征
Selection-as-Nonlinearity: Bridging Attention and Activation via a Joint Game-Decision Lens for Interpretable, Discriminative Visual Representations
提出将注意力视为成本约束下的选择操作，为理解视觉Transformer中注意力与FFN的互补性提供新视角，并据此设计CSaN模块，可即插即用地增强Swin、ViT、Hiera等骨干网络，在视觉基准上以更小规模媲美两倍大的同族模型。
原文依据 We term this the weak-independence challenge of attention.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论在温和条件下，具有独立前后投影的自注意力可以成为通用近似器（在紧致域上）。然而我们观察到一个显著差距：仅含注意力的Transformer（无FFN层）相对于标准交错的注意力-FFN基线表现出明显的准确率下降。我们将此称为注意力的弱独立性挑战。我们通过一个新的概念视角——选择即非线性（SaN）来研究这一问题，该视角将有效非线性解释为有向的、成本约束的选择，为注意力作为上下文门控激活提供了一致的解释。在这种联合博弈决策观点中，注意力对值执行资源约束的合作分配：每个查询在共享值上分配单位质量权重预算以优化表征效用，受归一化器（如softmax）约束，并由上下文导出的分数（如q-k相似度）引导。SaN将弱独立性解释为一种结构性张力：在共享预算下，值权重几乎无法同时达到解耦的逐查询（行方向）和逐值（列方向）最优，从而限制了注意力的独立能力。在SaN的指导下，我们引入了CSaN，一种可解释、高效的注意力补偿范式，具有两个关键见解：1）分层预算校准，通过查询间校正信号重新分配行预算；2）公共-私有合作，通过逐令牌私有值路径增强公共注意力路径，以解耦冲突需求。CSaN在各种视觉基准上评估，并在流行的Transformer家族（Swin、ViT、Hiera）中展现出显著增益，使模型能够媲美大得多的同族对应模型（约2倍大）。
英文原文摘要 1749 字符
Self-attention with separate pre- and post-projections can be a universal approximator (on compact domains) under mild conditions. Yet we observe a striking gap: an attention-only Transformer (w/o FFN layers) exhibits a marked accuracy drop relative to its standard interleaved attention--FFN baseline. We term this the weak-independence challenge of attention. We study this through a new conceptual lens, Selection-as-Nonlinearity (SaN), which interprets effective nonlinearity as directed, cost-constrained selection, offering a coherent account of attention as context-gated activation. In this joint game-decision view, attention performs a resource-constrained cooperative allocation over values: each query distributes a unit-mass weight budget over shared values to optimize representational utility, under a normalizer (e.g., \mathrm softmax ), and guided by context-derived scores (e.g., q-k similarities). SaN interprets weak-independence as a structural tension: the value weights almost cannot simultaneously attain the decoupled per-query (row-wise) and the per-value (column-wise) optimums under shared budgets, thereby limiting attention's stand-alone capacity. Guided by SaN, we introduce CSaN, an interpretable, efficient attention compensation paradigm with two key insights: 1) hierarchical budget calibration, re-allocate row budgets via inter-query correction signals; and 2) public-private cooperation, enhancing the public attention pathway with a per-token private value pathway to decouple conflicting demands. CSaN is evaluated on various vision benchmarks and demonstrates remarkable gains across popular Transformer families (Swin, ViT, Hiera), enabling models to rival much heavier same-family counterparts ~2xas large.