脉冲 Transformer 中的时序交互与多延迟混合器
Temporal Interaction in Spiking Transformers with Multi-Delay Mixer
MD-Mixer 作为即插即用模块替换脉冲 Transformer 中的空间自注意力，为事件相机等神经形态视觉输入提供显式时序混合，可直接用于事件驱动视觉骨干网络的改造。
原文依据 it can be integrated seamlessly into existing Spiking Transformers as a drop-in replacement while maintaining computational efficiency
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论脉冲神经网络（SNN）因其事件驱动的计算范式而受到广泛关注，在神经形态计算中具有广阔前景。近年来，SNN 与 Transformer 架构的结合在各类任务上取得了显著进展。然而，现有的脉冲自注意力机制主要关注空间信息，忽略了显式的时序建模，导致性能欠佳。本文引入时序交互系数（TIC）来分析这些仅关注空间的注意力机制中的时序依赖模式，揭示其时序交互有限、模式多样性受限的问题。为解决该问题，我们受神经系统时间延迟机制的启发，提出多延迟混合器（MD-Mixer）。具体而言，MD-Mixer 引入多个时间延迟以执行有效的时间混合，并促进时序信息更丰富的空间注意力。此外，它可以作为即插即用模块无缝集成到现有脉冲 Transformer 中，同时保持计算效率。在静态与神经形态基准上的大量评估表明，MD-Mixer 显著提升了脉冲 Transformer 的性能，优于现有最优方法。本工作确立了 MD-Mixer 作为事件驱动架构中时序建模的有效且通用的解决方案。
英文原文摘要 1453 字符
Spiking Neural Networks (SNNs) have gained significant attention due to their event-driven computational paradigm, making them promising for neuromorphic computing. In recent years, the integration of SNNs and Transformer architectures has made remarkable progress in various tasks. However, existing spiking self-attention mechanisms predominantly focus on spatial information while neglecting explicit temporal modeling, leading to suboptimal performance. In this paper, we introduce the Temporal Interaction Coefficient (TIC) to analyze temporal dependency patterns in these spatial-only attention mechanisms, revealing their limited temporal interactions and restricted pattern diversity. To overcome this issue, we propose the Multi-Delay Mixer (MD-Mixer), drawing inspiration from time delay mechanisms in the nervous system. Specifically, MD-Mixer introduces multiple temporal delays to perform effective time mixing and facilitate temporally enriched spatial attention. In addition, it can be integrated seamlessly into existing Spiking Transformers as a drop-in replacement while maintaining computational efficiency. Extensive evaluations on static and neuromorphic benchmarks demonstrate that MD-Mixer substantially improves the performance of Spiking Transformers, outperforming existing state-of-the-art (SOTA) methods. This work establishes MD-Mixer as an effective and general solution for temporal modeling in event-driven architectures.