面向无人机目标检测的三模态融合Transformer
Tri-Modal Fusion Transformers for UAV-based Object Detection
MAGE 的通道-空间门控与 BiTE 的双向词元级注意力提供了一套可复用的多模态融合模块，可直接插入 ViT 编码器的指定深度；其 61 组消融对融合位置与融合机制的对比，为设计多传感器检测骨干网络提供了实证参考。
原文依据 a Bidirectional Token Exchange (BiTE) module performs bidirectional token-level attention with depthwise-pointwise refinement
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论可靠的无人机目标检测需要在光照变化、运动模糊以及抑制RGB线索的场景动态下保持鲁棒性。热长波红外（LWIR）传感在低光条件下保持对比度，事件相机保留微秒级时序边缘，但在统一检测器中整合全部三种模态尚未得到系统研究。我们提出一个三模态框架，采用双流层次化视觉Transformer处理RGB、热红外与事件数据。在选定的编码器深度上，模态感知门控交换（MAGE）施加传感器间的通道与空间门控，双向词元交换（BiTE）模块执行带深度可分离-逐点细化的双向词元级注意力，为标准的特征金字塔与两阶段检测器生成保持分辨率的融合特征图。我们引入一个包含10,489帧的无人机数据集，具有同步且预对齐的RGB-热红外-事件数据流，以及跨昼夜飞行的24,223个标注车辆。通过61组受控消融实验，我们评估了融合位置、融合机制（基线MAGE+BiTE、CSSA、GAFF）、模态子集与骨干网络容量。三模态融合优于所有双模态基线，融合深度具有显著影响，轻量级CSSA变体以极小代价恢复了大部分收益。本工作为基于无人机的三模态目标检测提供了首个系统性基准与模块化骨干网络。
英文原文摘要 1437 字符
Reliable UAV object detection requires robustness to illumination changes, motion blur, and scene dynamics that suppress RGB cues. Thermal long-wave infrared (LWIR) sensing preserves contrast in low light, and event cameras retain microsecond-level temporal edges, but integrating all three modalities in a unified detector has not been systematically studied. We present a tri-modal framework that processes RGB, thermal, and event data with a dual-stream hierarchical vision transformer. At selected encoder depths, a Modality-Aware Gated Exchange (MAGE) applies inter-sensor channel and spatial gating, and a Bidirectional Token Exchange (BiTE) module performs bidirectional token-level attention with depthwise-pointwise refinement, producing resolution-preserving fused maps for a standard feature pyramid and two-stage detector. We introduce a 10,489-frame UAV dataset with synchronized and pre-aligned RGB-thermal-event streams and 24,223 annotated vehicles across day and night flights. Through 61 controlled ablations, we evaluate fusion placement, mechanism (baseline MAGE+BiTE, CSSA, GAFF), modality subsets, and backbone capacity. Tri-modal fusion improves over all dual-modal baselines, with fusion depth having a significant effect and a lightweight CSSA variant recovering most of the benefit at minimal cost. This work provides the first systematic benchmark and modular backbone for tri-modal UAV-based object detection.