BIT:面向可见光-红外行人重识别的基于匹配的双向交互变换网络
BIT: Matching-based Bi-directional Interaction Transformation Network for Visible-Infrared Person Re-Identification
把 VI-ReID 从「在共享嵌入空间里做刚性特征对齐」改成「对可见光-红外图像对做显式的双向匹配交互」,其中 query-aware scoring 与双向特征融合可作为跨模态检索/匹配任务的通用模块复用,尤其适用于红外样本远少于可见光样本的分布偏移场景。
原文依据 BIT adopts a matching-based strategy that explicitly models the interaction between visible and infrared image pairs
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论可见光-红外行人重识别(VI-ReID)是一项具有挑战性的检索任务,其原因在于可见光图像与红外图像之间存在显著的模态差异。现有方法试图通过在共享嵌入空间中学习模态不变特征来弥合这一差异,但往往忽略了模态之间复杂而隐式的相关性。在分布偏移的情况下,这一问题更为严重,因为红外样本的数量通常远少于可见光样本。为应对这些挑战,本文提出一种新的网络,称为双向交互变换(BIT)。BIT 不依赖刚性的特征对齐,而是采用基于匹配的策略,显式地建模可见光与红外图像对之间的交互。具体而言,BIT 采用编码器-解码器架构:编码器提取初步的特征表示,解码器则执行双向特征融合与查询感知打分,以增强跨模态对应关系。据我们所知,BIT 是首个在 VI-ReID 中引入这种由成对匹配驱动的交互的方法。在多个基准上的大量实验表明,BIT 达到了最优性能,凸显了其在 VI-ReID 任务上的有效性。
英文原文摘要 1274 字符
Visible-Infrared Person Re-Identification (VI-ReID) is a challenging retrieval task due to the substantial modality gap between visible and infrared images. While existing methods attempt to bridge this gap by learning modality-invariant features within a shared embedding space, they often overlook the complex and implicit correlations between modalities. This limitation becomes more severe under distribution shifts, where infrared samples are often far fewer than visible ones. To address these challenges, we propose a novel network termed Bi-directional Interaction Transformation (BIT). Instead of relying on rigid feature alignment, BIT adopts a matching-based strategy that explicitly models the interaction between visible and infrared image pairs. Specifically, BIT employs an encoder-decoder architecture where the encoder extracts preliminary feature representations, and the decoder performs bi-directional feature integration and query aware scoring to enhance cross-modality correspondence. To our best knowledge, BIT is the first to introduce such pairwise matching-driven interaction in VI-ReID. Extensive experiments on several benchmarks demonstrate that our BIT achieves state-of-the-art performance, highlighting its effectiveness in the VI-ReID task.