RAGTrack：基于检索增强生成的语言感知RGBT跟踪
RAGTrack: Language-aware RGBT Tracking with Retrieval-Augmented Generation
将检索增强生成（RAG）引入RGB-T跟踪，利用MLLM自动生成文本描述并构建动态知识库，通过时序语言推理增强目标建模，为多模态跟踪中的外观变化和背景干扰问题提供了可复用的语言引导思路。
原文依据 we propose a Context-aware Reasoning Module (CRM) to maintain a dynamic knowledge base and employ a Retrieval-Augmented Generation (RAG) to enable temporal linguistic reasoning for robust target modeling.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论RGB-热红外（RGBT）跟踪旨在通过融合可见光和热红外模态，在各种环境条件下实现鲁棒的目标定位。然而，现有的RGBT跟踪器仅依赖初始帧的视觉信息进行目标建模，由于缺乏语言引导而无法适应外观变化。此外，当前方法受限于冗余的搜索区域和异构模态间隙，导致背景干扰。为解决这些问题，我们首先将文本描述引入RGBT跟踪基准。这是通过一个利用多模态大语言模型（MLLM）自动生成文本标注的流程实现的。随后，我们提出了RAGTrack，一种用于鲁棒RGBT跟踪的新型检索增强生成框架。为此，我们引入了多模态Transformer编码器（MTE）进行统一的视觉-语言建模。然后，我们设计了自适应词元融合（ATF），基于跨模态相关性选择目标相关词元并进行通道交换，从而减轻搜索冗余和模态间隙。最后，我们提出了上下文感知推理模块（CRM），用于维护动态知识库，并采用检索增强生成（RAG）实现时序语言推理，以进行鲁棒的目标建模。在四个RGBT基准上的大量实验表明，我们的框架在各种挑战性场景下均达到了最优性能。源代码可在 https://github.com/IdolLab/RAGTrack 获取。
英文原文摘要 1557 字符
RGB-Thermal (RGBT) tracking aims to achieve robust object localization across diverse environmental conditions by fusing visible and thermal infrared modalities. However, existing RGBT trackers rely solely on initial-frame visual information for target modeling, failing to adapt to appearance variations due to the absence of language guidance. Furthermore, current methods suffer from redundant search regions and heterogeneous modality gaps, causing background distraction. To address these issues, we first introduce textual descriptions into RGBT tracking benchmarks. This is accomplished through a pipeline that leverages Multi-modal Large Language Models (MLLMs) to automatically produce texual annotations. Afterwards, we propose RAGTrack, a novel Retrieval-Augmented Generation framework for robust RGBT tracking. To this end, we introduce a Multi-modal Transformer Encoder (MTE) for unified visual-language modeling. Then, we design an Adaptive Token Fusion (ATF) to select target-relevant tokens and perform channel exchanges based on cross-modal correlations, mitigating search redundancies and modality gaps. Finally, we propose a Context-aware Reasoning Module (CRM) to maintain a dynamic knowledge base and employ a Retrieval-Augmented Generation (RAG) to enable temporal linguistic reasoning for robust target modeling. Extensive experiments on four RGBT benchmarks demonstrate that our framework achieves state-of-the-art performance across various challenging scenarios. The source code is available at https://github.com/IdolLab/RAGTrack.