VLM4RSDet：协同优化与视觉语言模型增强遥感目标检测
VLM4RSDet: Collaborative Optimization with Vision-Language Model for Enhancing Remote Sensing Object Detection
提出一种协同训练框架，利用视觉语言模型的先验知识增强闭集遥感目标检测器，推理时仅保留标准检测架构，避免额外开销；其全局-局部交叉注意力模块和可学习分层预测策略可迁移到其他检测任务中。
原文依据 we propose VLM4RSDet, a novel collaborative training framework that leverages vision-language model to enhance the performance of conventional closed-set remote sensing object detectors.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论遥感图像中的闭集目标检测已取得显著进展，但要实现高检测精度仍具挑战性。具备丰富先验知识的视觉语言模型为解决这一难题提供了有前景的方案。然而，现有大多数视觉语言模型面向开放词表任务设计，直接应用于闭集场景时存在固有局限，如精度明显下降和部署成本高。为解决这些问题，我们提出VLM4RSDet，一种新颖的协同训练框架，利用视觉语言模型增强传统闭集遥感目标检测器的性能。值得注意的是，在推理阶段，VLM4RSDet仅保留标准目标检测架构，从而避免任何额外的部署开销。此外，我们引入全局-局部交叉注意力模块和可学习分层预测策略，以进一步提升协同训练性能。在五个基准数据集上的大量实验证明了我们方法的有效性和鲁棒性。特别地，我们的方法在VisDrone2019数据集上以7.5%的mAP_0.5:0.95优于最优方法。我们的代码可在https://github.com/cszzshi/VLM4RSDet获取。
英文原文摘要 1336 字符
Closed-set object detection in remote sensing imagery has made significant progress, but achieving high detection accuracy remains challenging. Vision-Language Models (VLMs), which possess rich prior knowledge, offer a promising solution to this challenge. However, most existing VLMs are designed for open-vocabulary tasks and exhibit inherent limitations when directly applied to closed-set scenarios, such as notable accuracy degradation and high deployment costs. To address these issues, we propose VLM4RSDet, a novel collaborative training framework that leverages vision-language model to enhance the performance of conventional closed-set remote sensing object detectors. Notably, during inference, VLM4RSDet only retains the standard object detection architecture, thus avoiding any additional deployment overhead. Furthermore, we introduce a Global-Local Cross-Attention (GLCA) module and a Learnable Hierarchical Prediction Strategy (LHPS) to further improve collaborative training performance. Extensive experiments on five benchmark datasets demonstrate the effectiveness and robustness of our approach. In particular, our method outperforms the state-of-the-art by 7.5% in mAP_ 0.5:0.95 on the VisDrone2019 dataset. Our code is available at \href https://github.com/cszzshi/VLM4RSDet https://github.com/cszzshi/VLM4RSDet .