面向噪声对应的鲁棒遥感图像-文本检索
Robust Remote Sensing Image-Text Retrieval with Noisy Correspondence
针对视觉-语言检索中图文对不完美匹配的现实问题，提出基于损失幅度的样本可靠性加权与自步学习调度，可迁移到其他跨模态检索或对比学习训练流程中以提升对噪声对应的鲁棒性。
原文依据 we reveal an important but untouched problem in RSITR, i.e., Noisy Correspondence (NC).
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论作为连接遥感视觉与语言理解的关键任务，遥感图像-文本检索（RSITR）近年来引起了广泛的研究兴趣。然而，几乎所有RSITR方法都隐含地假设图像-文本对是完全匹配的。在实践中，获取大量良好对齐的数据对往往成本高昂甚至不可行。尽管已有若干研究认识到噪声对的存在，但很少有工作探索如何赋予神经网络对此类噪声的鲁棒性。基于上述观察，我们揭示了RSITR中一个重要但尚未被触及的问题，即噪声对应（NC）。为克服这些挑战，我们提出了一种新颖的鲁棒遥感图像-文本检索（RRSITR）范式，该范式设计了一种自步学习策略来模拟人类认知学习模式，从而在存在噪声对应的多模态数据中由易到难地进行学习。具体而言，我们首先根据每个样本对的损失大小将所有训练样本对分为三类，即干净样本对、模糊样本对和噪声样本对。然后，我们分别根据损失值为每个训练对分配权重，以估计其可靠性。进一步，我们分别设计了一种新的自步函数来动态调节样本的训练顺序和权重，从而建立渐进学习过程。最后，针对噪声样本对，我们提出了一种增强的三元组损失，根据语义相似度动态调整软间隔，从而增强对噪声的鲁棒性。在三个流行基准数据集上的大量实验表明，所提出的RRSITR显著优于最优方法，尤其是在高噪声率下。
英文原文摘要 1820 字符
As a pivotal task that bridges remote visual and linguistic understanding, Remote Sensing Image-Text Retrieval (RSITR) has attracted considerable research interest in recent years. However, almost all RSITR methods implicitly assume that image-text pairs are matched perfectly. In practice, acquiring a large set of well-aligned data pairs is often prohibitively expensive or even infeasible. Although several studies have acknowledged the presence of noisy pairs, little work has explored how to endow neural networks with robustness against such noise. Based on the above observations, we reveal an important but untouched problem in RSITR, i.e., Noisy Correspondence (NC). To overcome these challenges, we propose a novel Robust Remote Sensing Image-Text Retrieval (RRSITR) paradigm that designs a self-paced learning strategy to mimic human cognitive learning patterns, thereby learning from easy to hard from multi-modal data with NC. Specifically, we first divide all training sample pairs into three categories based on the loss magnitude of each pair, i.e., clean sample pairs, ambiguous sample pairs, and noisy sample pairs. Then, we respectively estimate the reliability of each training pair by assigning a weight to each pair based on the values of the loss. Further, we respectively design a new self-paced function to dynamically regulate the training sequence and weights of the samples, thus establishing a progressive learning process. Finally, for noisy sample pairs, we present an enhanced triplet loss to dynamically adjust the soft margin based on semantic similarity, thereby enhancing the robustness against noise. Extensive experiments on three popular benchmark datasets demonstrate that the proposed RRSITR significantly outperforms the state-of-the-art methods, especially in high noise rates.