重新审视跨模态锚框对齐以缓解误差累积
Rethinking Cross-Modal Anchor Alignment for Mitigating Error Accumulation
针对跨模态匹配中干净锚框对也会累积误差的问题，用傅里叶变换压制非关键细粒度扰动、以角度一致性重估软标签，可直接迁移到图文检索与跨模态对齐训练中抑制噪声累积。
原文依据 a novel source of error from clean sample pairs (also termed anchor pairs) is discovered in this paper
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论由于误差累积问题，缓解跨模态匹配中的噪声对应是一项严峻挑战。现有方法主要将这种累积归因于噪声样本对引起的误差。然而，本文发现了来自干净样本对（也称为锚框对）的一种新的误差来源。这种误差累积被认为源于模态不一致的相关性。为解决该问题，提出了一种名为几何-语义学习（GSL）的新方法。首先，GSL 利用傅里叶变换强调语义表示，并减少由非关键细粒度特征中的扰动所引起的跨模态不一致，从而缓解误差累积问题。随后，引入几何感知标签校正（GALC）方法，通过利用不同模态下噪声样本对与锚框对之间的角度一致性来重新估计软对应标签。最后，采用语义约束三元组损失，利用语义信息调控样本距离，从而在训练过程中实现干净对与噪声对的鲁棒分离。在三个基准数据集上的大量实验表明，GSL 在检索准确率上持续优于现有方法。
英文原文摘要 1306 字符
Mitigating noisy correspondence in cross-modal matching poses a serious challenge due to the problem of error accumulation. Existing methods primarily attribute this accumulation to errors caused by noisy sample pairs. However, a novel source of error from clean sample pairs (also termed anchor pairs) is discovered in this paper. Such error accumulation is considered to arise from modality-inconsistent correlations. To address this issue, a novel method termed Geometric-Semantic Learning (GSL) is proposed. Firstly, GSL leverages the Fourier transform to emphasize semantic representations and reduce cross-modal inconsistencies caused by perturbations in non-critical fine-grained features, thereby alleviating the error accumulation problem. After that, a Geometry-Aware Label Correction (GALC) method is introduced to re-estimate soft correspondence labels by leveraging angular consistency between noisy sample pairs and anchor pairs across different modalities. Finally, a semantically constrained triplet loss is employed to regulate sample distances using semantic information, enabling robust separation of clean and noisy pairs during the training process. Extensive experiments on three benchmark datasets demonstrate that GSL consistently outperforms existing methods in retrieval accuracy.