SynCLIP：面向鲁棒开放词表密集感知的同义词连贯语言-图像预训练
SynCLIP: Synonym-Coherent Language-Image Pretraining for Robust Open-Vocabulary Dense Perception
针对CLIP类开放词表密集感知中同义词导致的视觉定位不一致问题，提出语义一致空间注意力对齐模块和空间注意力细化模块，可复用于任何需要文本-图像空间对齐的视觉语言任务。
原文依据 synonym-induced grounding inconsistency, where semantically equivalent expressions yield disparate spatial attention patterns
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论开放词表密集感知（OVDP）旨在利用文本知识定位训练中未见过的物体。尽管近期基于CLIP的方法取得了显著进展，但我们发现了一个关键局限：同义词诱导的视觉定位不一致性，即语义等价的表达会产生不同的空间注意力模式。这种不一致性削弱了现有方法在真实世界OVDP应用中的鲁棒性和性能。为解决该问题，我们提出了SynCLIP，一个同义词连贯的语言-图像预训练框架，旨在增强OVDP的同义词鲁棒视觉定位。SynCLIP引入语义一致空间注意力对齐（SSA）模块，通过最小化原始表达与同义表达注意力图之间的差异来增强空间注意力一致性。此外，空间注意力细化（SAR）模块在对齐后的图中选择性增强语义最相关的空间区域，以实现更精确和稳定的视觉定位。为支持同义词连贯预训练，我们还构建了同义词增强视觉语料库（SEViC），为每个类别增加多个同义词和文本定义。在多个基准上的大量实验表明，SynCLIP在多样语言变体下显著提升了视觉定位一致性，并在基于CLIP的OVDP方法中达到了最优性能。代码见 https://github.com/Justlovesmile/SynCLIP。
英文原文摘要 1473 字符
Open-vocabulary dense perception (OVDP) aims to localize objects unseen during training by leveraging textual knowledge. Despite the remarkable progress of recent CLIP-based approaches, we identify a critical limitation: synonym-induced grounding inconsistency, where semantically equivalent expressions yield disparate spatial attention patterns. This inconsistency undermines the robustness and performance of existing methods in real-world OVDP applications. To address this issue, we propose SynCLIP, a Synonym-Coherent Language-Image Pretraining framework that enhances synonym-robust grounding for OVDP. SynCLIP introduces a Semantic-consistent Spatial Attention alignment (SSA) module to enhance spatial attention consistency by minimizing discrepancies between attention maps of original and synonymous expressions. Furthermore, a Spatial Attention Refinement (SAR) module selectively strengthens the most semantically relevant spatial regions within aligned maps for more precise and stable grounding. To support synonym-coherent pretraining, we also construct a Synonym-Enriched Visual Corpus (SEViC), which augments each category with multiple synonyms and textual definitions. Extensive experiments on multiple benchmarks demonstrate that SynCLIP substantially improves grounding consistency under diverse linguistic variants and achieves state-of-the-art performance among CLIP-based OVDP methods. Code is available at https://github.com/Justlovesmile/SynCLIP.