WikiCLIP：面向开放域视觉实体识别的高效对比基线
WikiCLIP: An Efficient Contrastive Baseline for Open-domain Visual Entity Recognition
提出用大语言模型嵌入作为实体表示，并通过视觉引导的知识适配器在图像块级别对齐文本语义与视觉线索，为视觉-语言检索提供了一种高效且可复用的对齐范式。
原文依据 WikiCLIP leverages large language model embeddings as knowledge-rich entity representations and enhances them with a Vision-Guided Knowledge Adaptor (VGKA) that aligns textual semantics with visual cues at the patch level.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论开放域视觉实体识别（VER）旨在将图像与维基百科等百科知识库中的实体关联起来。近期针对VER的生成式方法表现出强劲性能，但计算成本高昂，限制了其可扩展性和实际部署。在本工作中，我们重新审视用于VER的对比范式，并引入WikiCLIP，一个简单而有效的框架，为开放域VER建立了一个强大且高效的基线。WikiCLIP利用大语言模型嵌入作为知识丰富的实体表示，并通过视觉引导的知识适配器（VGKA）在图像块级别对齐文本语义与视觉线索，从而增强这些表示。为了进一步促进细粒度判别，一种难负样本合成机制在训练过程中生成视觉相似但语义不同的负样本。在OVEN等流行的开放域VER基准上的实验结果表明，WikiCLIP显著优于强基线。具体而言，WikiCLIP在具有挑战性的OVEN未见集上实现了16%的提升，同时与领先的生成式模型AutoVER相比，推理延迟降低了近100倍。项目页面见https://artanic30.github.io/project_pages/WikiCLIP/。
英文原文摘要 1293 字符
Open-domain visual entity recognition (VER) seeks to associate images with entities in encyclopedic knowledge bases such as Wikipedia. Recent generative methods tailored for VER demonstrate strong performance but incur high computational costs, limiting their scalability and practical deployment. In this work, we revisit the contrastive paradigm for VER and introduce WikiCLIP, a simple yet effective framework that establishes a strong and efficient baseline for open-domain VER. WikiCLIP leverages large language model embeddings as knowledge-rich entity representations and enhances them with a Vision-Guided Knowledge Adaptor (VGKA) that aligns textual semantics with visual cues at the patch level. To further encourage fine-grained discrimination, a Hard Negative Synthesis Mechanism generates visually similar but semantically distinct negatives during training. Experimental results on popular open-domain VER benchmarks, such as OVEN, demonstrate that WikiCLIP significantly outperforms strong baselines. Specifically, WikiCLIP achieves a 16% improvement on the challenging OVEN unseen set, while reducing inference latency by nearly 100 times compared with the leading generative model, AutoVER. The project page is available at https://artanic30.github.io/project_pages/WikiCLIP/.