注意你选择负文本的方式:在视觉语言模型的 OOD 检测中追求距离一致性
Mind the Way You Select Negative Texts: Pursuing the Distance Consistency in OOD Detection with VLMs
对做开放世界视觉识别的人,这篇给出了一个可直接复用的思路:用 CLIP 原生优化的跨模态距离(而非模态内距离)来挑选负文本并反演高置信 OOD 图像生成额外负文本嵌入,可作为即插即用的 OOD 打分与负样本构造模块。
原文依据 we propose InterNeg, a simple yet effective framework that systematically utilizes consistent inter-modal distance enhancement
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论分布外(OOD)检测旨在识别来自未知类别的样本,是机器学习模型在开放世界场景中部署的关键能力。近期研究表明,视觉语言模型(VLM)能够有效利用其多模态表征进行 OOD 检测。然而,现有方法在 OOD 检测中往往引入模态内距离,例如将负文本与 ID 标签比较,或将测试图像与图像代理比较。这种设计范式与 CLIP 类 VLM 所优化的跨模态距离之间存在内在的不一致性,可能导致性能次优。为解决这一局限,我们提出 InterNeg,一个简单而有效的框架,从文本和视觉两个视角系统性地利用一致的跨模态距离增强。在文本视角上,我们设计了一种用于选择负文本的跨模态准则。在视觉视角上,我们动态识别高置信度的 OOD 图像并将其反演到文本空间,在跨模态距离的引导下生成额外的负文本嵌入。在多个基准上的大量实验证明了我们方法的优越性。值得注意的是,与现有工作相比,InterNeg 取得了最优性能:在大规模 ImageNet 基准上 FPR95 降低 3.47%,在具有挑战性的近分布 OOD 基准上 AUROC 提升 5.50%。
英文原文摘要 1473 字符
Out-of-distribution (OOD) detection seeks to identify samples from unknown classes, a critical capability for deploying machine learning models in open-world scenarios. Recent research has demonstrated that Vision-Language Models (VLMs) can effectively leverage their multi-modal representations for OOD detection. However, current methods often incorporate intra-modal distance during OOD detection, such as comparing negative texts with ID labels or comparing test images with image proxies. This design paradigm creates an inherent inconsistency against the inter-modal distance that CLIP-like VLMs are optimized for, potentially leading to suboptimal performance. To address this limitation, we propose InterNeg, a simple yet effective framework that systematically utilizes consistent inter-modal distance enhancement from textual and visual perspectives. From the textual perspective, we devise an inter-modal criterion for selecting negative texts. From the visual perspective, we dynamically identify high-confidence OOD images and invert them into the textual space, generating extra negative text embeddings guided by inter-modal distance. Extensive experiments across multiple benchmarks demonstrate the superiority of our approach. Notably, our InterNeg achieves state-of-the-art performance compared to existing works, with a 3.47% reduction in FPR95 on the large-scale ImageNet benchmark and a 5.50% improvement in AUROC on the challenging Near-OOD benchmark.