ANTS:通过测试时 MLLM 理解与推理实现 OOD 检测的自适应负文本空间塑造
ANTS: Adaptive Negative Textual Space Shaping for OOD Detection via Test-Time MLLM Understanding and Reasoning
把 MLLM 当作测试时的『负空间生成器』:缓存疑似 OOD 图像与视觉相似的 ID 类,用提示词让 MLLM 分别生成描述性负句和近邻负标签,再用自适应加权分数融合,可直接迁移到任何基于 CLIP 式文本空间的 OOD 检测或开放词表识别流程中,且无需训练。
原文依据 we cache images likely to be OOD samples from the historical test images and prompt the MLLM to describe these images
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论负标签(NL)的引入已被证明能有效增强分布外(OOD)检测。然而,现有方法往往缺乏对 OOD 图像的理解,难以构建准确的负空间。此外,缺少与 ID 标签语义相似的负标签,限制了其在近 OOD 检测中的能力。为解决这些问题,我们提出利用多模态大语言模型(MLLM)的理解与推理能力来塑造自适应负文本空间(ANTS)。具体而言,我们从历史测试图像中缓存可能是 OOD 样本的图像,并提示 MLLM 描述这些图像,生成富有表现力的负向句子,从而精确刻画 OOD 分布并增强远 OOD 检测。对于 OOD 样本与分布内(ID)子集相似的近 OOD 设定,我们缓存与历史测试图像视觉相似的 ID 类子集,然后利用 MLLM 推理生成针对该子集定制的视觉相似负标签,有效减少假阴性并提升近 OOD 检测。为平衡这两类负文本空间,我们设计了一种自适应加权分数,使方法能够处理不同的 OOD 任务设定(近 OOD 与远 OOD),从而在开放环境中具有高度适应性。在 ImageNet 基准上,我们的 ANTS 将 FPR95 显著降低 3.1%,确立了新的最先进水平。此外,我们的方法无需训练且为零样本,具有高度可扩展性。代码见 https://github.com/ZhuWenjie98/ANTS。
英文原文摘要 1645 字符
The introduction of negative labels (NLs) has proven effective in enhancing Out-of-Distribution (OOD) detection. However, existing methods often lack an understanding of OOD images, making it difficult to construct an accurate negative space. Furthermore, the absence of negative labels semantically similar to ID labels constrains their capability in near-OOD detection. To address these issues, we propose shaping an Adaptive Negative Textual Space (ANTS) by leveraging the understanding and reasoning capabilities of multimodal large language models (MLLMs). Specifically, we cache images likely to be OOD samples from the historical test images and prompt the MLLM to describe these images, generating expressive negative sentences that precisely characterize the OOD distribution and enhance far-OOD detection. For the near-OOD setting, where OOD samples resemble the in-distribution (ID) subset, we cache the subset of ID classes that are visually similar to historical test images and then leverage MLLM reasoning to generate visually similar negative labels tailored to this subset, effectively reducing false negatives and improving near-OOD detection. To balance these two types of negative textual spaces, we design an adaptive weighted score that enables the method to handle different OOD task settings (near-OOD and far-OOD), making it highly adaptable in open environments. On the ImageNet benchmark, our ANTS significantly reduces the FPR95 by 3.1%, establishing a new state-of-the-art. Furthermore, our method is training-free and zero-shot, enabling high scalability. Codes are available at https://github.com/ZhuWenjie98/ANTS.