TTL:面向预训练视觉语言模型 OOD 检测的测试时文本学习
TTL: Test-time Textual Learning for OOD Detection with Pretrained Vision-Language Models
把 CLIP 的可学习 prompt 在无标签测试流上做在线更新,并用伪标签筛选与文本知识库做跨批次分数校准,可直接迁移到任何需要测试时分布漂移检测的视觉流水线(如开放世界分类、异常检测)。
原文依据 TTL updates learnable prompts using pseudo-labeled test samples to capture emerging OOD knowledge.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论CLIP 等视觉语言模型(VLM)通过对齐视觉与文本表示,展现出强大的分布外(OOD)检测能力。近期基于 CLIP 的测试时自适应方法通过引入外部 OOD 标签进一步提升了检测性能。然而,这类标签是有限且固定的,而真实的 OOD 语义空间本质上是开放式的。因此,固定标签无法表示测试流中遇到的多样且不断演化的 OOD 语义。为解决这一局限,我们提出测试时文本学习(TTL)框架,它无需依赖外部 OOD 标签,即可从无标签测试流中动态学习 OOD 文本语义。TTL 利用伪标签测试样本更新可学习提示,以捕获新出现的 OOD 知识。为抑制伪标签引入的噪声,我们提出一种 OOD 知识净化策略,在自适应过程中筛选可靠的 OOD 样本并抑制噪声。此外,TTL 维护一个 OOD 文本知识库,用于存储高质量文本特征,从而在跨批次时提供稳定的分数校准。在两个标准基准、九个 OOD 数据集上的大量实验表明,TTL 持续取得最优性能,凸显了文本自适应对鲁棒测试时 OOD 检测的价值。代码已开源:https://github.com/figec/TTL。
英文原文摘要 1397 字符
Vision-language models (VLMs) such as CLIP exhibit strong Out-of-distribution (OOD) detection capabilities by aligning visual and textual representations. Recent CLIP-based test-time adaptation methods further improve detection performance by incorporating external OOD labels. However, such labels are finite and fixed, while the real OOD semantic space is inherently open-ended. Consequently, fixed labels fail to represent the diverse and evolving OOD semantics encountered in test streams.To address this limitation, we introduce Test-time Textual Learning (TTL), a framework that dynamically learns OOD textual semantics from unlabeled test streams, without relying on external OOD labels.TTL updates learnable prompts using pseudo-labeled test samples to capture emerging OOD knowledge. To suppress noise introduced by pseudo-labels, we introduce an OOD knowledge purification strategy that selects reliable OOD samples for adaptation while suppressing noise. In addition, TTL maintains an OOD Textual Knowledge Bank that stores high-quality textual features, providing stable score calibration across batches. Extensive experiments on two standard benchmarks with nine OOD datasets demonstrate that TTL consistently achieves state-of-the-art performance, highlighting the value of textual adaptation for robust test-time OOD detection. Our code is available at https://github.com/figec/TTL.