DLVP-CLIP：通过动态局部视觉提示增强细粒度零样本异常检测
DLVP-CLIP: Enhancing Fine-Grained Zero-Shot Anomaly Detection via Dynamic Local Visual Prompting
用语义感知的局部特征选择器从图像关键区域动态抽取视觉提示词 token，替代预定义文本提示词模板来补足 CLIP 的局部细节感知，这一动态视觉提示思路可直接迁移到其他需要细粒度局部判别的 CLIP 下游任务（如缺陷检测、病灶定位）。
原文依据 DLVP dynamically identifies and extracts local visual features from key regions in images as prompt tokens
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论零样本异常检测（ZSAD）旨在利用辅助数据训练模型，实现对未见类别的泛化学习，在工业质检、医学诊断等领域具有重要应用价值。尽管基于 CLIP 的方法展现出潜力，但其预训练目标聚焦于图像与文本之间的整体语义对齐，导致模型对局部细节不敏感，这与异常检测对细粒度局部特征的需求存在内在矛盾。现有改进方法依赖预定义的文本提示词框架来感知局部信息，但难以有效解决局部感知不足的问题。为此，本文提出一种基于 CLIP 的动态局部视觉提示方法（DLVP-CLIP）。DLVP 利用语义感知局部特征选择器（SLFS）模块，从图像关键区域动态识别并提取局部视觉特征作为提示词 token，并借助多模态局部提示（MLoP）模块在视觉与文本空间中联合优化表示，实现更精确的跨模态对齐。此外，引入高低频分解模块（HFD），通过小波变换分离并处理全局结构信息与局部纹理信息，从而增强细节感知。在 13 个异常检测数据集上的大量实验表明，DLVP-CLIP 在工业与医学领域数据集上均取得了出色的零样本异常检测性能。
英文原文摘要 1543 字符
Zero-shot anomaly detection (ZSAD) aims to utilize auxiliary data to train models for generalized learning of unseen categories, which has important application value in fields such as industrial quality inspection and medical diagnosis. Although methods based on CLIP show potential, their pre-training objective of focusing on overall semantic alignment between images and text makes the model insensitive to local details, which is inherently contradictory to the need for fine-grained local features in anomaly detection. Existing improvement methods rely on predefined text prompt frameworks to perceive local information, but struggle to effectively address the issue of insufficient local perception. To address this, this paper proposes a dynamic local visual prompting method based on CLIP (DLVP-CLIP). DLVP dynamically identifies and extracts local visual features from key regions in images as prompt tokens using the Semantic-Aware Local Feature Selector (SLFS) module, and utilizes the multi-modal local prompt (MLoP) module to jointly optimize representations in both visual and textual spaces, achieving more precise cross-modal alignment. Additionally, the high-low frequency decomposition module (HFD) is introduced to separate and process global structural and local textural information via wavelet transformation, thereby enhancing detail perception. Extensive experiments on 13 anomaly detection datasets demonstrate that DLVP-CLIP achieves outstanding ZSAD performance on datasets from the industrial and medical domains.