ADSeeker：面向工业异常检测与推理的知识驱动推理框架
ADSeeker: A Knowledge-Grounded Reasoning Framework for Industry Anomaly Detection and Reasoning
把 MLLM 与图像-文档知识库的检索增强生成（Q2K RAG）结合，用于零样本工业异常检测与推理；其分层稀疏提示词与类型级特征提取思路可迁移到其他需要细粒度缺陷/异常描述的视觉-语言任务。
原文依据 we introduce the Query Image-Knowledge Retrieval-Augmented Generation (Q2K RAG) framework
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论自动视觉检测在工业检测中具有重要意义。尽管多模态大语言模型（MLLM）展现出强大的语言理解能力，并有望用于该任务，但其性能仍显著低于人类专家。在此背景下，我们识别出两个关键挑战：（i）预训练阶段对异常检测（AD）知识的整合不足；（ii）缺乏技术精确且具备上下文感知能力的语言生成以支撑异常推理。为解决这些问题，我们提出 ADSeeker，一个旨在通过知识驱动推理提升检测性能的异常任务助手。ADSeeker 首先利用我们构建的精选视觉文档知识库 SEEK-MVTec&VisA（SEEK-M&V），以弥补现有资源仅依赖非结构化文本的局限。SEEK-M&V 包含语义丰富的描述与图像-文档对，能够实现更全面的异常理解。为有效检索并利用该知识，我们引入查询图像-知识检索增强生成（Q2K RAG）框架。为进一步提升零样本异常检测（ZSAD）性能，ADSeeker 采用分层稀疏提示词机制与类型级特征来高效提取异常模式。此外，为应对工业异常检测（IAD）数据有限的挑战，我们引入目前最大规模的异常检测数据集 Multi-type Anomaly（MulA），涵盖 26 个类别下的 72 种多尺度缺陷类型。大量实验表明，我们的即插即用框架 ADSeeker 在多个基准数据集上取得了最优的零样本性能。
英文原文摘要 1710 字符
Automatic vision inspection holds significant importance in industry inspection. While multimodal large language models (MLLMs) exhibit strong language understanding capabilities and hold promise for this task, their performance remains significantly inferior to that of human experts. In this context, we identify two key challenges: (i) insufficient integration of anomaly detection (AD) knowledge during pre-training and (ii) the lack of technically precise and context-aware language generation for anomaly reasoning. To address these issues, we propose ADSeeker, an anomaly task assistant designed to enhance inspection performance through knowledge-grounded reasoning. ADSeeker first leverages a curated visual document knowledge base, SEEK-MVTec&VisA (SEEK-M&V), which we construct to address the limitations of existing resources that rely solely on unstructured text. SEEK-M&V includes semantic-rich descriptions and image-document pairs, enabling more comprehensive anomaly understanding. To effectively retrieve and utilize this knowledge, we introduce the Query Image-Knowledge Retrieval-Augmented Generation (Q2K RAG) framework. To further enhance the performance in zero-shot anomaly detection (ZSAD), ADSeeker leverages the Hierarchical Sparse Prompt mechanism and type-level features to efficiently extract anomaly patterns. Furthermore, to tackle the challenge of limited industry anomaly detection (IAD) data, we introduce the largest-scale AD dataset, Multi-type Anomaly (MulA), encompassing 72 multi-scale defect types across 26 categories. Extensive experiments show that our plug-and-play framework, ADSeeker, achieves state-of-the-art zero-shot performance on several benchmark datasets.