无需真实异常：多模态大语言模型赋能的零样本视频异常检测
No Need For Real Anomaly: MLLM Empowered Zero-Shot Video Anomaly Detection
利用多模态大语言模型和伪异常采样器实现零样本视频异常检测，为开放世界场景下的异常理解提供了无需真实异常数据的训练范式；基于反向注意力的词元压缩方法可复用于降低时空稀疏模式的计算开销。
原文依据 LAVIDA employs an Anomaly Exposure Sampler that transforms segmented objects into pseudo-anomalies to enhance model adaptability to unseen anomaly categories.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论由于视频异常数据发生频率低且时空稀疏，其收集与检测长期面临挑战。现有视频异常检测（VAD）方法在开放世界场景下表现不佳，关键原因包括数据集多样性有限以及对上下文相关异常语义理解不足。为解决这些问题，i）我们提出 LAVIDA，一个端到端的零样本视频异常检测框架。ii）LAVIDA 采用异常暴露采样器，将分割后的对象转化为伪异常，以增强模型对未见异常类别的适应性。它进一步集成多模态大语言模型（MLLM）以提升语义理解能力。此外，iii）我们设计了一种基于反向注意力的词元压缩方法，以处理异常模式的时空稀疏性并降低计算成本。训练过程仅使用伪异常数据，不涉及任何 VAD 数据。在四个基准 VAD 数据集上的评估表明，LAVIDA 在零样本设置下于帧级和像素级异常检测均达到 SOTA 性能。我们的代码见 https://github.com/VitaminCreed/LAVIDA。
英文原文摘要 1247 字符
The collection and detection of video anomaly data has long been a challenging problem due to its rare occurrence and spatio-temporal scarcity. Existing video anomaly detection (VAD) methods under perform in open-world scenarios. Key contributing factors include limited dataset diversity, and inadequate understanding of context-dependent anomalous semantics. To address these issues, i) we propose LAVIDA, an end-to-end zero-shot video anomaly detection framework. ii) LAVIDA employs an Anomaly Exposure Sampler that transforms segmented objects into pseudo-anomalies to enhance model adaptability to unseen anomaly categories. It further integrates a Multimodal Large Language Model (MLLM) to bolster semantic comprehension capabilities. Additionally, iii) we design a token compression approach based on reverse attention to handle the spatio-temporal scarcity of anomalous patterns and decrease computational cost. The training process is conducted solely on pseudo anomalies without any VAD data. Evaluations across four benchmark VAD datasets demonstrate that LAVIDA achieves SOTA performance in both frame-level and pixel-level anomaly detection under the zero-shot setting. Our code is available in https://github.com/VitaminCreed/LAVIDA.