数据集与评测 · CVPR2026 ·2026.06.03

AVFakeBench:面向音视频大语言模型的综合音视频伪造检测基准

AVFakeBench: A Comprehensive Audio-Video Forgery Detection Benchmark for AV-LMMs

Shuhan Xia、Peipei Li、Xuannan Liu、Dongsen Zhang、Xinyu Guo、Zekun Li

与你有关的部分

为视觉研究者提供一个覆盖人类主体与一般场景、含七类伪造与四级标注的音视频伪造检测基准,并给出二分类判断、伪造类型分类、伪造细节选择与解释性推理的多任务评测协议,可直接用于检验 AV-LMM 在细粒度伪造感知与推理上的短板。

原文依据 we introduce AVFakeBench, the first comprehensive audio-video forgery detection benchmark
可迁移到 音视频伪造检测多模态大模型评测伪造类型分类解释性推理

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

音视频伪造的威胁正迅速演变,已从以人为中心的深度伪造扩展到复杂自然场景中更多样的篡改。然而,现有基准仍局限于基于深度伪造的伪造与单一粒度的标注,因而无法刻画真实世界伪造场景的多样性与复杂性。为解决这一问题,我们提出 AVFakeBench,这是首个覆盖人类主体与一般主体丰富伪造语义的综合音视频伪造检测基准。AVFakeBench 包含 12K 条精心构建的音视频问题,覆盖七种伪造类型与四个层级的标注。为确保伪造数据的高质量与多样性,我们提出一个多阶段混合伪造框架,将用于任务规划的专有模型与用于精确篡改的专家生成模型相结合。该基准建立了多任务评测框架,涵盖二分类判断、伪造类型分类、伪造细节选择与解释性推理。我们在 AVFakeBench 上评测了 11 个音视频大语言模型(AV-LMM)与 2 种主流检测方法,展示了 AV-LMM 作为新兴伪造检测器的潜力,同时揭示了其在细粒度感知与推理方面的显著不足。

英文原文摘要 1307 字符

The threat of Audio-Video (AV) forgery is rapidly evolving beyond human-centric deepfakes to include more diverse manipulations across complex natural scenes. However, existing benchmarks are still confined to DeepFake-based forgeries and single-granularity annotations, thus failing to capture the diversity and complexity of real-world forgery scenarios. To address this, we introduce AVFakeBench, the first comprehensive audio-video forgery detection benchmark that spans rich forgery semantics across both human subject and general subject. AVFakeBench comprises 12K carefully curated audio-video questions, covering seven forgery types and four levels of annotations. To ensure high-quality and diverse forgeries, we propose a multi-stage hybrid forgery framework that integrates proprietary models for task planning with expert generative models for precise manipulation. The benchmark establishes a multi-task evaluation framework covering binary judgment, forgery types classification, forgery detail selection, and explanatory reasoning. We evaluate 11 Audio-Video Large Language Models (AV-LMMs) and 2 prevalent detection methods on AVFakeBench, demonstrating the potential of AV-LMMs as emerging forgery detectors while revealing their notable weaknesses in fine-grained perception and reasoning.

标签

会议 CVPR2026
页码 35416-35426
Bibkey
来源 CVF Open Access