鲁棒性与安全 · CVPR2026 ·2026.06.03

像素不会说谎(但你的检测器可能会):自举多模态大语言模型作为评判器以实现可信的深度伪造检测与推理监督

Pixels Don't Lie (But Your Detector Might): Bootstrapping MLLM-as-a-Judge for Trustworthy Deepfake Detection and Reasoning Supervision

Kartik Kuckreja、Parul Gupta、Muhammad Haris Khan、Abhinav Dhall

与你有关的部分

提出以多模态大语言模型作为评判器(MLLM-as-a-Judge),通过自举生成-评估过程将人类反馈扩展为结构化推理监督,可用于评估和提升深度伪造检测模型推理的视觉证据忠实性,为可解释的伪造检测提供可扩展的监督信号。

原文依据 We propose DeepfakeJudge, a framework for scalable reasoning supervision and evaluation
可迁移到 深度伪造检测视觉推理评估多模态大语言模型评估

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

深度伪造检测模型通常生成自然语言解释,然而其推理往往缺乏视觉证据支撑,限制了可靠性。现有评估仅衡量分类准确率,却忽视了推理的忠实性。我们提出 DeepfakeJudge,一个用于可扩展推理监督与评估的框架,它集成了一个包含近期生成与编辑伪造的分布外基准、一个带有视觉推理标签的人工标注子集,以及一套评估模型,专门用于评估推理依据而无需显式的真值推理依据。该评判器通过自举的生成器-评估器过程进行优化,将人类反馈扩展为结构化推理监督,并支持逐点与成对评估。在所提出的元评估基准上,我们的推理自举模型达到了 96.2% 的准确率,优于参数量大 30 倍的基线模型。该推理评判器与人类评分高度相关,在人工标注的元评估子集上达到 98.9% 的成对一致率。这些结果确立了推理忠实性作为深度伪造检测的一个可量化维度,并展示了面向可解释深度伪造推理的可扩展监督。我们的用户研究表明,与其他模型和数据集相比,人类在 70% 的情况下更偏好我们框架生成的推理,在忠实性、有据性和有用性方面更优。我们的所有数据集、模型和代码库都将开源。

英文原文摘要 1551 字符

Deepfake detection models often generate natural-language explanations, yet their reasoning is frequently ungrounded in visual evidence, limiting reliability. Existing evaluations measure classification accuracy but overlook reasoning fidelity. We propose DeepfakeJudge, a framework for scalable reasoning supervision and evaluation, that integrates an out-of-distribution benchmark containing recent generative and editing forgeries, a human-annotated subset with visual reasoning labels, and a suite of evaluation models, that specialize in evaluating reasoning rationales without the need for explicit ground truth reasoning rationales. The Judge is optimized through a bootstrapped generator-evaluator process that scales human feedback into structured reasoning supervision and supports both pointwise and pairwise evaluation. On the proposed meta-evaluation benchmark, our reasoning-bootstrapped model achieves an accuracy of 96.2%, outperforming \texttt 30x larger baselines. The reasoning judge attains very high correlation with human ratings and 98.9% percent pairwise agreement on the human annotated meta-evaluation subset. These results establish reasoning fidelity as a quantifiable dimension of deepfake detection and demonstrate scalable supervision for interpretable deepfake reasoning. Our user study indicates that humans prefer reasonings generated by our framework 70% of the time, in faithfullness, groundedness and usefulness compared to other models and datasets. All of our datasets, models, and codebase will be open-sourced.

标签

会议 CVPR2026
页码 25871-25881
Bibkey
来源 CVF Open Access