释放视觉语言语义用于深度伪造视频检测
Unleashing Vision-Language Semantics for Deepfake Video Detection
利用预训练VLM的跨模态语义作为深度伪造检测的判别线索，并通过身份先验文本提示增强语义特异性，为视频伪造检测提供可复用的语义增强思路。
原文依据 We propose VLAForge, a novel DFD framework that unleashes the potential of such cross-modal semantics to enhance model's discriminability in deepfake detection.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近期深度伪造视频检测（DFD）研究表明，预训练的视觉语言模型（如CLIP）在检测不同身份间的伪造痕迹方面展现出强大的泛化能力。然而，现有方法仅关注利用视觉特征，忽视了其最显著的优势——潜空间中嵌入的丰富视觉语言语义。我们提出VLAForge，一种新颖的DFD框架，释放此类跨模态语义的潜力以增强模型在深度伪造检测中的判别力。该工作：i）通过ForgePerceiver增强VLM的视觉感知，该模块作为独立学习器，以细粒度和整体方式捕捉多样、细微的伪造线索，同时保留预训练的视觉语言对齐（VLA）知识；ii）提供一种互补的判别线索——身份感知VLA分数，通过将跨模态语义与ForgePerceiver学习到的伪造线索耦合而得出。值得注意的是，该VLA分数通过身份先验文本提示进行增强，以捕捉针对每个身份定制的真实性线索，从而实现更具判别力的跨模态语义。在视频DFD基准（包括经典换脸伪造和近期全脸生成伪造）上的综合实验表明，我们的VLAForge在帧级和视频级均显著优于最优方法。代码见https://github.com/mala-lab/VLAForge。
英文原文摘要 1504 字符
Recent Deepfake Video Detection (DFD) studies have demonstrated that pre-trained Vision-Language Models (VLMs) such as CLIP exhibit strong generalization capabilities in detecting artifacts across different identities. However, existing approaches focus on leveraging visual features only, overlooking their most distinctive strength -- the rich vision-language semantics embedded in the latent space. We propose VLAForge, a novel DFD framework that unleashes the potential of such cross-modal semantics to enhance model's discriminability in deepfake detection. This work i) enhances the visual perception of VLM through a ForgePerceiver, which acts as an independent learner to capture diverse, subtle forgery cues both granularly and holistically, while preserving the pretrained Vision-Language Alignment (VLA) knowledge, and ii) provides a complementary discriminative cue -- Identity-Aware VLA score, derived by coupling cross-modal semantics with the forgery cues learned by ForgePerceiver. Notably, the VLA score is augmented by an identity prior-informed text prompting to capture authenticity cues tailored to each identity, thereby enabling more discriminative cross-modal semantics. Comprehensive experiments on video DFD benchmarks, including classical face-swapping forgeries and recent full-face generation forgeries, demonstrate that our VLAForge substantially outperforms state-of-the-art methods at both frame and video levels. Code is available at https://github.com/mala-lab/VLAForge.