X-AVDT：面向鲁棒深度伪造检测的音视频交叉注意力
X-AVDT: Audio-Visual Cross-Attention for Robust Deepfake Detection
把生成器内部的音视频交叉注意力当作取证线索：用 DDIM 反演取回生成过程中的模态对齐特征与反演差异图，作为检测器输入，这一思路可迁移到其他「利用生成模型内部表征做取证/溯源」的视觉任务。
原文依据 internal cross-attention mechanisms in these models encode fine-grained speech-motion alignment
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论当代生成系统产生的高度逼真合成视频激增，显著提高了恶意使用的风险，对人类与现有检测器都构成挑战。在此背景下，我们从生成器一侧切入观察，发现这些模型内部的交叉注意力机制编码了细粒度的语音—运动对齐，为伪造检测提供了有用的对应线索。基于这一观察，我们提出 X-AVDT，一种鲁棒且可泛化的深度伪造检测器，它通过 DDIM 反演获取生成器内部的音视频信号以揭示上述线索。X-AVDT 提取两种互补信号：(i) 捕捉反演所引发差异的视频合成图，(ii) 反映生成过程中所施加模态对齐的音视频交叉注意力特征。为支持忠实的跨生成器评测，我们进一步提出 MMDF，一个覆盖多样篡改类型与快速演进的合成范式（包括 GAN、扩散与流匹配）的新多模态深度伪造数据集。大量实验表明，X-AVDT 在 MMDF 上取得领先性能，并对外部基准与未见生成器表现出强泛化能力，准确率较现有方法提升 +13.1%。我们的发现凸显了利用内部音视频一致性线索以在未来生成器面前保持鲁棒性的重要性。代码与数据集即将公开。
英文原文摘要 1471 字符
The surge of highly realistic synthetic videos produced by contemporary generative systems has significantly increased the risk of malicious use, challenging both humans and existing detectors. Against this backdrop, we take a generator-side view and observe that internal cross-attention mechanisms in these models encode fine-grained speech-motion alignment, offering useful correspondence cues for forgery detection. Building on this insight, we propose X-AVDT, a robust and generalizable deepfake detector that probes generator-internal audio-visual signals accessed via DDIM inversion to expose these cues. X-AVDT extracts two complementary signals: (i) a video composite capturing inversion-induced discrepancies, and (ii) audio-visual cross-attention feature reflecting modality alignment enforced during generation. To enable faithful, cross-generator evaluation, we further introduce MMDF, a new multi-modal deepfake dataset spanning diverse manipulation types and rapidly evolving synthesis paradigms, including GANs, diffusion, and flow-matching. Extensive experiments demonstrate that X-AVDT achieves leading performance on MMDF and generalizes strongly to external benchmarks and unseen generators, outperforming existing methods with accuracy improved by +13.1%. Our findings highlight the importance of leveraging internal audio-visual consistency cues for robustness to future generators in deepfake detection. The code and dataset will soon be available.