GEM-TFL:通过EM引导分解与时序精炼弥合弱监督与全监督伪造定位之间的差距
GEM-TFL: Bridging Weak and Full Supervision for Forgery Localization through EM-Guided Decomposition and Temporal Refinement
论文提出的基于EM的弱监督时序定位框架和图提案精炼模块,可迁移到视频动作定位、视频异常检测等需要从视频级标签学习帧级预测的视觉任务中。
原文依据 Temporal Forgery Localization (TFL) aims to precisely identify manipulated segments within videos or audio streams
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论时序伪造定位(TFL)旨在精确识别视频或音频流中被篡改的片段,为多媒体取证与安全提供可解释的证据。现有大多数TFL方法以全监督方式依赖密集的帧级标签,而弱监督TFL(WS-TFL)仅从视频级二值标签学习,降低了标注成本。然而,当前的WS-TFL方法存在训练与推理目标不匹配、二值标签监督有限、不可微的top-k聚合导致梯度阻塞以及缺乏对提案间关系的显式建模等问题。为解决这些问题,我们提出GEM-TFL(基于图的EM驱动时序伪造定位),一种两阶段分类-回归框架,有效弥合了训练与推理之间的监督差距。在此基础上,(1)我们通过基于EM的优化过程将二值标签重新表述为多维潜变量属性,从而增强弱监督;(2)我们引入一种无需训练时的时序一致性精炼方法,重新对齐帧级预测以获得更平滑的时序动态;(3)我们设计了一个基于图的提案精炼模块,对提案间的时序-语义关系进行建模,以实现全局一致的置信度估计。在基准数据集上的大量实验表明,GEM-TFL实现了更准确、更鲁棒的时序伪造定位,显著缩小了与全监督方法之间的差距。
英文原文摘要 1550 字符
Temporal Forgery Localization (TFL) aims to precisely identify manipulated segments within videos or audio streams, providing interpretable evidence for multimedia forensics and security. While most existing TFL methods rely on dense frame-level labels in a fully supervised manner, Weakly Supervised TFL (WS-TFL) reduces labeling cost by learning only from binary video-level labels. However, current WS-TFL approaches suffer from mismatched training and inference objectives, limited supervision from binary labels, gradient blockage caused by non-differentiable top-k aggregation, and the absence of explicit modeling of inter-proposal relationships. To address these issues, we propose GEM-TFL (Graph-based EM-powered Temporal Forgery Localization), a two-phase classification-regression framework that effectively bridges the supervision gap between training and inference. Built upon this foundation, (1) we enhance weak supervision by reformulating binary labels into multi-dimensional latent attributes through an EM-based optimization process; (2) we introduce a training-free temporal consistency refinement that realigns frame-level predictions for smoother temporal dynamics; and (3) we design a graph-based proposal refinement module that models temporal-semantic relationships among proposals for globally consistent confidence estimation. Extensive experiments on benchmark datasets demonstrate that GEM-TFL achieves more accurate and robust temporal forgery localization, substantially narrowing the gap with fully supervised methods.