可解释性与可视化 · CVPR2026 ·2026.06.03

回到特征:用视频反事实解释说明视频分类器

Back to the Feature: Explaining Video Classifiers with Video Counterfactual Explanations

Chao Wang、Chengan Che、Xinyue Chen、Sophia Tsoka、Luis C. Garcia-Peraza-Herrera

与你有关的部分

把图像反事实解释扩展到视频时,关键难点是时序一致与运动平滑;BTTF 用「首帧条件反演初始潜变量 + 两阶段邻域搜索 + 渐进增加去噪步数」的纯分类器引导优化,为视频模型的归因与调试提供了可复用的生成式解释框架。

原文依据 an optimization framework that generates video CFEs
可迁移到 视频动作识别反事实解释生成模型归因分析

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

反事实解释(CFE)是对模型输入所做的最小且语义上有意义的修改,用以改变模型预测。它们突出模型所依赖的决定性特征,为分类器提供对比式解释。最优的视觉反事实解释方法主要集中于解释图像分类器,视频模型这一领域则相对缺乏探索。要使视频 CFE 有用,它们必须具备物理合理性、时序一致性,并呈现平滑的运动轨迹。现有的基于图像的 CFE 方法是为解释图像分类器而设计的,缺乏生成时序一致、平滑且物理合理的视频 CFE 的能力。为此,我们提出 Back To The Feature(BTTF),一个生成视频 CFE 的优化框架。该方法引入两项新特性:1)一种优化方案,用于检索以输入视频首帧为条件的初始潜变量噪声;2)一种两阶段优化策略,使得能够在输入视频附近搜索反事实视频。两个优化过程均仅由目标分类器引导,从而确保解释的忠实性。为加速收敛,我们还引入一种渐进式优化策略,逐步增加去噪步数。在 Shape-Moving(运动分类)、MEAD(情绪分类)和 NTU RGB+D(动作分类)等视频数据集上的大量实验表明,我们的 BTTF 能有效生成有效、视觉相似且逼真的反事实视频,为分类器的决策机制提供具体洞见。

英文原文摘要 1709 字符

Counterfactual explanations (CFEs) are minimal and semantically meaningful modifications of the input of a model that alter the model predictions. They highlight the decisive features the model relies on, providing contrastive interpretations for classifiers. State-of-the-art visual counterfactual explanation methods have primarily focused on interpreting image classifiers, leaving the domain of video models relatively underexplored. For the video CFEs to be useful, they have to be physically plausible, temporally coherent, and exhibit smooth motion trajectories. Existing CFE image-based methods, designed to explain image classifiers, lack the capacity to generate temporally coherent, smooth and physically plausible video CFEs. To address this, we propose Back To The Feature (BTTF), an optimization framework that generates video CFEs. Our method introduces two novel features, 1) an optimization scheme to retrieve the initial latent noise conditioned by the first frame of the input video, 2) a two-stage optimization strategy to enable the search for counterfactual videos in the vicinity of the input video. Both optimization processes are guided solely by the target classifier, ensuring the explanation is faithful. To accelerate convergence, we also introduce a progressive optimization strategy that incrementally increases the number of denoising steps. Extensive experiments on video datasets such as Shape-Moving (motion classification), MEAD (emotion classification), and NTU RGB+D (action classification) show that our BTTF effectively generates valid, visually similar and realistic counterfactual videos that provide concrete insights into the classifier's decision-making mechanism.

标签

会议 CVPR2026
页码 9921-9931
Bibkey
来源 CVF Open Access