渐进式跨模态因果干预用于长时动作识别
Progressive Cross-Modal Causal Intervention for Long-Term Action Recognition
将因果推断中的前门调整和反事实干预引入视觉语言模型，用于去除长时动作识别中的视觉混淆和动作共现幻觉，为视频理解中的跨模态对齐提供可复用的去偏框架。
原文依据 PCMCI first mitigates co-occurrence hallucination via causal intervention grounded in optimal transport theory.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论长时动作识别（LTAR）中原子动作之间的复杂关联以及固有的视觉混淆因素，导致该领域持续面临挑战。基于视觉语言模型并利用标签文本进行监督的方法为处理视觉混淆因素提供了潜力，但它们依赖统计相关性而非因果机制，这带来了两个弱点：（1）在跨模态交互过程中与非因果共现视觉特征产生虚假对齐；（2）对动作之间相互依赖关系的误解。为解决这些局限，本文提出渐进式跨模态因果干预（PCMCI）。PCMCI首先通过基于最优传输理论的因果干预来缓解共现幻觉。随后，一种动作关系感知机制对抗由相互依赖错觉引起的后门路径，从而推导出去混淆的文本嵌入。最后，这些去混淆嵌入作为中介变量实施前门调整以去除视觉混淆因素。这一渐进式因果干预框架有助于为LTAR学习鲁棒的表示。在三个长时动作基准上的实验证明了所提模型的有效性。
英文原文摘要 1278 字符
Intricate correlations among atomic actions and inherent visual confounders in long-term action recognition (LTAR) contribute to the persistent challenges in this domain. While methods based on vision-language models that employ label text for supervision offer potential for handling visual confounders, their reliance on statistical correlations rather than causal mechanisms introduces two vulnerabilities: (1) spurious alignments with non-causal co-occurring visual features during cross-modal interaction, and (2) misinterpretation of codependencies among actions. To address these limitations, this paper introduces Progressive Cross-Modal Causal Intervention (PCMCI). PCMCI first mitigates co-occurrence hallucination via causal intervention grounded in optimal transport theory. Subsequently, an action relation-aware mechanism counters the backdoor path induced by codependency illusion, enabling the derivation of deconfounded text embeddings. Finally, these deconfounded embeddings serve as mediator to implement front-door adjustment to remove visual confounders. This progressive causal intervention framework facilitates learning robust representations for LTAR. Experiments on three long-term action benchmarks demonstrate the effectiveness of the proposed model.