视觉语言模型 · CVPR2026 ·2026.06.03

直击要点:基于事件链的免训练多模态摘要

Cut to the Chase: Training-free Multimodal Summarization via Chain-of-Events

Xiaoxing You、Qiang Huang、Lingyu Li、Xiaojun Chang、Jun Yu

与你有关的部分

把分层事件图当作免训练的结构化推理骨架,为视频-文本-图像融合提供显式的跨模态视觉定位与时序事件转移建模,可复用于视频问答、视频描述等需要事件级时序推理的视觉-语言任务。

原文依据 a training-free MMS framework that performs structured reasoning through a Chain-of-Events guided by a Hierarchical Event Graph (HEG)
可迁移到 视频多模态摘要视频描述生成视频问答时序事件理解

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

多模态摘要(MMS)旨在通过理解并整合视频、转录文本与图像中的信息,生成简洁的文本摘要。然而,现有方法仍面临三大挑战:(1)依赖领域特定的监督信号;(2)隐式融合导致跨模态视觉定位能力薄弱;(3)时序建模扁平化,缺乏事件转移。为解决这些问题,本文提出 CoE,一种免训练的多模态摘要框架,它在分层事件图(HEG)的引导下通过事件链进行结构化推理。HEG 将文本语义编码为显式的事件层级结构,为跨模态视觉定位与时序推理提供支撑。在该结构的引导下,CoE 定位关键视觉线索,建模事件演化与因果转移,并通过轻量级风格适配进行输出精炼以实现领域对齐。在八个多样化数据集上的大量实验表明,CoE 一致优于最优的视频思维链基线,平均提升 +3.04 ROUGE、+9.51 CIDEr 与 +1.88 BERTScore,凸显了其鲁棒性、可解释性与跨域泛化能力。代码已开源至 https://github.com/youxiaoxing/CoE。

英文原文摘要 1244 字符

Multimodal Summarization (MMS) aims to generate concise textual summaries by understanding and integrating information across videos, transcripts, and images. However, existing approaches still suffer from three main challenges: (1) reliance on domain-specific supervision, (2) implicit fusion with weak cross-modal grounding, and (3) flat temporal modeling without event transitions. To address these issues, we introduce **CoE**, a training-free MMS framework that performs structured reasoning through a **Chain-of-Events** guided by a Hierarchical Event Graph (HEG). The HEG encodes textual semantics into an explicit event hierarchy that scaffolds cross-modal grounding and temporal reasoning. Guided by this structure, **CoE** localizes key visual cues, models event evolution and causal transitions, and refines outputs via lightweight style adaptation for domain alignment. Extensive experiments on eight diverse datasets demonstrate that **CoE** consistently outperforms state-of-the-art video CoT baselines, achieving average gains of **+3.04 ROUGE**, **+9.51 CIDEr**, and **+1.88 BERTScore**, highlighting its robustness, interpretability, and cross-domain generalization. Our code is available at https://github.com/youxiaoxing/CoE.

标签

会议 CVPR2026
页码 26219-26229
Bibkey
来源 CVF Open Access