ELV-Halluc：面向视频理解中语义聚合幻觉的基准评测
ELV-Halluc: Benchmarking Semantic Aggregation Hallucinations in Video Understanding
把视频幻觉从「帧级感知错误」重新拆解为「帧级语义聚合为事件级语义时的错误」，并给出可复用的细粒度评测协议与 DPO 缓解方案；做视频 MLLM 时序推理或幻觉评测的工作可直接沿用其 SAH 定义与 8K 对抗视频-文本对数据。
原文依据 We term this phenomenon Semantic Aggregation Hallucination (SAH)
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论本文从语义聚合的视角重新审视多模态大语言模型（Video-MLLM）中的视频幻觉问题。已有工作将幻觉归因于语言先验、帧缺失或视觉编码器偏差，但这些解释忽略了在将正确的帧级语义聚合为事件级解释过程中所产生的错误。我们将这一现象称为语义聚合幻觉（Semantic Aggregation Hallucination, SAH），它在具有丰富时序依赖的复杂多事件视频理解任务中愈发普遍。为系统研究 SAH，我们提出 ELV-Halluc，这是首个用于细粒度评估语义聚合错误的基准。实验表明，SAH 与语义复杂度以及快速的语义转换均相关。我们进一步提出缓解策略：改进的位置编码可保留时序结构，而 DPO 等强化学习方法可增强模型区分事件内部与跨事件语义的能力。借助精心构建的 8K 对抗视频-文本对数据集，我们的方法在多个基准上取得一致提升，其中在 ELV-Halluc 与 Video-MME 上的 SAH 率降低 27.7%。数据与代码见 https://github.com/hlsv02/ELV-Halluc。
英文原文摘要 1265 字符
We revisit video hallucination in multimodal large language models (Video-MLLMs) from a semantic aggregation perspective. While prior work attributes hallucinations to language priors, missing frames, or visual encoder biases, these explanations overlook errors arising during the aggregation of correct frame-level semantics into event-level interpretations. We term this phenomenon Semantic Aggregation Hallucination (SAH), which becomes increasingly prevalent in complex, multi-event video understanding tasks with rich temporal dependencies. To systematically study SAH, we introduce ELV-Halluc, the first benchmark designed for fine-grained evaluation of semantic aggregation errors. Our experiments reveal that SAH correlates with both semantic complexity and rapid semantic transitions. We further propose mitigation strategies: improved positional encoding preserves temporal structure, and reinforcement learning such as DPO enhances the model's ability to distinguish semantics within and across events. Using a curated 8K adversarial video-text pair dataset, our approach achieves consistent gains across benchmarks, including a 27.7% reduction in SAH rate on ELV-Halluc and Video-MME. Data and code are available at https://github.com/hlsv02/ELV-Halluc.