HERBench：视频问答中多证据整合的基准
HERBench: A Benchmark for Multi-Evidence Integration in Video Question Answering
HERBench 提供了强制多证据整合的视频问答基准，并引入最小所需帧集（MRFS）量化证据需求，可用于诊断视频大模型在检索与融合上的瓶颈，指导设计更鲁棒的多帧推理模型。
原文依据 We introduce HERBench, a benchmark designed to make multi-evidence integration unavoidable: each question requires at least three non-overlapping cues drawn from distinct video segments.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视频大语言模型（Video-LLMs）正在快速进步，然而当前的视频问答（VideoQA）基准往往允许单线索捷径，未能充分测试需要跨时间整合证据的推理能力。我们提出了 HERBench，一个旨在使多证据整合不可避免的基准：每个问题要求至少三个来自不同视频片段的不重叠线索。HERBench 包含 12 个组合任务中的 26,806 个五选一多项选择题。为了使证据需求可度量，我们引入了最小所需帧集（MRFS），即模型必须融合才能正确回答的最小帧数，并表明 HERBench 比先前基准施加了更高的证据需求。评估 13 个最优 Video-LLMs 仅得到 31-42% 的准确率，仅略高于 20% 的随机猜测基线。我们将这一失败分解为两个关键瓶颈：（1）检索缺陷，即帧选择器忽略了关键证据；（2）融合缺陷，即即使提供了所有必要证据，模型也无法整合信息。因此，HERBench 为研究鲁棒的多证据视频理解提供了一个有原则的基准。
英文原文摘要 1205 字符
Video Large Language Models (Video-LLMs) are improving rapidly, yet current Video Question Answering (VideoQA) benchmarks often admit single-cue shortcuts, under-testing reasoning that must integrate evidence across time. We introduce HERBench, a benchmark designed to make multi-evidence integration unavoidable: each question requires at least three non-overlapping cues drawn from distinct video segments. HERBench contains 26,806 five-way multiple-choice questions across 12 compositional tasks. To make evidential demand measurable, we introduce the Minimum Required Frame-Set (MRFS), the smallest number of frames a model must fuse to answer correctly, and show that HERBench imposes higher evidential demand than prior benchmarks. Evaluating 13 state-of-the-art Video-LLMs yields only 31-42% accuracy, only modestly above the 20% random-guess baseline. We disentangle this failure into two critical bottlenecks: (1) a retrieval deficit, where frame selectors overlook key evidence, and (2) a fusion deficit, where models fail to integrate information even when all necessary evidence is provided. HERBench thus provides a principled benchmark for studying robust multi-evidence video understanding.