CURVE：面向文化与多语言长视频推理的基准
CURVE: A Benchmark for Cultural and Multilingual Long Video Reasoning
提供了一个以原生语言、区域文化视频构建的长视频推理基准，并给出基于推理链构建证据图来定位细粒度推理错误的迭代评测策略，可直接用于诊断 Video-LLM 在视觉文化感知环节的失效点。
原文依据 we leverage CURVE's reasoning traces to construct evidence-based graphs and propose a novel iterative strategy using these graphs to identify fine-grained errors in reasoning
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近期视频模型的进展已取得巨大突破，尤其是在长视频理解方面。然而，当前基准大多以西方为中心的数据为主，并以英语为dominant语言，从而在评测中引入了显著偏差。为解决这一问题，我们提出 CURVE，一个具有挑战性的多文化与多语言视频推理基准。CURVE 包含来自 18 个全球地区的、具有区域文化特色的视频，其高质量标注完全由人工生成。与依赖自动翻译的先前工作不同，CURVE 提供复杂问题、答案以及多步推理步骤，且全部以原生语言撰写。要在 CURVE 上取得进展，需要对视觉文化语境有深度情境化的理解。此外，我们利用 CURVE 的推理轨迹构建基于证据的图，并提出一种利用这些图来识别推理中细粒度错误的新颖迭代策略。我们的评测表明，当前最先进的视频大语言模型表现显著不佳，准确率大幅低于人类水平，其错误主要源于对文化元素的视觉感知。我们将发布 CURVE，以促进更公平、能力更强的多模态基础模型的发展。
英文原文摘要 1273 字符
Recent advancements in video models have shown tremendous progress, particularly in long video understanding. However, current benchmarks predominantly feature western-centric data and English as the dominant language, introducing significant biases in evaluation. To address this, we introduce CURVE, a challenging benchmark for multicultural and multilingual video reasoning. CURVE comprises high-quality, entirely human-generated annotations from diverse, region-specific cultural videos across 18 global locales. Unlike prior work that relies on automatic translations, CURVE provides complex questions, answers, and multi-step reasoning steps, all crafted in native languages. Making progress on CURVE requires a deeply situated understanding of visual cultural context. Furthermore, we leverage CURVE's reasoning traces to construct evidence-based graphs and propose a novel iterative strategy using these graphs to identify fine-grained errors in reasoning. Our evaluations reveal that SoTA Video-LLMs struggle significantly, performing substantially below human-level accuracy, with errors primarily stemming from the visual perception of cultural elements. We will release CURVE to foster the development of more equitable and capable multimodal foundation models.