FAVE：面向多模态大模型细粒度音视频时序评测的结构化基准
FAVE: A Structured Benchmark for Fine-Grained Audio-Visual Temporal Evaluation in Multimodal LLMs
对做视频-语言或多模态大模型的人，FAVE 提供了三个可复用的评测维度（跨模态时序对齐、事件时序关系、时刻描述）和一套基于镜头边界检测+自动描述+GPT 精修的标注流水线，可直接用来诊断模型的时间戳定位与音视频联合推理短板。
原文依据 a comprehensive benchmark targeting three core dimensions of temporal perception: cross-modal temporal alignment (FAVE-align), event temporal relationship (FAVE-low), and detailed moment captioning (FAVE-high)
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论音视频大语言模型（AVLLM）在理解视觉与听觉内容方面已取得显著进展。然而，其捕捉音频流与视觉流之间细粒度时序关系的能力仍未得到充分评测。为此，我们提出 FAVE（细粒度音视频时序评测），这是一个面向时序感知三个核心维度的综合基准：跨模态时序对齐（FAVE-align）、事件时序关系（FAVE-low）以及细粒度时刻描述（FAVE-high）。为构建 FAVE，我们提出一种可扩展的标注流水线，整合镜头边界检测、自动描述生成与 GPT 辅助精修，以产出具有时序定位的高质量数据。在十二个最优多模态大语言模型（涵盖开源与闭源）上的大量实验揭示了多模态融合、时序关系与时间戳定位方面的关键局限，音视频联合任务尤为明显。这些发现凸显了改进时序建模以提升 AVLLM 对真实世界视频内容理解的必要性。FAVE 可作为推进时序感知多模态系统研究的严格测试平台，并将在论文录用后公开发布。
英文原文摘要 1254 字符
Audio-visual large language models (AVLLMs) have made significant strides in understanding visual and auditory content. However, their ability to capture fine-grained temporal relationships between audio and visual streams remains insufficiently evaluated. To address this, we introduce FAVE (Fine-grained Audio-Visual Temporal Evaluation), a comprehensive benchmark targeting three core dimensions of temporal perception: cross-modal temporal alignment (FAVE-align), event temporal relationship (FAVE-low), and detailed moment captioning (FAVE-high). To construct FAVE, we propose a scalable annotation pipeline that integrates shot boundary detection, automated captioning, and GPT-assisted refinement to produce temporally grounded, high-quality data. Extensive experiments on twelve state-of-the-art multimodal LLMs, both open-source and closed-source, reveal key limitations in multimodal integration, temporal relationship and timestamp localization, especially for joint audio-visual tasks. These findings highlight the need for better temporal modeling to improve AVLLMs' understanding of real-world video content. FAVE serves as a rigorous testbed for advancing temporally aware multimodal systems, and will be publicly released upon acceptance.