VABench：面向音视频生成的综合基准
VABench: A Comprehensive Benchmark for Audio-Video Generation
为做视频生成的研究者提供了一套可直接复用的音视频同步评测协议：15 个维度覆盖文本-视频、文本-音频、视频-音频两两相似度、唇音一致性以及音视频 QA，可用于诊断生成模型在同步物理音效、复杂场景等类别上的失效模式。
原文依据 VABench encompasses three primary task types: text-to-audio-video (T2AV), image-to-audio-video (I2AV), and stereo audio-video generation.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近期视频生成领域进展显著，模型已能生成画面逼真且音频同步的视频。然而，现有视频生成基准虽为视觉质量提供了较全面的指标，却缺乏对音视频生成的可信评测，尤其是针对以生成同步音视频输出为目标的模型。为填补这一空白，我们提出 VABench，一个综合、多维度的基准框架，旨在系统评估同步音视频生成的能力。VABench 涵盖三类主要任务：文本到音视频（T2AV）、图像到音视频（I2AV）以及立体声音视频生成。它进一步建立了两大评测模块，覆盖 15 个维度。这些维度具体评估成对相似性（文本-视频、文本-音频、视频-音频）、音视频同步性、唇音一致性，以及精心构建的音视频问答（QA）对等。此外，VABench 覆盖七大内容类别：动物、人声、音乐、环境声、同步物理音效、复杂场景与虚拟世界。我们对评测结果进行了系统分析与可视化，旨在为评估具备同步音频能力的视频生成模型建立新标准，并推动该领域的全面发展。
英文原文摘要 1429 字符
Recent advances in video generation have been remarkable, enabling models to produce visually compelling videos with synchronized audio. While existing video generation benchmarks provide comprehensive metrics for visual quality, they lack convincing evaluations for audio-video generation, especially for models aiming to generate synchronized audio-video outputs. To address this gap, we introduce VABench, a comprehensive and multi-dimensional benchmark framework designed to systematically evaluate the capabilities of synchronous audio-video generation. VABench encompasses three primary task types: text-to-audio-video (T2AV), image-to-audio-video (I2AV), and stereo audio-video generation. It further establishes two major evaluation modules covering 15 dimensions. These dimensions specifically assess pairwise similarities (text-video, text-audio, video-audio), audio-video synchronization, lip-speech consistency, and carefully curated audio and video question-answering (QA) pairs, among others. Furthermore, VABench covers seven major content categories: animals, human sounds, music, environmental sounds, synchronous physical sounds, complex scenes, and virtual worlds. We provide a systematic analysis and visualization of the evaluation results, aiming to establish a new standard for assessing video generation models with synchronous audio capabilities and to promote the comprehensive advancement of the field.