VideoRealBench：面向生成式人类中心视频的思维链真实性评估基准
VideoRealBench: A Chain-of-Thought Realism Evaluation Benchmark for Generated Human-Centric Videos
为生成视频的真实性评估提供了带思维链推理的细粒度标注数据集和可解释评估器，视觉研究者可将其用于诊断视频生成模型在人体视频上的真实性缺陷，或借鉴其三步推理标注范式构建其他生成任务的评估基准。
原文依据 we propose VideoRealBench, a comprehensive benchmark for evaluating the realism of generated human-centric videos
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论随着视频生成模型的巨大进步，越来越多的内容创作者和研究人员利用这些技术来生成大量人类中心视频，用于内容创作和特定任务的定制数据生成。尽管现有的视频生成模型能够生成具有高视觉质量的视频，但它们对视频真实性理解不足，导致生成不真实的视频。虽然已经出现了各种评估器来评估生成视频的质量，但它们是从低质量的生成视频和数据标注中训练的，导致评分与人类偏好不一致。由于缺乏思维链推理，它们也缺乏可解释性。为了解决这些问题，我们提出了VideoRealBench，一个用于评估生成式人类中心视频真实性的综合基准。我们利用根据人类偏好设计的评分量表对视频进行评分，并提供三步推理依据，从而创建了一个精细标注的数据集VideoRealDataset，并提出了一个能够提供可靠分数和详细推理依据的评估器VideoRealEval。VideoRealEval在VideoRealDataset上达到了57.07%的皮尔逊线性相关系数（PLCC）和56.78%的斯皮尔曼等级相关系数（SROCC），表明其与人类偏好的对齐程度优于现有评估器。我们的代码和数据集可在https://github.com/MCG-NJU/VideoRealBench获取。
英文原文摘要 1492 字符
With the great advancement of video generation models, a growing number of content creators and researchers are leveraging these technologies to produce large volumes of human-centric videos for content creation and customized data generation for specific tasks. Although existing video generation models are capable of producing videos with high visual quality, their inadequate understanding of video realism results in generating unrealistic videos. While various evaluators have emerged to assess the quality of generated videos, they are trained from low-quality generated videos and data annotations, leading to misaligned ratings with human preferences. They also lack interpretability due to the absence of chain-of-thought reasoning. To address these issues, we propose VideoRealBench, a comprehensive benchmark for evaluating the realism of generated human-centric videos. We leverage a rating scale designed from human preferences to score videos and provide three-step rationales, thereby creating a finely-annotated dataset VideoRealDataset and proposing an evaluator VideoRealEval capable of providing reliable scores along with detailed rationales. VideoRealEval achieves a Pearson's linear correlation coefficient (PLCC) of 57.07% and a Spearman's rank correlation coefficient (SROCC) of 56.78% on VideoRealDataset, demonstrating closer alignment with human preferences than existing evaluators. Our code and dataset are available at https://github.com/MCG-NJU/VideoRealBench.