超越单视角充分性:CVBench 跨视角人体理解基准
Beyond Single-View Sufficiency: CVBench for Cross-View Human Understanding
该基准通过强制单视角信息不足的设定,暴露了现有多模态大模型在跨视角融合上的系统性缺陷,为设计具备几何 grounding 和身份持久性的视觉架构提供了诊断工具与明确方向。
原文依据 every item is designed with verifiable single-view insufficiency, mandating that models synthesize disparate evidence to resolve ambiguities.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论人类对社会环境的感知本质上是一个多视角综合问题,需要整合跨空间和时间互补且常被遮挡的信息。然而,现有多模态大语言模型(MLLMs)的基准绝大多数基于“充分视角”假设,奖励单视角模式识别,却未能评估跨视角融合。为填补这一关键空白,我们提出 CVBench,一个大规模、多任务的跨视角人体理解基准。CVBench 包含 12 项空间和时序任务中的 3,000 个挑战性问题,每个问题都设计为具有可验证的单视角信息不足性,要求模型综合不同证据以解决歧义。我们对最优开源和闭源 MLLMs 的全面评估揭示了显著性能差距,最佳模型落后人类表现近 50 分。我们识别出所有模型共有的系统性失败机制:主导性的“单视角偏差”,即模型忽略冲突证据,默认采用最自信但错误的单视角预测。这表明当前 MLLMs 缺乏几何 grounding、身份持久性和真正时空融合的基本机制。CVBench 提供了一个严格的诊断框架,以促进下一代跨视角感知架构的发展。
英文原文摘要 1437 字符
Human perception of social environments is inherently a multi-view synthesis problem, requiring the integration of complementary and often occluded information across space and time. However, existing benchmarks for Multimodal Large Language Models (MLLMs) are overwhelmingly predicated on a "sufficient-view" assumption, rewarding single-view pattern recognition while failing to evaluate cross-view fusion. To address this critical gap, we introduce CVBench, a large-scale, multi-task benchmark for cross-view human understanding. CVBench comprises 3,000 challenging questions across 12 spatial and temporal tasks, where every item is designed with verifiable single-view insufficiency, mandating that models synthesize disparate evidence to resolve ambiguities. Our comprehensive evaluation of state-of-the-art open and closed-source MLLMs reveals a substantial performance gap, with the best models falling nearly 50 points behind human performance. We identify a systemic failure mechanism across all models: a dominant "Single-View Bias", whereby models ignore conflicting evidence and default to the most confident but incorrect single-view prediction. This demonstrates that current MLLMs lack the fundamental mechanisms for geometric grounding, identity persistence, and true spatio-temporal fusion. CVBench provides a rigorous diagnostic framework to catalyze the development of next-generation, cross-view-aware architectures.