UNICBench：面向多模态大模型的统一计数基准
UNICBench: UNIfied Counting Benchmark for MLLM
为视觉研究者提供了一套跨图像/文本/音频的统一计数评测协议与工具包（固定划分、确定性数值解析、分层报告），可直接用于诊断 MLLM 在视觉计数上的长尾失效，并作为计数类视觉任务的数据与指标参照。
原文依据 a unified multimodal, multi-level counting benchmark and evaluation toolkit with accurate ground truth, deterministic numeric parsing, and stratified reporting
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论计数是多模态大模型（MLLM）的一项核心能力，然而目前尚缺乏一个统一的计数数据集来严格评估其在图像、文本和音频上的这一能力。我们提出 UNICBench，这是一个统一的多模态、多层级计数基准与评测工具包，具备准确的真值、确定性的数值解析以及分层报告。该语料库包含 5,300 张图像（5,508 条问答）、872 篇文档（5,888 条问答）和 2,069 段音频片段（2,905 条问答），并标注了三级能力分类体系与难度标签。在固定划分、固定提示词、固定随机种子并采用模态特定匹配规则的标准化协议下，我们跨模态评估了 45 个最优 MLLM。结果表明，模型在某些基础计数任务上表现强劲，但在推理任务和最难的划分上存在显著差距，凸显出长尾错误以及提升通用计数能力的巨大空间。UNICBench 为测量提供了严格且可比较的基础，并提供了公开工具包以加速进展。
英文原文摘要 1019 字符
Counting is a core capability for multimodal large language models (MLLMs), yet there is no unified counting dataset to rigorously evaluate this ability across image, text, and audio. We present UNICBench, a unified multimodal, multi-level counting benchmark and evaluation toolkit with accurate ground truth, deterministic numeric parsing, and stratified reporting. The corpus comprises 5,300 images (5,508 QA), 872 documents (5,888 QA), and 2,069 audio clips (2,905 QA), annotated with a three-level capability taxonomy and difficulty tags. Under a standardized protocol with fixed splits/prompts/seeds and modality-specific matching rules, we evaluate 45 state-of-the-art MLLMs across modalities. Results show strong performance on some basic counting tasks but significant gaps on reasoning and the hardest partitions, highlighting long-tail errors and substantial headroom for improving general counting. UNICBench offers a rigorous and comparable basis for measurement and a public toolkit to accelerate progress.