AVA-Bench：面向视觉基础模型的原子视觉能力基准
AVA-Bench: Atomic Visual Ability Benchmark for Vision Foundation Models
把 VQA 评测拆成 14 个原子视觉能力并让训练/测试分布对齐，可直接用来给任意 VFM 做能力指纹诊断，避免把 LLM 指令微调数据不匹配误判为视觉能力缺陷；同时证明 0.5B LLM 头即可复现 7B 头的模型排序，为视觉评测省下 8 倍 GPU 时间。
原文依据 the first benchmark that explicitly disentangles 14 Atomic Visual Abilities (AVAs)
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉基础模型（VFM）的兴起呼唤系统化的评测。一种常见做法是将 VFM 与大型语言模型（LLM）配对作为通用头部，随后在广泛的视觉问答（VQA）基准上进行评测。然而，这一评测协议存在两个关键盲点：（i）指令微调数据可能与 VQA 测试分布不一致，这意味着错误的预测可能源于这种数据不匹配，而非 VFM 的视觉缺陷；（ii）VQA 基准往往在单个问题中要求多种视觉能力，因而难以判断错误是源于所有所需能力的缺失，还是仅源于某一项关键能力的缺失。为弥补这些空白，我们提出 AVA-Bench，这是首个显式解耦 14 项原子视觉能力（AVA）的基准；这些能力是定位、深度估计与空间理解等基础技能，共同支撑复杂的视觉推理任务。通过解耦各项 AVA 并在每项能力内部对齐训练与测试分布，AVA-Bench 能精确指出某个 VFM 究竟在何处表现优异、在何处表现欠佳。将 AVA-Bench 应用于领先的 VFM，可揭示出各自独特的“能力指纹”，从而使 VFM 的选型从凭经验的猜测转变为有章可循的工程实践。值得注意的是，我们发现 0.5B 的 LLM 与 7B 的 LLM 给出的 VFM 排序相近，同时将 GPU 小时数削减为原来的八分之一，从而实现更高效的评测。通过提供一个全面且透明的基准，我们希望 AVA-Bench 能为下一代 VFM 奠定基础。
英文原文摘要 1498 字符
The rise of vision foundation models (VFMs) calls for systematic evaluation. A common approach pairs VFMs with large language models (LLMs) as general-purpose heads, followed by evaluation on broad Visual Question Answering (VQA) benchmarks. However, this protocol has two key blind spots: (i) Instruction tuning data may not align with VQA test distributions, meaning a wrong prediction can stem from such data mismatch rather than VFMs' visual shortcomings; (ii) VQA benchmarks often require multiple visual abilities in a single question, making it difficult to determine whether errors arise from the lack of all required abilities or just one key ability. To address these gaps, we introduce AVA-Bench, the first benchmark that explicitly disentangles 14 Atomic Visual Abilities (AVAs), foundational skills such as localization, depth estimation, and spatial understanding, which collectively support complex visual reasoning tasks. By decoupling AVAs and matching training and test distributions within each, AVA-Bench pinpoints exactly where a VFM excels or falters. Applying AVA-Bench to leading VFMs thus reveals distinctive "ability fingerprints," turning VFM selection from educated guesswork into principled engineering. Notably, we find that a 0.5B LLM yields similar VFM rankings as a 7B LLM while cutting GPU hours by 8x, enabling more efficient evaluation. By offering a comprehensive and transparent benchmark, we hope AVA-Bench lays the foundation for the next generation of VFMs.