Spot The Ball：视觉社会推理基准
Spot The Ball: A Benchmark for Visual Social Inference
提供了一个需要从注视方向、身体姿态等社会线索推断隐藏目标的评测设定，可用于检验 VLM 是否真正利用结构化行为线索而非空间捷径，并启发在视觉编码中显式建模人体朝向与视线几何。
原文依据 Humans excel at visual social inference, the ability to infer hidden elements of a scene from subtle behavioral cues
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论人类擅长视觉社会推理，即从他人的注视、姿态和朝向等细微行为线索推断场景中隐藏元素的能力。这种能力驱动着人类日常的社会推理，并且对于开发更接近人类的 AI 智能体至关重要。我们提出 Spot The Ball，一个以体育为测试领域的、用于评估视觉语言模型（VLM）视觉社会推理能力的挑战性基准。任务是从足球、篮球和排球图像中定位一个被移除的球。我们给出了带有人类基线的精选评测集，以及一个可扩展的、用于生成额外测试样本的流水线。我们使用三种提示策略评估了四个最优 VLM（Gemini、GPT、LLaMA、Qwen），发现人类在所有体育项目上的准确率（20-34%）始终是模型（≤17%）的两到三倍。我们的分析表明，模型依赖表面的空间启发式——例如猜测靠近图像中心或靠近球员的位置——而人类则利用注视方向和身体姿态等社会线索。这些发现揭示了视觉社会推理中持续存在的人机差距，并凸显了需要显式编码结构化行为线索的架构，以实现稳健的、类人的推理。
英文原文摘要 1277 字符
Humans excel at visual social inference, the ability to infer hidden elements of a scene from subtle behavioral cues such as other people's gaze, pose, and orientation. This capacity drives everyday social reasoning in humans and is critical for developing more human-like AI agents. We introduce Spot The Ball, a challenging benchmark for evaluating visual social inference in vision-language models (VLMs) using sports as a test domain. The task is to localize a removed sports ball from soccer, basketball, and volleyball images. We present a curated evaluation set with human baselines and a scalable pipeline for generating additional test items. We evaluate four state-of-the-art VLMs (Gemini, GPT, LLaMA, Qwen) using three prompting strategies, finding that humans are consistently two to three times more accurate (20-34%) than models (<=17%) across all sports. Our analyses show that models rely on superficial spatial heuristics--such as guessing near the image center or nearby players--while humans leverage social cues like gaze direction and body pose. These findings reveal a persistent human-model gap in visual social reasoning and underscore the need for architectures that explicitly encode structured behavioral cues to achieve robust, human-like inference.