EgoProx：在认知层级上评估多模态大语言模型的自我中心三维邻近推理
EgoProx: Evaluating MLLMs on Egocentric 3D Proximity Reasoning Across a Cognitive Hierarchy
为视觉研究者提供了一个以自我中心视角评估 MLLM 三维邻近关系推理的基准与智能体式数据生成引擎，可用于诊断模型的空间知识是否被有效调用，并指导空间推理 VQA 的指令微调设计。
原文依据 we introduce EgoProx, a benchmark for egocentric 3D proximity reasoning
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论人类在日常生活中不断对三维邻近关系进行推理，即自身身体与周围物体之间的关系，以指导感知与行动。多模态大语言模型（MLLMs）能否执行此类具身三维推理仍不清楚。为此，我们提出 EgoProx，一个面向自我中心三维邻近推理的基准。我们沿一条认知链组织任务，涵盖意图、探索、利用以及动作链推理。我们还设计了一个基于智能体的数据引擎，可大规模生成多样且一致的问答对。我们在 EgoProx 上对主流 MLLMs 进行了基准评测，并开展了针对数据集与任务的指令微调的额外分析。我们观察到显著的跨域增益，表明当前 MLLMs 已具备一定的空间知识；然而，它们仍难以有效利用这些知识进行空间推理视觉问答。
英文原文摘要 878 字符
Humans constantly reason about 3D proximity, the relations between their body and surrounding objects, to guide perception and action in daily life. Whether multimodal large language models (MLLMs) can perform such embodied 3D reasoning remains unclear. To this end, we introduce EgoProx, a benchmark for egocentric 3D proximity reasoning. We organize our tasks along a cognitive chain, covering intention, exploration, exploitation, and chain-of-actions reasoning. We also design an agent based data engine that produces diverse and consistent QA pairs at scale. We benchmark prevailing MLLMs on EgoProx and conduct additional analyses with dataset specific and task specific instruction tuning. We observe large cross-domain gains, indicating that current MLLMs contain some spatial knowledge; however, they still struggle to effectively leverage it for spatial reasoning VQA.