将具身问答从感知扩展到决策
Extending Embodied Question Answering from Perception to Decision
为视觉语言模型在具身环境中的空间理解与交互推理提供了统一的四层级评测框架(静态场景构建、空间理解、任务动态推理、即时决策),并配套 RoboDecision 基线,可直接用于诊断 VLM 在感知到行动决策链条上的短板。
原文依据 EQA-Decision, a large-scale embodied QA dataset that systematically covers four complementary dimensions of embodied reasoning
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论具身问答(EQA)在具身环境中连接感知、推理与交互。然而,现有数据集与基准仍然彼此割裂,各自只关注有限的一部分推理技能,例如空间理解或程序性推理,缺乏一个统一的大规模框架来进行全面评测。我们提出 EQA-Decision,一个大规模具身问答数据集,系统覆盖具身推理的四个互补维度:静态场景构建、空间理解、任务动态推理与即时决策。该数据集包含超过四百万个问答对,并在多样化的具身场景中提供分层标注。此外,我们开发了 RoboDecision,一个与 EQA-Decision 基准对齐的强基线模型,提供统一框架,在具身环境中联合评测感知、推理与动作层面的决策。结果表明,EQA-Decision 能有效评测并增强视觉语言模型在空间与交互推理方面的能力,为推进具身智能研究提供了坚实基础。
英文原文摘要 1164 字符
Embodied Question Answering (EQA) connects perception, reasoning, and interaction within embodied environments. However, existing datasets and benchmarks remain fragmented, each focusing on a limited subset of reasoning skills such as spatial understanding or procedural reasoning, without offering a unified large-scale framework for comprehensive evaluation. We present EQA-Decision, a large-scale embodied QA dataset that systematically covers four complementary dimensions of embodied reasoning: static scene construction, spatial understanding, task dynamics reasoning, and instant decision. The dataset contains over four million question-answer pairs with hierarchical annotations across diverse embodied scenarios. In addition, we develop RoboDecision, a strong baseline model aligned with the EQA-Decision Benchmark, providing a unified framework that jointly evaluates perception, reasoning, and action-level decision-making in embodied environments. Results demonstrate that EQA-Decision effectively benchmarks and enhances VLM capabilities in spatial and interaction reasoning, providing a solid foundation for advancing embodied intelligence research.