机器人何时该说「我不知道」：具身问答中弃权行为的基准评测
When Robots Should Say ''I Don't Know'': Benchmarking Abstention in Embodied Question Answering
为视觉语言模型在具身场景中引入「拒答」能力提供了可复用的评测基准与五类弃权分类法，做 VLM 可靠性或交互式问答时可据此设计拒答召回指标与歧义问题构造流程。
原文依据 we find that even the best frontier model only attains 42.79% abstention recall, while humans achieve 91.17%
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论具身问答（EQA）要求智能体理解语言、感知环境并在三维场景中导航以给出回答。现有 EQA 基准假定每个问题都必须被回答，但具身智能体应当知道自己何时缺乏足够信息作答。本文聚焦于 EQA 智能体的一项最低要求——弃权：知道何时应拒绝作答。基于对 500 条人类查询的初步研究，我们发现其中 32.4% 包含缺失或未充分说明的上下文。依据该初步研究以及关于人类交流错误的认知理论，我们归纳出五类需要弃权的代表性情形：可执行性受限、指代不充分、偏好依赖、信息不可得以及错误预设。我们对 OpenEQA 进行扩充，由标注者按照上述类别将表述良好的问题改写为歧义变体。所得数据集 AbstainEQA 包含 1,636 个标注弃权案例，并与 1,636 个原始 OpenEQA 实例配对，以实现均衡评测。在 AbstainEQA 上的评测显示，即便最先进的前沿模型也仅达到 42.79% 的弃权召回率，而人类可达 91.17%。我们还发现，规模扩展、提示工程与推理仅带来边际提升，且微调模型会过拟合于文本线索。这些结果共同表明，弃权是具身场景中可靠交互的基本前提，也是有效澄清的必要基础。
英文原文摘要 1523 字符
Embodied Question Answering (EQA) requires an agent to interpret language, perceive its environment, and navigate within 3D scenes to produce responses. Existing EQA benchmarks assume that every question must be answered, but embodied agents should know when they do not have sufficient information to answer. In this work, we focus on a minimal requirement for EQA agents, abstention: knowing when to withhold an answer. From an initial study of 500 human queries, we find that 32.4% contain missing or underspecified context. Drawing on this initial study and cognitive theories of human communication errors, we derive five representative categories requiring abstention: actionability limitation, referential underspecification, preference dependence, information unavailability, and false presupposition. We augment OpenEQA by having annotators transform well-posed questions into ambiguous variants outlined by these categories. The resulting dataset, AbstainEQA, comprises 1,636 annotated abstention cases paired with 1,636 original OpenEQA instances for balanced evaluation. Evaluating on AbstainEQA, we find that even the best frontier model only attains 42.79% abstention recall, while humans achieve 91.17%. We also find that scaling, prompting, and reasoning only yield marginal gains, and that fine-tuned models overfit to textual cues. Together, these results position abstention as a fundamental prerequisite for reliable interaction in embodied settings and as a necessary basis for effective clarification.