BridgeEQA：面向真实桥梁巡检的虚拟具身智能体
BridgeEQA: Virtual Embodied Agents for Real Bridge Inspections
将基础设施巡检建模为情景记忆具身问答任务，并提出以马尔可夫决策过程求解的 EMVR 框架；其『引用相关图像』的评测指标与长程多尺度空间推理设定，可迁移到需要跨视角检索证据的视觉语言任务。
原文依据 we propose Embodied Memory Visual Reasoning (EMVR), which formulates the inspection EQA task as a Markov decision process
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论在真实世界场景中部署能够回答周围环境问题的具身智能体仍然困难，部分原因在于缺乏用于情景记忆具身问答（EQA）的基准。受基础设施巡检挑战的启发，我们提出巡检 EQA 这一具有吸引力的新问题类别，以推动情景记忆 EQA 的发展，因为它要求多尺度推理与长程空间理解，同时提供标准化评测、作为依据的专业巡检报告以及第一人称图像。我们提出 BridgeEQA，一个包含 2,200 个开放词表问答对的基准（采用 OpenEQA 的风格），其依据是覆盖 200 个真实桥梁场景的专业巡检报告，每个场景平均包含 47.93 张图像。我们进一步提出一种新的 EQA 指标——图像引用相关性，用于评估模型引用相关图像的能力。对最优视觉语言模型的评测揭示了显著的性能差距。为解决这一问题，我们提出具身记忆视觉推理（EMVR），将巡检 EQA 任务建模为马尔可夫决策过程。EMVR 在基线方法之上表现出强劲的性能。代码与数据集见：https://drags99.github.io/bridge-eqa/
英文原文摘要 1258 字符
Deploying embodied agents that can answer questions about their surroundings in realistic real-world settings remains difficult, partly due to the scarcity of benchmarks for episodic memory Embodied Question Answering (EQA). Inspired by the challenges of infrastructure inspections, we propose Inspection EQA as a compelling problem class for advancing episodic memory EQA, as it demands multi-scale reasoning and long-range spatial understanding, while offering standardized evaluation, professional inspection reports as grounding, and egocentric imagery. We introduce BridgeEQA, a benchmark of 2,200 open-vocabulary question-answer pairs (in the style of OpenEQA) grounded in professional inspection reports across 200 real-world bridge scenes with 47.93 images on average per scene. We further propose a new EQA metric Image Citation Relevance to evaluate the ability of a model to cite relevant images. Evaluations of state-of-the-art vision-language models reveal substantial performance gaps. To address this, we propose Embodied Memory Visual Reasoning (EMVR), which formulates the inspection EQA task as a Markov decision process. EMVR shows strong performance over the baselines. Code and dataset available at: https://drags99.github.io/bridge-eqa/