强化学习与具身智能 · CVPR2026 ·2026.06.03

先预测再探索:面向具身问答的专用记忆预测式规划

Predict Before You Explore: Predictive Planning with Specialized Memory for Embodied Question Answering

Bowen Yuan、Sisi You、Bing-Kun Bao

与你有关的部分

把「预测-纠错」循环引入具身导航规划:高层规划器先预测问题相关证据可能出现的位置并生成少量长时程探索分支,低层执行器在分支内降低不确定性并在预测失败时修正;同时用结构先验记忆与问题相关视觉证据记忆分离的双记忆设计,可直接迁移到任何部分可观测下的视觉主动探索与长时程导航任务。

原文依据 A high-level planner predicts where question-relevant evidence is likely to appear and generates a compact set of actionable exploration branches
可迁移到 具身问答视觉导航主动探索视觉语言导航

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

具身问答(EQA)要求智能体在三维环境中导航、累积视觉证据,并对部分观测进行推理以回答问题。然而,现有智能体难以维持连贯的长时程行为:规划仍是反应式的,导致动作不一致;而单体式记忆将所有观测纠缠在一起,阻碍了对稀疏但关键证据的检索。我们通过预测处理的视角重新审视 EQA 来解决这些问题:在该视角下,连贯行为源于以稳定先验为基础的预测-纠错循环。在这一视角的指导下,我们提出「先预测再探索」(Pred-EQA),一种将预测式规划与专用记忆相结合的架构。高层规划器预测与问题相关的证据可能出现的位置,并生成一小组编码长时程意图的可执行探索分支。随后,低层执行器在这些分支内降低不确定性,并在预测失败时修正预测。双记忆系统通过将缓慢演化的结构先验与紧凑的、与问题相关的视觉证据分离,对这一过程形成补充,从而实现一致的规划与高效的证据累积。通过这种预测引导的探索,Pred-EQA 在部分可观测条件下获得了连贯的轨迹。在 OpenEQA 与 Express-Bench 上的实验表明,Pred-EQA 在准确率与探索效率两方面均取得最优结果,验证了预测驱动的具身推理的优势。

英文原文摘要 1554 字符

Embodied Question Answering (EQA) requires agents to navigate 3D environments, accumulate visual evidence, and reason over partial observations to answer questions. However, current agents struggle to maintain coherent, long-horizon behavior: planning remains reactive, causing inconsistent actions, while monolithic memories entangle all observations, hindering retrieval of the sparse but crucial evidence. We address these issues by reframing EQA through the lens of predictive processing, in which coherent behavior emerges from a prediction-correction loop grounded in stable priors. Guided by this perspective, we propose Predict Before You Explore (Pred-EQA), an architecture that integrates predictive planning with specialized memory. A high-level planner predicts where question-relevant evidence is likely to appear and generates a compact set of actionable exploration branches encoding long-horizon intent. A low-level executor then reduces uncertainty within these branches, revising predictions when they fail. A dual-memory system complements this process by separating slowly evolving structural priors from compact, question-relevant visual evidence, enabling consistent planning and efficient evidence accumulation. Through this prediction-guided exploration, Pred-EQA achieves coherent trajectories under partial observability. Experiments on OpenEQA and Express-Bench show that Pred-EQA achieves state-of-the-art results in both accuracy and exploration efficiency, demonstrating the benefits of prediction-driven embodied reasoning.

标签

会议 CVPR2026
页码 29610-29619
Bibkey
来源 CVF Open Access