EagleVision:基于鸟瞰图视觉定位思维链的双阶段空间智能框架
EagleVision: A Dual-Stage Framework with BEV-grounding-based Chain-of-Thought for Spatial Intelligence
提出基于鸟瞰图(BEV)的主动视角选择与空间思维链推理框架,可复用于需要多视角几何推理的视觉语言任务,如具身导航与三维场景理解。
原文依据 a dual-stage framework that combines geometry-aware frame selection with active, Bird's-Eye-View (BEV)-grounded reasoning
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论基于视频的空间推理——例如估计距离、判断方向或从多视角理解布局——需要选择信息丰富的帧,并在推理过程中按需主动寻求额外视角。现有多模态大语言模型(MLLM)仅消费固定数量的均匀采样帧,且一旦开始推理便无法请求新视角,常常缺失可靠空间判断所需的几何线索。我们提出 EagleVision,一个双阶段框架,将几何感知的帧选择与主动的、基于鸟瞰图(BEV)视觉定位的推理相结合。在第一阶段(宏观感知),语义-视角融合行列式点过程(SPF-DPP)在固定词元预算下选择一组紧凑的关键帧,联合最大化语义相关性与视角多样性。在第二阶段(微观验证),模型执行迭代式空间思维链:每一步它可以在文本中推理,或在 BEV 平面上预测一个位姿以检索最近的真实帧,形成闭环的“假设-观察-验证”循环。查询策略完全通过带有空间视觉定位奖励的强化学习训练,无需人工标注的推理轨迹。在 VSI-Bench 和 SQA3D 上,EagleVision 在开源视觉语言模型中取得了最优性能。
英文原文摘要 1375 字符
Video-based spatial reasoning -- such as estimating distances, judging directions, or understanding layouts from multiple views -- requires selecting informative frames and, when needed, actively seeking additional viewpoints during inference. Existing multimodal large language models (MLLMs) consume a fixed set of uniformly sampled frames and cannot request new views once reasoning begins, often missing the geometric cues necessary for reliable spatial judgments. We present EagleVision, a dual-stage framework that combines geometry-aware frame selection with active, Bird's-Eye-View (BEV)-grounded reasoning. In the first stage (macro perception), a semantics-perspective-fusion determinantal point process (SPF-DPP) selects a compact set of keyframes that jointly maximize semantic relevance and viewpoint diversity under a fixed token budget. In the second stage (micro verification), the model performs iterative spatial Chain-of-Thought: at each step it can either reason in text or predict a pose on the BEV plane to retrieve the nearest real frame, forming a closed-loop hypothesize-look-verify cycle. The querying policy is trained purely via reinforcement learning with a spatial grounding reward, requiring no human-annotated reasoning traces. On VSI-Bench and SQA3D, EagleVision achieves state-of-the-art performance among open-source vision-language models.