弥合2D-3D鸿沟：面向视觉语言导航的分层语义-几何地图
Bridging the 2D-3D Gap: A Hierarchical Semantic-Geometric Map for Vision Language Navigation
将3D几何信息编码为多通道俯视地图（几何/语义/决策三层），使VLM能直接读取空间布局并选择几何有效的路点，为视觉语言导航提供了一种可复用的空间表示与解耦规划框架。
原文依据 we propose a Hierarchical Semantic-Geometric Map (HSGM) that transforms 3D geometric information into a structured representation compatible with VLMs
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉语言导航（VLN）使具身智能体能够遵循语言指令到达未见环境中的目标位置。尽管视觉语言模型（VLM）近期取得了进展，但仍存在关键的语义-几何鸿沟：VLM擅长语言和2D视觉理解，却难以进行3D空间推理，且无法捕捉动作与空间转换之间的因果动态，导致导航不可靠，尤其是在零样本设定下。为弥合这一鸿沟，我们提出分层语义-几何地图（HSGM），将3D几何信息转换为与VLM兼容的结构化表示，有效将其与物理世界连接。具体而言，HSGM表示为多通道俯视地图，组织为三个层级：（1）几何层，记录可导航区域和障碍物；（2）语义层，表示物体及其关系；（3）决策层，支持高层任务推理和目标选择。在导航过程中，VLM作为高层语义规划器，解释HSGM编码的空间布局以选择几何有效的路点，而路点之间的低层无碰撞移动由经典路径规划算法执行，从而完全将语义推理与动作执行解耦。此外，复杂指令被分解为子任务，以缓解长时程导航中的进度遗忘或幻觉问题。在R2R-CE和RxR-CE基准上的大量实验表明，我们的零样本框架达到了最优性能，甚至超越了一些监督方法。代码见https://github.com/Teacher-Tom/HSGM_public。
英文原文摘要 1758 字符
Vision-Language Navigation (VLN) enables embodied agents to reach target locations in unseen environments by following language instructions. Despite recent progress with vision-language models (VLMs), a critical semantic-geometric gap remains: while VLMs excel at language and 2D visual understanding, they struggle with 3D spatial reasoning and fail to capture the causal dynamics between actions and spatial transitions, resulting in unreliable navigation, particularly in zero-shot settings. To bridge this gap, we propose a Hierarchical Semantic-Geometric Map (HSGM) that transforms 3D geometric information into a structured representation compatible with VLMs, effectively linking them to the physical world. Specifically, HSGM is represented as a multi-channel top-down map organized into three levels: (1) geometric level that records navigable regions and obstacles, (2) semantic level that represents objects and their relations, and (3) decision level that supports high-level task reasoning and goal selection. During navigation, the VLM acts as a high-level semantic planner, interpreting the spatial layout encoded in the HSGM to select geometrically valid waypoints, while low-level, collision-free movements between waypoints are executed by a classical path-planning algorithm, fully decoupling semantic reasoning from action execution. Additionally, complex instructions are decomposed into subtasks to alleviate the problem of progress forgetting or hallucinating in long-horizon navigation. Extensive experiments on R2R-CE and RxR-CE benchmarks demonstrate that our zero-shot framework achieves state-of-the-art performance and even outperforms several supervised methods. Code is available at https://github.com/Teacher-Tom/HSGM_public.