TrajRAG:面向零样本目标导航的几何-语义经验检索
TrajRAG: Retrieving Geometric-Semantic Experience for Zero-Shot Object Navigation
把导航过程中产生的历史观测压缩成拓扑-极坐标轨迹表示并做分层检索,为大模型推理提供可复用的几何-语义先验;做具身导航或VLA的读者可以直接借用这套「经验库+粗到细检索」的框架来替代纯常识先验。
原文依据 we propose a topological-polar (topo-polar) trajectory representation that compactly encodes spatial layouts and semantic contexts
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论现有的零样本目标导航(ObjectNav)方法通常利用大语言模型或视觉语言模型中的常识知识来引导导航。然而,这类知识源自互联网规模的文本而非具身三维经验,且导航过程中收集的片段式观测通常被丢弃,无法积累终身经验。为此,我们提出轨迹检索增强生成(TrajRAG),一个通过检索几何-语义经验来增强大模型推理的检索增强生成框架。TrajRAG 增量式地积累过往导航片段中的片段式观测。为对这些观测进行结构化组织,我们提出拓扑-极坐标(topo-polar)轨迹表示,紧凑地编码空间布局与语义上下文,有效去除原始片段式观测中的冗余。分层分块结构进一步将相似的拓扑-极坐标轨迹组织为统一摘要,从而实现由粗到细的检索。在导航过程中,候选前沿生成多条轨迹假设,向 TrajRAG 查询相似的过往轨迹,从而引导大模型推理进行路点选择。新经验被持续整合进 TrajRAG,实现终身导航经验的积累。在 MP3D、HM3D-v1 与 HM3D-v2 上的实验表明,TrajRAG 能有效检索相关的几何-语义经验,并提升零样本 ObjectNav 的性能。
英文原文摘要 1490 字符
Existing zero-shot Object Goal Navigation (ObjectNav) methods often exploit commonsense knowledge from large language or vision-language models to guide navigation. However, such knowledge arises from internet-scale text rather than embodied 3D experience, and episodic observations collected during navigation are typically discarded, preventing the accumulation of lifelong experience.To this end, we propose Trajectory RAG (TrajRAG), a retrieval-augmented generation framework that enhances large-model reasoning by retrieving geometric-semantic experiences. TrajRAG incrementally accumulates episodic observations from past navigation episodes. To structure these observations, we propose a topological-polar (topo-polar) trajectory representation that compactly encodes spatial layouts and semantic contexts, effectively removing redundancies in raw episodic observations. A hierarchical chunking structure further organizes similar topo-polar trajectories into unified summaries, enabling coarse-to-fine retrieval. During navigation, candidate frontiers generate multiple trajectory hypotheses that query TrajRAG for similar past trajectories, guiding large-model reasoning for waypoint selection. New experiences are continually consolidated into TrajRAG, enabling the accumulation of lifelong navigation experience.Experiments on MP3D, HM3D-v1, and HM3D-v2 show that TrajRAG effectively retrieves relevant geometric-semantic experiences and improves zero-shot ObjectNav performance.