Progress-Think:面向视觉语言导航的语义进度推理
Progress-Think: Semantic Progress Reasoning for Vision-Language Navigation
把「指令执行到哪一步」显式建模为语义进度状态,并用视觉历史与指令前缀的可微对齐自举出该状态,再注入策略上下文——这套进度表征可直接迁移到任何长程视觉-语言序列决策任务(如视频指令跟随、具身操作),也可作为VLA模型的辅助推理头。
原文依据 predicting instruction-style progress from visual observations to enable more accurate navigation
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉语言导航要求智能体在长时程上连贯行动,不仅要理解局部视觉上下文,还要理解自己在多步指令中已推进到何处。然而,近期的视觉-语言-动作模型侧重于直接预测动作,早期的进度方法则预测数值化的完成度;二者都忽略了观测序列与指令序列的单调协同推进特性。基于这一洞见,Progress-Think 提出语义进度推理,从视觉观测中预测指令风格的进度,从而实现更准确的导航。为在无标注条件下实现这一点,我们提出一个三阶段框架。在第一阶段,自对齐进度预训练通过视觉历史与指令前缀之间新颖的可微对齐来引导一个推理模块。随后,进度引导策略预训练将学到的进度状态注入导航上下文,引导策略产生一致的动作。最后,进度-策略协同微调以定制的进度感知强化目标联合优化两个模块。在 R2R-CE 与 RxR-CE 上的实验在成功率、效率与可解释性上均取得显著提升,表明语义进度为导航推进提供了一种更一致、更具泛化性的表征。
英文原文摘要 1344 字符
Vision-Language Navigation requires agents to act coherently over long horizons by understanding not only local visual context but also how far they have advanced within a multi-step instruction.However, recent Vision-Language-Action models focus on direct action prediction and earlier progress methods predict numeric achievements; both overlook the monotonic co-progression property of the observation and instruction sequences.Building on this insight, Progress-Think introduces semantic progress reasoning, predicting instruction-style progress from visual observations to enable more accurate navigation.To achieve this without annotations, we propose a three-stage framework.In the initial stage, Self-Aligned Progress Pretraining bootstraps a reasoning module via a novel differentiable alignment between visual history and instruction prefixes.Then, Progress-Guided Policy Pretraining injects learned progress states into the navigation context, guiding the policy toward consistent actions.Finally, Progress-Policy Co-Finetuning jointly optimizes both modules with tailored progress-aware reinforcement objectives.Experiments on R2R-CE and RxR-CE show substantial gains in success, efficiency, and interpretability, demonstrating semantic progress provides a more consistent and generalizable representation of navigation advancement.