从左脑到右脑：面向视觉语言导航的自适应文本想象器
Cross from Left to Right Brain: Adaptive Text Dreamer for Vision-and-Language Navigation
用 LLM 以语言形式想象未来场景语义、替代视觉合成来缓解 VLN 的部分可观测性，并给出 decoder-free 潜空间接口把想象结果注入导航专家模块，这套「文本想象 + 潜空间对齐」思路可迁移到其他需要预测未来观测的具身导航与探索任务。
原文依据 we propose to adaptively imagine key environmental semantics via language form, enabling a more reliable and efficient strategy
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉语言导航（VLN）要求智能体依据自然语言指令进行导航。由于部分可观测性，该任务颇具挑战，使得感知与语言难以对齐。近期方法通过想象未来场景来缓解这一问题，但它们依赖基于视觉的合成，导致计算开销高且细节冗余。为此，我们提出以语言形式自适应地想象关键环境语义，从而获得更可靠、更高效的策略。具体而言，我们引入自适应文本想象器（ATD），这是一种构建于大语言模型（LLM）之上的双分支自引导想象策略。ATD 采用类人左右脑架构设计，其中左脑侧重于逻辑整合，右脑负责对未来场景进行想象性预测。为实现这一点，我们仅微调两个脑区中的 Q-former，以高效激活 LLM 中的领域特定知识，使逻辑推理与想象能够在导航过程中动态更新。此外，我们提出一种交叉交互机制，对想象得到的潜变量空间输出进行正则化，并通过无解码器的潜变量接口将其与导航专家模块集成，从而使 ATD 能够联合利用 LLM 的推理能力与导航模型的任务特定知识。我们在 R2R、REVERIE 和 R4R 基准上开展了大量实验，结果表明 ATD 以显著更少的参数取得了有竞争力的性能。代码将公开。
英文原文摘要 1586 字符
Vision-and-Language Navigation (VLN) requires the agent to navigate based on natural instructions. This task is challenging due to partial observability, which makes it difficult to align perception with language. Recent methods mitigate this by imagining future scenes, yet they rely on vision-based synthesis, leading to high computational cost and redundant details. To this end, we propose to adaptively imagine key environmental semantics via language form, enabling a more reliable and efficient strategy. Specifically, we introduce Adaptive Text Dreamer (ATD), a dual-branch self-guided imagination policy built upon a large language model (LLM). ATD is designed with a human-like left-right brain architecture, where the left brain focuses on logical integration, and the right brain is responsible for imaginative prediction of future scenes. To achieve this, we fine-tune only the Q-former within both brains to efficiently activate domain-specific knowledge in the LLM, enabling dynamic updates of logical reasoning and imagination during navigation. Furthermore, we propose a cross-interaction mechanism that regularizes the imagined latent-space outputs and integrates them with the navigation expert module via a decoder-free latent interface, thereby enabling ATD to jointly harness the reasoning ability of the LLM and the task-specific knowledge of the navigation model. We conduct extensive experiments across the R2R, REVERIE, and R4R benchmarks, demonstrating that ATD achieves competitive performance with significantly fewer parameters. The code will be available.