AURA：面向城市导航的多模态共享自主
AURA: Multi-modal Shared Autonomy for Urban Navigation
把城市导航拆成高层人类指令与低层 AI 控制，并用空间感知指令编码器把语言指令对齐到视觉与空间上下文，这个「指令-空间对齐 + 在线自适应」的接口设计可直接迁移到视觉语言导航与具身智能体的共享控制中。
原文依据 AURA incorporates a Spatial-Aware Instruction Encoder to align various human instructions with visual and spatial context.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论复杂城市环境中的长时程导航高度依赖人类持续操作，这会导致疲劳、效率下降与安全隐患。共享自主——由视觉语言 AI 智能体与人类操作员协同操控移动机器——为解决上述问题提供了有前景的方案。然而，现有共享自主方法往往要求人类与 AI 在同一动作空间内操作，带来较高的认知负担。我们提出辅助城市机器人自主（AURA），一种新的多模态框架，将城市导航分解为高层人类指令与低层 AI 控制。AURA 引入空间感知指令编码器，将各类人类指令与视觉及空间上下文对齐。为便于训练，我们构建了 MM-CoS，一个包含遥操作与视觉语言描述的大规模数据集。仿真与真实世界实验表明，AURA 能有效遵循人类指令、降低人工操作负担并提升导航稳定性，同时支持在线自适应。此外，在相近的接管条件下，我们的共享自主框架将接管频率降低超过 44%。项目主页提供了演示视频与更多细节。
英文原文摘要 1302 字符
Long-horizon navigation in complex urban environments relies heavily on continuous human operation, which leads to fatigue, reduced efficiency, and safety concerns. Shared autonomy, where a Vision-Language AI agent and a human operator collaborate on maneuvering the mobile machine, presents a promising solution to address these issues. However, existing shared autonomy methods often require humans and AI to operate within the same action space, leading to high cognitive overhead. We present Assistive Urban Robot Autonomy (AURA), a new multi-modal framework that decomposes urban navigation into high-level human instruction and low-level AI control. AURA incorporates a Spatial-Aware Instruction Encoder to align various human instructions with visual and spatial context. To facilitate training, we construct MM-CoS, a large-scale dataset comprising teleoperation and vision-language descriptions. Experiments in simulation and the real world demonstrate that AURA effectively follows human instructions, reduces manual operation effort, and improves navigation stability, while enabling online adaptation. Moreover, under similar takeover conditions, our shared autonomy framework reduces the frequency of takeovers by more than 44%. Demo video and more detail are provided in the project page.