STRNet:通过动态图聚合的时空表征视觉导航
STRNet: Visual Navigation with Spatio-Temporal Representation through Dynamic Graph Aggregation
提出一种可泛化的视觉骨干网络,通过空间图推理和混合时序移位模块融合图像序列与目标观测,可复用于目标条件控制任务中的视觉编码。
原文依据 a unified spatio-temporal representation framework that enhances visual encoding for robotic navigation
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉导航要求机器人基于一系列第一人称视觉观测到达指定目标(例如一张图像)。尽管近期基于学习的方法取得了显著进展,但它们往往专注于改进策略头或决策策略,而依赖简单的特征编码器和时序池化来表示视觉输入。这导致细粒度的空间和时间结构丢失,最终限制了准确的动作预测和进度估计。本文提出一个统一的时空表征框架,以增强机器人导航的视觉编码。我们的方法从图像序列和目标观测中提取特征,并使用设计的时空融合模块进行融合。该模块在每一帧内执行空间图推理,并使用结合多分辨率差异感知卷积的混合时序移位模块来建模时序动态。实验结果表明,我们的方法持续提升导航性能,并为目标条件控制提供了一个可泛化的视觉骨干网络。代码见 https://github.com/hren20/STRNet。
英文原文摘要 1200 字符
Visual navigation requires the robot to reach a specified goal such as an image, based on a sequence of first-person visual observations. While recent learning-based approaches have made significant progress, they often focus on improving policy heads or decision strategies while relying on simplistic feature encoders and temporal pooling to represent visual input. This leads to the loss of fine-grained spatial and temporal structure, ultimately limiting accurate action prediction and progress estimation. In this paper, we propose a unified spatio-temporal representation framework that enhances visual encoding for robotic navigation. Our approach extracts features from both image sequences and goal observations, and fuses them using the designed spatio-temporal fusion module. This module performs spatial graph reasoning within each frame and models temporal dynamics using a hybrid temporal shift module combined with multi-resolution difference-aware convolution. Experimental results demonstrate that our approach consistently improves navigation performance and offers a generalizable visual backbone for goal-conditioned control. Code is available at https://github.com/hren20/STRNet.