强化学习与具身智能 · CVPR2026 ·2026.06.03

重新审视相机选择:机器人操作中鱼眼相机特性的实证研究

Rethinking Camera Choice: An Empirical Study on Fisheye Camera Properties in Robotic Manipulation

Han Xue、Nan Min、Xiaotong Liu、Wendi Chen、Yuan Fang、Jun Lv 等 8 人

与你有关的部分

对做具身视觉的人,这篇给出了腕部鱼眼相机在模仿学习中的系统性选型依据:宽视场提升空间定位但依赖场景视觉复杂度,且跨相机迁移失败的主因是尺度过拟合,可用随机尺度增强(RSA)缓解——这提示视觉骨干的尺度不变性设计对策略泛化至关重要。

原文依据 we identify the root cause as scale overfitting and demonstrate that hardware generalization performance can be improved with a simple Random Scale Augmentation (RSA) strategy
可迁移到 imitation learningrobotic manipulationcross-camera generalization

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

鱼眼相机凭借其极宽的视场(FoV)在机器人操作中的应用正迅速普及,但其对策略学习的下游影响却缺乏系统性的理解。本文开展了首个全面的实证研究以弥合这一差距,严谨分析了腕部安装鱼眼相机在模仿学习中的特性。通过在仿真与真实世界中的大量实验,我们考察了三个关键研究问题:空间定位、场景泛化与硬件泛化。我们的研究表明:(1)宽视场显著增强了空间定位能力,但这一收益严重依赖于环境的视觉复杂度。(2)鱼眼训练的策略虽然在简单场景中易出现过拟合,但在具备充分环境多样性时训练可解锁更优的场景泛化能力。(3)尽管朴素的跨相机迁移会导致失败,我们将其根本原因识别为尺度过拟合,并证明通过简单的随机尺度增强(RSA)策略可以提升硬件泛化性能。总体而言,我们的发现为机器人学习中鱼眼数据集的大规模采集与有效使用提供了具体、可操作的指导。

英文原文摘要 1309 字符

The adoption of fisheye cameras in robotic manipulation, driven by their exceptionally wide Field of View (FoV), is rapidly outpacing a systematic understanding of their downstream effects on policy learning. This paper presents the first comprehensive empirical study to bridge this gap, rigorously analyzing the properties of wrist-mounted fisheye cameras for imitation learning. Through extensive experiments in both simulation and the real world, we investigate three critical research questions: spatial localization, scene generalization, and hardware generalization. Our investigation reveals that: (1) The wide FoV significantly enhances spatial localization, but this benefit is critically contingent on the visual complexity of the environment. (2) Fisheye-trained policies, while prone to overfitting in simple scenes, unlock superior scene generalization when trained with sufficient environmental diversity. (3) While naive cross-camera transfer leads to failures, we identify the root cause as scale overfitting and demonstrate that hardware generalization performance can be improved with a simple Random Scale Augmentation (RSA) strategy. Collectively, our findings provide concrete, actionable guidance for the large-scale collection and effective use of fisheye datasets in robotic learning.

标签

会议 CVPR2026
页码 35059-35069
Bibkey
来源 CVF Open Access