EgoRoC:面向第一人称机器人控制的任任务无关视觉对齐
EgoRoC: Towards Egocentric Robotic Control via Task-Agnostic Visual Alignment
把第一人称腕部相机的视角对齐做成可插拔的6-DoF位姿接口,前置到任意VLA策略之前,并用扩散式在线手眼模块在末端执行器坐标系下修正动作;对做具身视觉的人,这是一种把「看」与「做」解耦、免手眼标定即可跨硬件部署的复用范式。
原文依据 a plug-and-play egocentric alignment head that precedes any task policy and exposes only a thin 6-DoF pose interface
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近期的视觉-语言-动作(VLA)模型通过端到端架构将视觉-文本输入映射为机器人动作,但这一做法把视觉理解与任务特定动作纠缠在一起。这导致需要穷尽式地采集完整操作序列,并造成跨任务的参数冗余;而通用的第三人称相机设置由于隐含的手眼假设,需要针对不同硬件进行微调。我们认为,将机器人「如何看」与「如何做」解耦是VLA系统中缺失的一项基本能力。我们提出EgoRoC,一个即插即用的第一人称对齐检测头,它位于任意任务策略之前,仅暴露一个轻量的6自由度位姿接口。EgoRoC从腕部安装的(第一人称)相机建立任务无关的视角一致性,随后在对齐与操作之间交替进行,同时一个基于扩散的在线手眼模块在末端执行器坐标系下修正动作,从而实现硬件无关的部署。EgoRoC仅由静态腕部-目标图像对及其相对位姿一次性训练得到,而无需完整操作轨迹,且保持下游VLA不变。通过将第一人称对齐转化为一种可复用能力,EgoRoC减少了训练冗余、增强了零样本跨场景迁移,并可在无需人工标定的情况下跨VLA骨干网络扩展。在仿真与真实环境中,附加EgoRoC均能稳定提升成功率,尤其是在长时程与分布外任务上,并改善微调过程中的数据效率。
英文原文摘要 1486 字符
Recent Vision-Language-Action (VLA) models map visual-textual inputs to robotic actions via end-to-end architectures, yet this approach entangles visual understanding with task-specific actions. This leads to an exhaustive collection of full operational sequences and parameter redundancy across tasks, while generic third-person camera setups require fine-tuning for different hardware due to implicit hand-eye assumptions. We argue that decoupling how robots see from how robots act is a missing primitive in VLA systems. We present EgoRoC, a plug-and-play egocentric alignment head that precedes any task policy and exposes only a thin 6-DoF pose interface. EgoRoC establishes task-agnostic viewpoint consistency from a wrist-mounted (first-person) camera and then alternates alignment with manipulation, while a diffusion-based online hand-eye module corrects the action in the end-effector frame for hardware-agnostic deployment. Trained once from static wrist-target image pairs with relative poses, rather than full manipulation trajectories, EgoRoC leaves downstream VLAs unchanged. By turning egocentric alignment into a reusable capability, EgoRoC reduces training redundancy, strengthens zero-shot cross-scene transfer, and scales across VLA backbones without manual calibration. Across simulation and real settings, attaching EgoRoC consistently boosts success rates, especially on long-horizon and out-of-distribution tasks, and improves data efficiency during fine-tuning.