联合对齐潜动作：面向野外可扩展视觉-语言-动作预训练
Joint-Aligned Latent Action: Towards Scalable VLA Pretraining in the Wild
JALA 提出一种与逆动力学和真实动作联合对齐的预测性动作嵌入，绕开完整视觉动态重建，为从无标注人类视频中学习可迁移的潜动作表征提供了可复用方案，可直接用于 VLA 预训练的数据扩展。
原文依据 JALA bypasses full visual dynamic reconstruction, instead learns a predictive action embedding aligned with both inverse dynamics and real actions.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论尽管取得了进展，视觉-语言-动作模型（VLA）仍受限于大规模、多样化机器人数据的稀缺。虽然人类操作视频提供了丰富的替代来源，但现有方法不得不在小型精确标注数据集与海量野外视频（其手部跟踪标签不可靠）之间做出选择。我们提出 JALA，一个学习联合对齐潜动作的预训练框架。JALA 绕开完整的视觉动态重建，转而学习一种与逆动力学和真实动作同时对齐的预测性动作嵌入。这产生了一个具有转移感知、以行为为中心的潜空间，用于从异构人类数据中学习。我们通过 UniHand-Mix 扩展该方法，这是一个 750 万视频（超过 2000 小时）的语料库，混合了实验室与野外视频。实验表明，JALA 在受控与无约束场景下均能生成更真实的手部动作，并显著提升仿真与真实世界任务中的下游机器人操作性能。这些结果表明，联合对齐潜动作提供了一条从人类数据中进行可扩展 VLA 预训练的路径。
英文原文摘要 1129 字符
Despite progress, Vision-Language-Action models (VLAs) are limited by a scarcity of large-scale, diverse robot data. While human manipulation videos offer a rich alternative, existing methods are forced to choose between small, precisely-labeled datasets and vast in-the-wild footage with unreliable hand tracking labels. We present JALA, a pretraining framework that learns Jointly-Aligned Latent Actions. JALA bypasses full visual dynamic reconstruction, instead learns a predictive action embedding aligned with both inverse dynamics and real actions. This yields a transition-aware, behavior-centric latent space for learning from heterogeneous human data. We scale this approach with UniHand-Mix, a 7.5M video corpus (>2,000 hours) blending laboratory and in-the-wild footage. Experiments demonstrate that JALA generates more realistic hand motions in both controlled and unconstrained scenarios, significantly improving downstream robot manipulation performance in both simulation and real-world tasks. These results indicate that jointly-aligned latent actions offer a scalable pathway for VLA pretraining from human data.