从手册到动作：面向思维链手册生成与机器人操作统一视觉-语言-动作模型
From Manuals to Actions: A Unified VLA Model for Chain-of-Thought Manual Generation and Robotic Manipulation
把多模态手册（图像+视觉提示+文本）作为中间表示，经 ManualCoT 注入动作专家，为长程操作任务提供显式控制条件与隐式潜变量引导；其基于 3D 高斯泼溅的数字孪生数据自动生成管线，可用于低成本扩充机器人操作训练数据。
原文依据 we first equip ManualVLA with a planning expert that generates intermediate manuals consisting of images, visual prompts, and textual instructions
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉-语言-动作（VLA）模型近来兴起，在机器人场景理解与操作中展现出较强的泛化能力。然而，面对需要明确目标状态的长程任务（如乐高装配或物体重排），现有 VLA 模型在协调长程规划与精确操作方面仍面临挑战。因此，我们希望赋予 VLA 模型从「what」结果推断「how」过程的能力，将目标状态转化为可执行的流程。本文提出 ManualVLA，一个基于混合 Transformer（MoT）架构的统一 VLA 框架，使多模态手册生成与动作执行之间能够协同一致。与以往直接将感知输入映射为动作的 VLA 模型不同，我们首先为 ManualVLA 配备一个规划专家，用于生成由图像、视觉提示和文本指令构成的中间手册。基于这些多模态手册，我们设计了手册思维链（ManualCoT）推理过程，将其送入动作专家，其中每个手册步骤提供显式控制条件，而其潜变量表示则为精确操作提供隐式引导。为减轻数据收集负担，我们开发了一套基于 3D 高斯泼溅的高保真数字孪生工具包，可自动生成用于规划专家训练的手册数据。ManualVLA 在真实世界中表现强劲，在乐高装配与物体重排任务上相较此前分层最先进基线平均成功率高出 32%。
英文原文摘要 1652 字符
Vision-Language-Action (VLA) models have recently emerged, demonstrating strong generalization in robotic scene understanding and manipulation. However, when confronted with long-horizon tasks that require defined goal states, such as LEGO assembly or object rearrangement, existing VLA models still face challenges in coordinating long-horizon planning with precise manipulation.Therefore, we aim to endow a VLA model with the capability to infer the "how" process from the "what" outcomes, transforming goal states into executable procedures. In this paper, we introduce ManualVLA, a unified VLA framework built upon a Mixture-of-Transformers (MoT) architecture, enabling coherent collaboration between multimodal manual generation and action execution. Unlike prior VLA models that directly map sensory inputs to actions, we first equip ManualVLA with a planning expert that generates intermediate manuals consisting of images, visual prompts, and textual instructions. Building upon these multimodal manuals, we design a Manual Chain-of-Thought (ManualCoT) reasoning process that feeds them into the action expert, where each manual step provides explicit control conditions, while its latent representation offers implicit guidance for accurate manipulation. To alleviate the burden of data collection, we develop a high-fidelity digital-twin toolkit based on 3D Gaussian Splatting, which automatically generates manual data for planning expert training. ManualVLA demonstrates strong real-world performance, achieving an average success rate 32% higher than the previous hierarchical SOTA baseline on LEGO assembly and object rearrangement tasks.