EmbodMocap：面向具身智能体的野外四维人体-场景重建
EmbodMocap: In-the-Wild 4D Human-Scene Reconstruction for Embodied Agents
用两台移动 iPhone 的 RGB-D 序列做联合标定，在统一度量世界坐标系下同时重建人体与场景，为单目人体-场景重建提供无需静态相机与标记的野外采集方案与训练数据。
原文依据 jointly calibrate dual RGB-D sequences to reconstruct both humans and scenes within a unified metric world coordinate frame
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论真实世界中的人类行为天然编码了丰富的长期上下文信息，可用于训练具身智能体进行感知、理解与行动。然而，现有的采集系统通常依赖昂贵的影棚装置与可穿戴设备，限制了野外场景条件下人体运动数据的大规模采集。为此，我们提出 EmbodMocap，一套使用两台移动 iPhone 的便携且低成本的数据采集流程。其核心思想是联合标定双路 RGB-D 序列，在统一的度量世界坐标系中同时重建人体与场景。所提方法无需静态相机或标记，即可在日常环境中实现度量尺度且场景一致的采集，将人体运动与场景几何无缝衔接。基于所采集的数据，我们支撑了三项具身智能任务：单目人体-场景重建，即微调前馈模型以输出度量尺度、与世界空间对齐的人体与场景；基于物理的角色动画，我们证明该数据可用于扩展人-物交互技能与场景感知的运动跟踪；以及机器人运动控制，即通过仿真到现实的强化学习训练人形机器人复现视频中呈现的人体运动。实验结果验证了该流程的有效性及其对推进具身智能研究的贡献。
英文原文摘要 1407 字符
Human behaviors in the real world naturally encode rich, long-term contextual information that can be leveraged to train embodied agents for perception, understanding, and acting.However, existing capture systems typically rely on costly studio setups and wearable devices, limiting the large-scale collection of scene-conditioned human motion data in the wild.To address this, we propose EmbodMocap, a portable and affordable data collection pipeline using two moving iPhones. Our key idea is to jointly calibrate dual RGB-D sequences to reconstruct both humans and scenes within a unified metric world coordinate frame.The proposed method allows metric-scale and scene-consistent capture in everyday environments without static cameras or markers, bridging human motion and scene geometry seamlessly.Based on the collected data, we empower three embodied AI tasks: monocular human-scene-reconstruction, where we fine-tune on feedforward models that output metric-scale, world-space aligned humans and scenes; physics-based character animation, where we prove our data could be used to scale human-object interaction skills and scene-aware motion tracking; and robot motion control, where we train a humanoid robot via sim-to-real RL to replicate human motions depicted in videos. Experimental results validate the effectiveness of our pipeline and its contributions towards advancing embodied AI research.