Video2Robo：基于 3DGS 的单视频合成数据实现可扩展机器人学习
Video2Robo: 3DGS-based Synthetic Data from One Video Enables Scalable Robot Learning
利用 3DGS 作为可编辑场景表示，从单段人类演示视频中重建场景并显式编辑物体布局、背景、光照和相机视角，生成多样化的机器人操作训练数据，为视觉研究者提供了一种将 3D 重建与数据生成结合的可复用范式。
原文依据 Video2Robo leverages 3D Gaussian Splatting (3DGS) as a powerful scene representation, enabling high-fidelity rendering and explicit 3D scene editing.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论可扩展的机器人学习受到获取多样化、高质量具身数据的高成本的制约。现有的数据生成方法部分缓解了这一问题，但通常依赖难以获取的硬件和劳动密集型的人工操作，且对多样化的场景配置泛化能力有限。为克服这些限制，我们提出了 Video2Robo，一个直接从单个人类演示视频生成高质量、多样化机器人数据的框架，能够无缝部署到物理机器人上。Video2Robo 的核心在于利用 3D 高斯泼溅（3DGS）作为强大的场景表示，实现高保真渲染和显式的三维场景编辑。该框架从原始视频中跟踪任务相关物体的时间一致运动轨迹，并识别关键任务技能，引导机器人在新颖的物体布局下以运动学合理的方式执行任务。此外，通过增强背景、纹理、光照和相机视角，Video2Robo 进一步提升了生成数据的多样性。在仿真和真实环境中的大量评估表明，使用 Video2Robo 数据训练的策略取得了更优的泛化与迁移性能。项目网页：Video2Robo。
英文原文摘要 1271 字符
Scalable robot learning is hindered by the high cost of acquiring diverse, high-quality embodied data. Existing data generation approaches partially mitigate this issue but typically depend on hard-to-access hardware and labor-intensive manual effort, with limited generalization to diverse scene configurations. To overcome these limitations, we propose Video2Robo, a framework that generates high-quality and diverse robot data directly from a single human demonstration video, enabling seamless deployment on physical robots. At its core, Video2Robo leverages 3D Gaussian Splatting (3DGS) as a powerful scene representation, enabling high-fidelity rendering and explicit 3D scene editing. The framework tracks temporally consistent motion trajectories of task-relevant objects from raw video footage and identifies key task skills, guiding robots to execute tasks kinematically plausibly under novel object arrangements. Furthermore, by augmenting backgrounds, textures, lighting, and camera views, Video2Robo further enhances the diversity of generated data. Extensive evaluations in both simulation and real-world environments demonstrate that policies trained on Video2Robo data achieve superior generalization and transfer performance. Project webpage: Video2Robo.