ORV：以4D占用为中心的机器人视频生成
ORV: 4D Occupancy-centric Robot Video Generation
把4D语义占用的2D渲染作为软引导注入视频扩散生成，并用Action-Expert AdaLN把分块7-DoF动作对齐到视频潜空间，为动作条件视频生成提供了一种「稀疏控制—稠密像素」之间的中间表示桥梁，可直接复用到机器人数据引擎与策略学习的合成数据生成。
原文依据 inject 2D renderings of 4D semantic occupancy into the generation process as soft guidance
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近期具身智能面临数据稀缺问题，而传统仿真器缺乏视觉真实感。可控视频生成正成为一种有前景的数据引擎，但当前的动作条件方法仍存在不足：生成的视频在保真度与时序一致性上受限，与控制的对应关系较差，且往往局限于单视角设定。我们将这些问题归因于稀疏控制输入与稠密像素输出之间的表示鸿沟。为此，我们提出ORV，一个以4D占用为中心的机器人视频生成框架，将动作先验与由占用导出的视觉先验相耦合。具体而言，我们通过Action-Expert AdaLN调制将分块的7自由度动作与视频潜变量对齐，并将4D语义占用的2D渲染作为软引导注入生成过程。同时，一个核心障碍是具身场景缺乏占用数据；因此我们构建了ORV-Data，一个面向机器人操作的大规模高质量4D语义占用数据集。在BridgeV2、DROID和RT-1上，ORV提升了视频生成质量与可控性，相比当前最优方法FVD降低18.8%，视觉规划成功率提升3.5%，策略学习成功率提升6.4%。除单视角生成外，ORV原生支持多视角一致性合成，并能在存在显著域差距的情况下实现仿真到现实的迁移。代码、模型与数据将在录用后公开。
英文原文摘要 1464 字符
Recent embodied intelligence suffers from data scarcity, while conventional simulators lack visual realism. Controllable video generation is emerging as a promising data engine, yet current action-conditioned methods still fall short: generated videos are limited in fidelity and temporal consistency, poorly aligned with controls, and often constrained to singleview settings. We attribute these issues to the representational gap between sparse control inputs and dense pixel outputs. Thus, we introduce ORV, a 4D occupancy-centric framework for robot video generation that couples action priors with occupancy-derived visual priors. Concretely, we align chunked 7-DoF actions with video latents via an Action-Expert AdaLN modulation, and inject 2D renderings of 4D semantic occupancy into the generation process as soft guidance. Meanwhile, a central obstacle is the lack of occupancy data for embodied scenarios; we therefore curate ORV-Data, a large-scale, high-quality 4D semantic occupancy dataset of robot manipulation. Across BridgeV2, DROID, and RT-1, ORV improves video generation quality and controllability, achieving 18.8% lower FVD than state of the art, +3.5% success rate on visual planning, and +6.4% success rate on policy learning. Beyond singleview generation, ORV natively supports multiview consistent synthesis and enables simulation-to-real transfer despite significant domain gaps. Code, models, and data will be released upon acceptance.