强化学习与具身智能 · CVPR2026 ·2026.06.03

像素运动扩散正是机器人控制所需

Pixel Motion Diffusion is What We Need for Robot Control

E-Ro Nguyen、Yichi Zhang、Kanchana Ranasinghe、Xiang Li、Michael S. Ryoo

与你有关的部分

把像素运动作为高层意图与低层动作之间的可解释中间表示,并用扩散模型同时建模两层控制器;做视觉-语言-动作(VLA)或机器人操作的研究者可以借鉴这种「运动中心」的中间抽象来缓解仿真到现实的域差,并复用其仅需少量微调即可真机迁移的流程。

原文依据 bridges high-level motion intent and low-level robot action via structured pixel motion representation
可迁移到 language-conditioned manipulationvision-language-actionsim-to-real transfer

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

我们提出 DAWN(Diffusion is All We Need for robot control),一个面向语言条件机器人操作任务的统一扩散框架,它通过结构化的像素运动表示,将高层运动意图与低层机器人动作衔接起来。在 DAWN 中,高层与低层控制器均被建模为扩散过程,从而构成一个完全可训练、端到端且具有可解释中间运动抽象的系统。DAWN 在具有挑战性的 CALVIN 基准上取得了最优结果,展现出强大的多任务性能,并在 MetaWorld 上进一步验证了其有效性。尽管仿真与真实之间存在显著的域差且真实数据有限,我们仅通过极少量微调即实现了可靠的真实世界迁移,说明基于扩散的运动抽象在机器人控制中具有实际可行性。结果表明,将扩散建模与以运动为中心的表示相结合,可作为可扩展且鲁棒的机器人学习的一个强基线。

英文原文摘要 1047 字符

We present DAWN (Diffusion is All We Need for robot control), a unified diffusion-based framework for language-conditioned robotic manipulation that bridges high-level motion intent and low-level robot action via structured pixel motion representation. In DAWN, both the high-level and low-level controllers are modeled as diffusion processes, yielding a fully trainable, end-to-end system with interpretable intermediate motion abstractions. DAWN achieves state-of-the-art results on the challenging CALVIN benchmark, demonstrating strong multi-task performance, and further validates its effectiveness on MetaWorld. Despite the substantial domain gap between simulation and reality and limited real-world data, we demonstrate reliable real-world transfer with only minimal finetuning, illustrating the practical viability of diffusion-based motion abstractions for robotic control. Our results show the effectiveness of combining diffusion modeling with motion-centric representations as a strong baseline for scalable and robust robot learning.

标签

会议 CVPR2026
页码 23663-23672
Bibkey
来源 CVF Open Access