强化学习与具身智能 · CVPR2026 ·2026.06.03

DynBridge:通过交互动力学桥接想象与控制以实现机器人操作

DynBridge: Bridging Imagination and Control through Interaction Dynamics for Robot Manipulation

Alex Wang、Zhiwei Dong、Qicheng Bai、Chenshi Zhang、Yujie Yi、Guang Dai 等 8 人

与你有关的部分

把「动作—环境」的双向耦合显式建模为交互动力学表示,并用它桥接未来视觉想象与可执行动作预测;对做视觉-动作联合建模、视觉预测驱动策略的研究者,这是一个可复用的共享动力学表征与动作条件聚合思路。

原文依据 We formalize the coupling as interaction dynamics, which specify where environmental changes occur and how actions cause them.
可迁移到 机器人操作视觉预测动作条件视频生成策略学习

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

近期的生成模型使机器人能够生成未来的视觉结果以指导动作,然而大多数方法仍将想象与控制分开处理,导致生成的推演在视觉上连贯但在物理上不一致。尽管结构先验增强了空间定位能力,这些方法仍由视觉相关性驱动而非因果驱动,忽视了机器人动作与演化环境之间的双向耦合。我们将这种耦合形式化为交互动力学,它指明了环境变化发生在何处以及动作如何导致这些变化。基于这一形式化,我们提出 DynBridge,一个通过共享动力学表示统一想象与控制的端到端框架。具体而言,DynBridge 通过三个组件实现这一点:(1) 交互动力学生成器,通过联合轨迹生成与动作预测来预测交互动力学;(2) 动作条件动力学聚合器,在控制信号下整合动力学;(3) 动力学引导动作预测器,利用聚合后的动力学产生可执行、上下文感知的动作。结果表明,DynBridge 在无需外部预训练的情况下,在仿真与真实世界基准上持续优于先前方法。

英文原文摘要 1293 字符

Recent generative models allow robots to generate future visual outcomes for action guidance, yet most still address imagination and control independently, resulting in visually coherent rollouts but physically inconsistent behaviors. While structural priors enhance spatial grounding, these methods remain visually correlation-driven rather than causally informed, overlooking the bidirectional coupling between robot actions and the evolving environment. We formalize the coupling as interaction dynamics, which specify where environmental changes occur and how actions cause them. Based on this formulation, we introduce DynBridge, an end-to-end framework that unifies imagination and control through the shared dynamics representation. Specifically, DynBridge realizes this via three components: (1) an Interaction Dynamics Generator that forecasts interaction dynamics via joint trajectory generation and action prediction; (2) an Action-Conditioned Dynamics Aggregator that integrates dynamics under control signals; and (3) a Dynamics-Guided Action Predictor that leverages the aggregated dynamics to produce executable, context-aware actions. Results demonstrate that DynBridge consistently outperforms prior methods on simulated and real-world benchmarks without external pretraining.

标签

会议 CVPR2026
页码 22423-22432
Bibkey
来源 CVF Open Access