强化学习与具身智能 · CVPR2026 ·2026.06.03

跨域演示到代码:基于神经符号反事实推理的方法

Cross-Domain Demo-to-Code via Neurosymbolic Counterfactual Reasoning

Jooyoung Kim、Wonje Choi、Younguk Song、Honguk Woo

与你有关的部分

把视频演示抽象为符号轨迹、再用反事实状态搜索做可验证的程序修订,这一「感知→符号→可验证改写」的管线可迁移到视觉演示学习与跨域操作策略生成中,为 VLM 驱动的机器人编程提供可验证的中间表示。

原文依据 NeSyCR abstracts video demonstrations into symbolic trajectories that capture the underlying task procedure.
可迁移到 视频演示学习机器人操作跨域自适应代码策略生成

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

视觉语言模型(VLM)的最新进展使视频指令式机器人编程成为可能,智能体能够解读视频演示并生成可执行的控制代码。我们将视频指令式机器人编程建模为一个跨域自适应问题:演示环境与部署环境之间的感知与物理差异会引发程序层面的不匹配。然而,现有 VLM 缺乏必要的程序性理解能力,无法在此类域偏移下重新构建因果依赖关系并实现与任务兼容的行为。我们提出 NeSyCR,一个神经符号反事实推理框架,能够对任务程序进行可验证的适配,从而可靠地合成代码策略。NeSyCR 将视频演示抽象为符号轨迹,以刻画其背后的任务程序。给定部署观测,它推导出反事实状态,以揭示跨域不兼容之处。通过在符号状态空间中进行带有可验证检查的探索,NeSyCR 提出程序修订方案,使部署过程重新与演示程序保持兼容。相比最强基线 Statler,NeSyCR 的任务成功率提升 31.14%,在仿真与真实世界操作任务上均展现出稳健的跨域自适应能力。

英文原文摘要 1283 字符

Recent advances in Vision-Language Models (VLMs) have enabled video-instructed robotic programming, allowing agents to interpret video demonstrations and generate executable control code. We formulate video-instructed robotic programming as a cross-domain adaptation problem, where perceptual and physical differences between demonstration and deployment induce procedural mismatches. However, current VLMs lack the procedural understanding needed to reformulate causal dependencies and achieve task-compatible behavior under such domain shifts. We introduce NeSyCR, a neurosymbolic counterfactual reasoning framework that enables verifiable adaptation of task procedures, providing a reliable synthesis of code policies. NeSyCR abstracts video demonstrations into symbolic trajectories that capture the underlying task procedure. Given deployment observations, it derives counterfactual states that reveal cross-domain incompatibilities. By exploring the symbolic state space with verifiable checks, NeSyCR proposes procedural revisions that restore compatibility with the demonstrated procedure. NeSyCR achieves a 31.14% improvement in task success over the strongest baseline Statler, showing robust cross-domain adaptation across both simulated and real-world manipulation tasks.

标签

会议 CVPR2026
页码 18848-18858
Bibkey
来源 CVF Open Access