DiffuView：面向3D感知机器人操作的多视图扩散预训练
DiffuView: Multi-View Diffusion Pretraining for 3D Aware Robotic Manipulation
将多视图扩散模型的条件生成能力作为视觉骨干网络，通过隐式恢复场景几何来获得对视角变化鲁棒的3D一致表征，可直接用于提升视觉模仿学习在视角偏移下的泛化性。
原文依据 DiffuView models the conditional generation of target views given source observations within a diffusion framework, enabling the network to implicitly recover scene geometry and enforce view consistency.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论基于视觉观测的机器人操作仍然具有挑战性，原因是缺乏能够在不同视角和传感器配置下泛化的3D一致表征。现有方法通常依赖掩码自编码器或神经场景表示，它们无法捕捉跨视图对应关系。关键的是，尽管多视图扩散模型最近在3D感知生成合成方面取得了巨大成功，但其强大的表征为实现视角鲁棒的视觉运动控制提供了有前景的方向。本文介绍了DiffuView，这是一个新颖的框架，通过多视图扩散预训练学习统一的3D感知表征，并将其部署用于模仿学习。具体而言，DiffuView在扩散框架内对给定源观测条件下目标视图的条件生成进行建模，使网络能够隐式恢复场景几何并强制视图一致性。然后，预训练的扩散网络被用作动作策略的强大视觉骨干网络，从而在不同视角和视觉条件下实现鲁棒控制。我们在两个具有挑战性的基准MetaWorld和Libero上评估了DiffuView。在仿真和真实世界场景中的大量实验表明，DiffuView实现了卓越的泛化能力，与现有方法相比，在视角偏移下的成功率提高了近20%。
英文原文摘要 1422 字符
Robotic manipulation from visual observations remains challenging due to the lack of 3D consistent representations that can generalize across diverse viewpoints and sensor configurations. Existing approaches often rely on masked autoencoders or neural scene representations, which fail to capture cross view correspondences. Crucially, while multi-view diffusion models have recently shown tremendous success in 3D aware generative synthesis, their powerful representations offer a promising direction for achieving viewpoint robust visuomotor control. In this paper, we introduce DiffuView, a novel framework that learns unified 3D aware representations through multi-view diffusion pretraining and deploys them for imitation learning. Specifically, DiffuView models the conditional generation of target views given source observations within a diffusion framework, enabling the network to implicitly recover scene geometry and enforce view consistency. The pretrained diffusion network is then utilized as a powerful visual backbone for an action policy, allowing robust control under varying viewpoints and visual conditions. We evaluate DiffuView on two challenging benchmarks, MetaWorld and Libero. Extensive experiments in both simulation and realworld scenarios demonstrate that DiffuView achieves superior generalization, improving success rates under viewpoint shifts by nearly 20% compared with existing methods.