探索扩散模型在机器人控制中的条件设计
Exploring Conditions for Diffusion Models in Robotic Control
将预训练文生图扩散模型冻结后作为任务自适应的视觉表征提取器，用可学习任务提示与帧级视觉提示替代纯文本条件，为视觉研究者提供了一种无需微调即可把生成模型迁移到下游控制任务的条件设计思路。
原文依据 we explore leveraging pre-trained text-to-image diffusion models to obtain task-adaptive visual representations for robotic control, without fine-tuning the model itself
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论尽管预训练视觉表征已显著推动了模仿学习的发展，但它们通常是任务无关的，因为在策略学习过程中这些表征保持冻结。在本工作中，我们探索利用预训练的文本到图像扩散模型，在不微调模型本身的情况下，为机器人控制获取任务自适应的视觉表征。然而我们发现，朴素地施加文本条件——这一在其他视觉领域取得成功的策略——在控制任务中带来的收益微乎其微，甚至为负。我们将此归因于扩散模型训练数据与机器人控制环境之间的域差距，并由此主张条件应当考虑控制所需的特定、动态的视觉信息。为此，我们提出 ORCA，引入可学习且能适应控制环境的任务提示，以及能够捕捉细粒度、逐帧细节的视觉提示。通过我们新设计的条件来促进任务自适应表征，我们的方法在多个机器人控制基准上取得了最优性能，显著超越了先前方法。
英文原文摘要 1130 字符
While pre-trained visual representations have significantly advanced imitation learning, they are often task-agnostic as they remain frozen during policy learning. In this work, we explore leveraging pre-trained text-to-image diffusion models to obtain task-adaptive visual representations for robotic control, without fine-tuning the model itself. However, we find that naively applying textual conditions--a successful strategy in other vision domains--yields minimal or even negative gains in control tasks. We attribute this to the domain gap between the diffusion model's training data and robotic control environments, leading us to argue for conditions that consider the specific, dynamic visual information required for control. To this end, we propose ORCA, which introduces learnable task prompts that adapt to the control environment and visual prompts that capture fine-grained, frame-specific details. Through facilitating task-adaptive representations with our newly devised conditions, our approach achieves state-of-the-art performance on various robotic control benchmarks, significantly surpassing prior methods.