面向视觉无监督强化学习的显著性引导表征与一致性策略学习
Saliency-Guided Representation with Consistency Policy Learning for Visual Unsupervised Reinforcement Learning
对做视觉表征与决策结合的研究者，这篇论文给出一个可复用的思路：用显著性引导的动力学预测任务把表征学习与后继度量训练解耦，使视觉URL中的表征聚焦于动力学相关区域；同时把一致性模型（快速采样）与无分类器引导引入技能条件策略，用于建模多模态动作分布，可直接迁移到视觉条件策略学习与技能可控生成。
原文依据 SRCP decouples representation learning from successor training by introducing a saliency-guided dynamics task to capture dynamics-relevant representations
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论零样本无监督强化学习（URL）为构建无需额外监督即可泛化到未见任务的通用智能体提供了有前景的方向。在现有方法中，后继表征（SR）因在结构化、低维设定下的有效性而成为主流范式。然而，SR 方法难以扩展到高维视觉环境。通过实证分析，我们识别出 SR 在视觉 URL 中的两个关键局限：（1）SR 目标往往导致次优表征，使表征关注与动力学无关的区域，从而产生不准确的后继度量并降低任务泛化能力；（2）这些有缺陷的表征阻碍 SR 策略建模多模态的技能条件动作分布并保证技能可控性。为解决这些局限，我们提出显著性引导表征与一致性策略学习（SRCP），这是一个提升 SR 方法在视觉 URL 中零样本泛化能力的新框架。SRCP 通过引入显著性引导的动力学任务来捕获与动力学相关的表征，从而将表征学习与后继训练解耦，进而改进后继度量与任务泛化。此外，它集成了快速采样的一致性策略、面向 URL 的无分类器引导以及定制的训练目标，以改进技能条件策略建模与可控性。在 ExORL 基准的 4 个数据集共 16 个任务上的大量实验表明，SRCP 在视觉 URL 中取得了最优的零样本泛化性能，并且与多种 SR 方法兼容。
英文原文摘要 1667 字符
Zero-shot unsupervised reinforcement learning (URL) offers a promising direction for building generalist agents capable of generalizing to unseen tasks without additional supervision. Among existing approaches, successor representations (SR) have emerged as a prominent paradigm due to their effectiveness in structured, low-dimensional settings. However, SR methods struggle to scale to high-dimensional visual environments. Through empirical analysis, we identify two key limitations of SR in visual URL: (1) SR objectives often lead to suboptimal representations that attend to dynamics-irrelevant regions, resulting in inaccurate successor measures and degraded task generalization; and (2) these flawed representations hinder SR policies from modeling multi-modal skill-conditioned action distributions and ensuring skill controllability. To address these limitations, we propose Saliency-Guided Representation with Consistency Policy Learning (SRCP), a novel framework that improves zero-shot generalization of SR methods in visual URL. SRCP decouples representation learning from successor training by introducing a saliency-guided dynamics task to capture dynamics-relevant representations, thereby improving successor measure and task generalization. Moreover, it integrates a fast-sampling consistency policy with URL-specific classifier-free guidance and tailored training objectives to improve skill-conditioned policy modeling and controllability. Extensive experiments on 16 tasks across 4 datasets from the ExORL benchmark demonstrate that SRCP achieves state-of-the-art zero-shot generalization in visual URL and is compatible with various SR methods.