通过互补持续评论家解决强化学习中的稳定性-可塑性困境
Resolving the Stability-Plasticity Dilemma in Reinforcement Learning via Complementary Continual Critics
对做视觉RL的人，可复用其「双评论家+交叉注意力融合」架构：用CBP维持表征可塑性、用EWC维持稳定性，并按观测特征动态加权两路价值估计，直接缓解非平稳视觉输入下的表征漂移与策略退化。
原文依据 design a cross-attention based fusion mechanism that balances the value estimates from the dual critics according to observation characteristics
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论本文提出面向视觉强化学习的交叉注意力持续双评论家（CD-CCA）框架，以解决可塑性与稳定性之间的冲突。该方法将持续学习技术引入视觉强化学习架构，利用持续反向传播（CBP）与弹性权重巩固（EWC）构建两个互补的评论家——其一用于维持表征可塑性以实现对环境的快速适应，其二用于保持知识稳定性以防止灾难性遗忘。此外，本文设计了基于交叉注意力的融合机制，依据观测特征对双评论家的价值估计进行平衡。在 DeepMind Control 与 CARLA 基准上的实验结果表明，CD-CCA 有效缓解了表征漂移与策略退化问题。与现有视觉强化学习方法相比，该方法在非平稳环境与长时程决策任务中展现出更强的鲁棒性与适应性，为持续强化学习的发展提供了一种新的架构范式。
英文原文摘要 1120 字符
This paper proposes the Continual Dual-Critic with Cross-Attention (CD-CCA) framework for visual reinforcement learning to address the plasticity-stability conflict. Our method introduces continual learning techniques into the visual RL architecture, constructing two complementary critics using Continual Backpropagation (CBP) and Elastic Weight Consolidation (EWC) -- one for maintaining representational plasticity for rapid environmental adaptation, and the other for preserving knowledge stability to prevent catastrophic forgetting. Furthermore, we design a cross-attention based fusion mechanism that balances the value estimates from the dual critics according to observation characteristics. Experimental results on DeepMind Control and CARLA benchmarks show that CD-CCA effective mitigates issues of representation drift and policy degradation. Compared to existing visual RL methods, our approach exhibits enhanced robustness and adaptability in non-stationary environments and long-horizon decision-making tasks, providing a new architectural paradigm for the advancement of continual reinforcement learning.