SEBA：面向视觉强化学习的高样本效率黑盒攻击
SEBA: Sample-Efficient Black-Box Attacks on Visual Reinforcement Learning
对做视觉 RL 或机器人策略的研究者，这篇给出了一个只用少量环境交互就能攻击图像输入连续控制策略的黑盒框架：用世界模型替代真实环境查询、用影子 Q 模型估计对抗回报、用 GAN 生成不可感知扰动，可直接用于评估视觉策略的对抗脆弱性。
原文依据 a world model that simulates environment dynamics to reduce real-world queries
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉强化学习在视觉控制与机器人领域取得了显著进展，但其对对抗扰动的脆弱性仍未得到充分研究。现有黑盒攻击大多针对基于向量或离散动作的强化学习，受限于庞大的动作空间与过多的环境查询，这些方法在基于图像的连续控制上效果有限。我们提出 SEBA，一个面向视觉强化学习智能体的高样本效率黑盒对抗攻击框架。SEBA 集成了一个影子 Q 模型，用于估计对抗条件下的累积奖励；一个生成对抗网络，用于生成视觉上不可察觉的扰动；以及一个世界模型，用于模拟环境动力学以减少真实环境查询。通过在学习影子模型与优化生成器之间交替进行的两阶段迭代训练流程，SEBA 在保持高效的同时取得了强攻击性能。在 MuJoCo 与 Atari 基准上的实验表明，与先前的黑盒和白盒方法相比，SEBA 显著降低了累积奖励、保持了视觉保真度，并大幅减少了环境交互次数。代码已开源于 https://github.com/tairanhuang/seba。
英文原文摘要 1233 字符
Visual reinforcement learning has achieved remarkable progress in visual control and robotics, but its vulnerability to adversarial perturbations remains underexplored. Most existing black-box attacks focus on vector-based or discrete-action RL, and their effectiveness on image-based continuous control is limited by the large action space and excessive environment queries. We propose SEBA, a sample-efficient framework for black-box adversarial attacks on visual RL agents. SEBA integrates a shadow Q model that estimates cumulative rewards under adversarial conditions, a generative adversarial network that produces visually imperceptible perturbations, and a world model that simulates environment dynamics to reduce real-world queries. Through a two-stage iterative training procedure that alternates between learning the shadow model and refining the generator, SEBA achieves strong attack performance while maintaining efficiency. Experiments on MuJoCo and Atari benchmarks show that SEBA significantly reduces cumulative rewards, preserves visual fidelity, and greatly decreases environment interactions compared to prior black-box and white-box methods. The code is available at https://github.com/tairanhuang/seba online.