鲁棒性与安全 · CVPR2026 ·2026.06.03

当机器人服从补丁:针对视觉-语言-动作模型的通用可迁移补丁攻击

When Robots Obey the Patch: Universal Transferable Patch Attacks on Vision-Language-Action Models

Hui Lu、Yi Yu、Yiming Yang、Chenyu Yi、Qixin Zhang、Bingquan Shen 等 8 人

与你有关的部分

提出一种在共享特征空间中学习通用对抗补丁的框架,结合特征偏移、注意力主导和语义错配损失,可迁移到未知架构的VLA模型,为视觉-语言-动作系统的安全性评估和防御提供了可复用的攻击基线。

原文依据 UPA-RFAS combines (i) a feature-space objective with an l_1 deviation prior and repulsive InfoNCE loss to induce transferable representation shifts
可迁移到 对抗攻击视觉语言动作模型机器人操作

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

视觉-语言-动作(VLA)模型易受对抗攻击,但通用且可迁移的攻击仍未得到充分探索,因为现有补丁大多过拟合于单一模型,在黑盒设置下失效。为填补这一空白,我们系统研究了针对未知架构、微调变体及仿真到现实偏移下VLA驱动机器人的通用可迁移对抗补丁。我们提出UPA-RFAS(通过鲁棒特征、注意力和语义的通用补丁攻击),一个统一框架,在共享特征空间中学习单一物理补丁,同时促进跨模型迁移。UPA-RFAS结合了:(i)带有l_1偏差先验和排斥InfoNCE损失的特征空间目标,以诱导可迁移的表征偏移;(ii)鲁棒性增强的两阶段最小-最大过程,其中内循环学习不可见的样本级扰动,外循环针对该硬化邻域优化通用补丁;(iii)两个VLA特定损失:补丁注意力主导以劫持文本到视觉的注意力,以及补丁语义错配以在无标签情况下诱导图像-文本不匹配。跨多种VLA模型、操作套件和物理执行的实验表明,UPA-RFAS在模型、任务和视角间持续迁移,暴露了实用的基于补丁的攻击面,并为未来防御建立了强基线。

英文原文摘要 1422 字符

Vision-Language-Action (VLA) models are vulnerable to adversarial attacks, yet universal and transferable attacks remain underexplored, as most existing patches overfit to a single model and fail in black-box settings. To address this gap, we present a systematic study of universal, transferable adversarial patches against VLA-driven robots under unknown architectures, finetuned variants, and sim-to-real shifts. We introduce UPA-RFAS (Universal Patch Attack via Robust Feature, Attention, and Semantics), a unified framework that learns a single physical patch in a shared feature space while promoting cross-model transfer. UPA-RFAS combines (i) a feature-space objective with an l_1 deviation prior and repulsive InfoNCE loss to induce transferable representation shifts, (ii) a robustness-augmented two-phase min-max procedure where an inner loop learns invisible sample-wise perturbations and an outer loop optimizes the universal patch against this hardened neighborhood, and (iii) two VLA-specific losses: Patch Attention Dominance to hijack text to vision attention and Patch Semantic Misalignment to induce image-text mismatch without labels. Experiments across diverse VLA models, manipulation suites, and physical executions show that UPA-RFAS consistently transfers across models, tasks, and viewpoints, exposing a practical patch-based attack surface and establishing a strong baseline for future defenses.

标签

会议 CVPR2026
页码 22867-22878
Bibkey
来源 CVF Open Access