VCP-Attack:面向大型视觉语言模型的可迁移黑盒目标攻击的视觉对比投影
VCP-Attack: Visual-Contrastive Projection for Transferable Black-Box Targeted Attacks on Large Vision-Language Models
把动态 PCA 子空间投影与多样本对比损失结合,用于约束对抗扰动并使其对齐目标语义,这套「低维语义子空间 + 对比监督」的扰动优化思路可直接迁移到视觉编码器的对抗样本生成与特征空间分析中。
原文依据 we employ a dynamic PCA-based projection to constrain perturbations within semantically meaningful low-dimensional subspaces
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论大型视觉语言模型(LVLM)在多种多模态任务上取得了令人瞩目的性能,但仍易受目标对抗攻击影响,尤其是在黑盒设定下。本文提出 VCP-Attack,一种可迁移的目标攻击框架,将结构化对比监督与子空间引导的扰动优化相结合。具体而言,我们采用基于 PCA 的动态投影,将扰动约束在具有语义意义的低维子空间内,并设计多样本对比损失,使对抗特征与目标语义对齐,同时将其推离源语义。在七个开源和三个专有 LVLM(包括 GPT-4o、Claude 和 Gemini)上的大量实验表明,VCP-Attack 在黑盒目标攻击中取得了最优性能。在固定扰动预算(epsilon = 16/255)下,本方法在开源模型上取得 94.2% 的平均攻击成功率(ASR),在专有模型上取得 83.1%,分别超过最强基线 23.3% 和 16.8%。值得注意的是,VCP-Attack 在 GPT-4o 上取得了 95.6% 的 ASR。全面的消融实验与可视化进一步验证了动态子空间投影与语义对比监督的有效性。尽管评估是在图像描述生成任务上进行的,本方法与模型无关,并在视觉语言任务中更广泛的黑盒对抗设定下展现出强大潜力。
英文原文摘要 1501 字符
Large vision-language models (LVLMs) have achieved impressive performance across a variety of multimodal tasks, yet remain vulnerable to targeted adversarial attacks, particularly in black-box settings. In this paper, we propose VCP-Attack, a transferable targeted attack framework that combines structured contrastive supervision with subspace-guided perturbation optimization. Specifically, we employ a dynamic PCA-based projection to constrain perturbations within semantically meaningful low-dimensional subspaces, and design a multi-sample contrastive loss to align adversarial features with target semantics while pushing them away from the source semantics. Extensive experiments on seven open-source and three proprietary LVLMs--including GPT-4o, Claude, and Gemini--show that VCP-Attack achieves state-of-the-art performance in black-box targeted attacks. Under a fixed perturbation budget (epsilon = 16/255), our method achieves an average attack success rate (ASR) of 94.2% on open-source models and 83.1% on proprietary models, surpassing the strongest baselines by 23.3% and 16.8%, respectively. Notably, VCP-Attack achieves a 95.6% ASR on GPT-4o. Comprehensive ablation studies and visualizations further validate the effectiveness of the dynamic subspace projection and semantic contrastive supervision. While evaluated on image captioning, our approach is model-agnostic and exhibits strong potential for broader applications to black-box adversarial settings in vision-language tasks.