鲁棒性与安全 · CVPR2026 ·2026.06.03

从变换到迁移:提升视觉语言预训练模型上的对抗攻击迁移性

Transform to Transfer: Boosting Adversarial Attack Transferability on Vision-Language Pre-training Models

Yang Li、Jia-Li Yin、Luojun Lin、Wei Lin

与你有关的部分

可学习的变换组合机制(自适应选择最优变换以最大化输入多样性)可直接迁移到任何黑盒迁移攻击流程中,替代手工固定的变换堆叠;积分梯度用于缓解对源模型的过拟合,也是提升跨模型迁移性的通用手段。

原文依据 a learnable transformation mechanism that adaptively selects optimal combinations of transformations to maximize input diversity
可迁移到 对抗攻击对抗鲁棒性评估视觉语言预训练模型

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

视觉语言预训练(VLP)模型虽在多模态任务上取得最优性能,却对多模态对抗样本表现出显著脆弱性。在 VLP 模型的黑盒攻击场景中,关键挑战在于这些对抗样本的迁移性有限。现有提升迁移性的方法往往过度依赖源模型,并依赖有限且固定的变换技术。为克服这些局限,我们提出一种新的「从变换到迁移攻击」(TTA)方法。该方法引入可学习的变换机制,自适应地选择最优变换组合以最大化输入多样性,并引入积分梯度以缓解对源模型的过度依赖,从而改进攻击优化过程。大量实验表明,TTA 在下游任务中取得出色的攻击性能,在不同 VLP 架构上均优于当前最优的攻击方法。

英文原文摘要 1059 字符

Vision-Language Pre-training (VLP) models, while achieving state-of-the-art performance on various multimodal tasks, exhibit significant vulnerability to multimodal adversarial examples. In black-box attack scenarios of VLP models, a key challenge lies in the limited transferability of these adversarial examples. Existing methods to enhance transferability often suffer from an excessive dependence on the source model and a reliance on limited and fixed transformation techniques. To overcome these limitations, we propose a novel Transform to Transfer Attack (TTA) method. Our approach introduces a learnable transformation mechanism that adaptively selects optimal combinations of transformations to maximize input diversity, and incorporates integrated gradients to mitigate over-reliance on the source model, thereby refining the attack optimization process. Extensive experiments demonstrate that TTA achieves outstanding attack performance in downstream tasks, outperforming current state-of-the-art attack methods across different VLP architectures.

标签

会议 CVPR2026
页码 30141-30150
Bibkey
来源 CVF Open Access