鲁棒性与安全 · CVPR2026 ·2026.06.03

FORCE:通过特征过度依赖校正实现可迁移的视觉越狱攻击

FORCE: Transferable Visual Jailbreaking Attacks via Feature Over-Reliance CorrEction

Runqi Lin、Alasdair Paren、Suqin Yuan、Muyang Li、Philip Torr、Adel Bibi 等 7 人

与你有关的部分

对做多模态大模型红队与安全评测的人,这篇给出了一个可复用的攻击优化思路:用损失景观平坦化(类似 sharpness-aware 优化)加频域语义重加权,把视觉对抗扰动从高曲率窄区域推向平坦可行域,从而提升跨模型迁移性;这套「平坦化+频域重标定」也可迁移到对抗样本迁移性、模型鲁棒性诊断等视觉安全问题。

原文依据 we analyse the loss landscape of these jailbreaking attacks and find that the generated attacks tend to reside in high-sharpness regions
可迁移到 多模态大模型安全评测对抗攻击迁移性红队测试

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

新模态的引入增强了多模态大语言模型(MLLMs)的能力,但也带来了额外的脆弱性。特别是,简单的视觉越狱攻击比复杂的文本攻击更容易操纵开源 MLLMs。然而,这些尚不成熟的攻击表现出极其有限的跨模型迁移性,无法可靠地识别闭源 MLLMs 中的脆弱性。在本工作中,我们分析了这些越狱攻击的损失景观,发现生成的攻击往往位于高锐度区域,其有效性对迁移过程中哪怕微小的参数变化都高度敏感。为进一步解释这种高锐度定位,我们分析了它们在中间层和频域中的特征表示,揭示出对狭窄层表示和语义贫乏的频率成分的不当依赖。基于此,我们提出了特征过度依赖校正(FORCE)方法,引导攻击在跨层特征上探索更广阔的可行区域,并根据频率特征的语义内容重新缩放其影响。通过消除对层特征和频谱特征不可泛化的依赖,我们的方法为视觉越狱攻击发现了平坦的可行区域,从而提升了跨模型迁移性。大量实验表明,我们的方法有效促进了对闭源 MLLMs 的视觉红队评估。

英文原文摘要 1535 字符

The integration of new modalities enhances the capabilities of multimodal large language models (MLLMs) but also introduces additional vulnerabilities. In particular, simple visual jailbreaking attacks can manipulate open-source MLLMs more readily than sophisticated textual attacks. However, these underdeveloped attacks exhibit extremely limited cross-model transferability, failing to reliably identify vulnerabilities in closed-source MLLMs. In this work, we analyse the loss landscape of these jailbreaking attacks and find that the generated attacks tend to reside in high-sharpness regions, whose effectiveness is highly sensitive to even minor parameter changes during transfer. To further explain the high-sharpness localisations, we analyse their feature representations in both the intermediate layers and the spectral domain, revealing an improper reliance on narrow layer representations and semantically poor frequency components. Building on this, we propose a Feature Over-Reliance CorrEction (FORCE) method, which guides the attack to explore broader feasible regions across layer features and rescales the influence of frequency features according to their semantic content. By eliminating non-generalizable reliance on both layer and spectral features, our method discovers flattened feasible regions for visual jailbreaking attacks, thereby improving cross-model transferability. Extensive experiments demonstrate that our approach effectively facilitates visual red-teaming evaluations against closed-source MLLMs.

标签

会议 CVPR2026
页码 8610-8620
Bibkey
来源 CVF Open Access