PA-Attack：以原型与注意力引导针对 LVLM 视觉编码器的灰盒攻击
PA-Attack: Guiding Gray-Box Attacks on LVLM Vision Encoders with Prototypes and Attention
把视觉编码器当作跨模型稳定的灰盒攻击支点，并用 token 级注意力分数定位关键视觉 token 施加扰动，这一思路可直接迁移到对 CLIP/ViT 类视觉骨干的对抗鲁棒性评测与防御设计上。
原文依据 leverage token-level attention scores to concentrate perturbations on critical visual tokens
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论大型视觉语言模型（LVLM）是现代多模态应用的基础，但其对对抗攻击的脆弱性仍是一个关键问题。此前的白盒攻击很少能跨任务泛化，而黑盒方法依赖昂贵的迁移，限制了效率。视觉编码器已被标准化，且常在多个 LVLM 之间共享，因而提供了一个稳定、具有强跨模型迁移能力的灰盒支点。基于这一前提，我们提出 PA-Attack（原型锚定的注意力攻击）。PA-Attack 首先采用原型锚定引导，朝一个通用且不相似的原型提供稳定的攻击方向，从而解决原始攻击受属性限制的问题以及任务泛化能力有限的问题。在此基础上，我们提出两阶段注意力增强机制：（i）利用 token 级注意力分数将扰动集中在关键视觉 token 上；（ii）自适应地重新校准注意力权重，以跟踪对抗过程中不断演化的注意力。在多种下游任务和 LVLM 架构上的大量实验表明，PA-Attack 取得平均 75.1% 的分数下降率（SRR），在 LVLM 上展现出很强的攻击有效性、效率与任务泛化能力。
英文原文摘要 1270 字符
Large Vision-Language Models (LVLMs) are foundational to modern multimodal applications, yet their susceptibility to adversarial attacks remains a critical concern. Prior white-box attacks rarely generalize across tasks, and black-box methods depend on expensive transfer, which limits efficiency. The vision encoder, standardized and often shared across LVLMs, provides a stable gray-box pivot with strong cross-model transfer. Building on this premise, we introduce PA-Attack (Prototype-Anchored Attentive Attack). PA-Attack begins with a prototype-anchored guidance that provides a stable attack direction towards a general and dissimilar prototype, tackling the attribute-restricted issue and limited task generalization of vanilla attacks. Building on this, we propose a two-stage attention enhancement mechanism: (i) leverage token-level attention scores to concentrate perturbations on critical visual tokens, and (ii) adaptively recalibrate attention weights to track the evolving attention during the adversarial process. Extensive experiments across diverse downstream tasks and LVLM architectures show that PA-Attack achieves an average 75.1% score reduction rate (SRR), demonstrating strong attack effectiveness, efficiency, and task generalization in LVLMs.