LaSM：用于防御 GUI 智能体弹窗攻击的逐层缩放机制
LaSM: Layer-wise Scaling Mechanism for Defending Pop-up Attack on GUI Agents
对做多模态大模型智能体或视觉语言模型安全的研究者，这篇论文给出一个免训练、推理时即可用的防御手段：通过逐层放大关键层的注意力与 MLP 模块，把模型显著性重新拉回任务相关区域；其「逐层注意力分歧」分析也可直接迁移到其他视觉注入攻击（如对抗 patch、干扰物注入）的脆弱性诊断上。
原文依据 we propose LaSM, a Layer-wise Scaling Mechanism that selectively amplifies attention and MLP modules in critical layers
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论基于多模态大语言模型（MLLM）构建的图形用户界面（GUI）智能体近期在基于屏幕的交互任务中展现出强大的决策能力。然而，它们对基于弹窗的环境注入攻击仍高度脆弱：恶意视觉元素会转移模型注意力，导致不安全或错误的动作。现有防御方法要么需要代价高昂的重新训练，要么在归纳性干扰下表现不佳。在本工作中，我们系统研究了此类攻击如何改变 GUI 智能体的注意力行为，并揭示了正确输出与错误输出之间存在逐层注意力分歧模式。基于这一洞见，我们提出 LaSM，一种逐层缩放机制，可选择性地放大关键层中的注意力与 MLP 模块。LaSM 在无需额外训练的情况下改善了模型显著性与任务相关区域之间的对齐。在多个数据集上的大量实验表明，我们的方法显著提升了防御成功率并展现出强鲁棒性，同时对模型通用能力的影响可忽略不计。我们的发现揭示，注意力错位是 MLLM 智能体的核心脆弱性，并可通过选择性的逐层调制加以有效解决。
英文原文摘要 1297 字符
Graphical user interface (GUI) agents built on multimodal large language models (MLLMs) have recently demonstrated strong decision-making abilities in screen-based interaction tasks. However, they remain highly vulnerable to pop-up-based environmental injection attacks, where malicious visual elements divert model attention and lead to unsafe or incorrect actions. Existing defense methods either require costly retraining or perform poorly under inductive interference. In this work, we systematically study how such attacks alter the attention behavior of GUI agents and uncover a layer-wise attention divergence pattern between correct and incorrect outputs. Based on this insight, we propose LaSM, a Layer-wise Scaling Mechanism that selectively amplifies attention and MLP modules in critical layers. LaSM improves the alignment between model saliency and task-relevant regions without additional training. Extensive experiments across multiple datasets demonstrate that our method significantly improves the defense success rate and exhibits strong robustness, while having negligible impact on the model's general capabilities. Our findings reveal that attention misalignment is a core vulnerability in MLLM agents and can be effectively addressed through selective layer-wise modulation.