AdapAction：针对 GUI 智能体的自适应目标动作后门攻击
AdapAction: Adaptive Target Action Backdoor Attack against GUI Agents
对做多模态 GUI 智能体或 VLM 智能体安全的研究者，这篇给出了一个「上下文感知型后门」的攻击范式：触发器不再映射到固定动作，而是让模型按当前界面状态与指令自行选择语义连贯的恶意动作，因此基于指令一致性、视觉连贯性、安全性的 LLM 防御难以识别；可作为构建智能体安全评测与防御（如动作级一致性校验、状态-动作因果审计）的基线攻击。
原文依据 AdapAction enables the agent to autonomously select environmentally coherent malicious actions based on the current GUI state and user instruction
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论由多模态大语言模型（MLLM）驱动的自主图形用户界面（GUI）智能体在复杂任务自动化中日益重要。然而，其自主决策能力引入了显著但尚未被充分探索的安全风险，其中后门攻击是一种尤为隐蔽且高影响的威胁。已有工作已表明 GUI 智能体易受此类攻击，但现有方法依赖静态的触发器—动作映射，执行固定的、与上下文无关的行为，因而极易被检测。为解决这一局限，我们提出 AdapAction，一种新型后门攻击，通过嵌入自适应的、上下文感知的策略来颠覆智能体的决策过程。与传统方法不同，AdapAction 使智能体能够根据当前 GUI 状态与用户指令自主选择与环境相协调的恶意动作，从而在规避检测的同时保持功能可用性。在 Android-In-The-Zoo（AitZ）与 AndroidControl 基准上的大量实验表明，AdapAction 在保持良性任务可用性的同时，攻击成功率（ASR）最高可达 100%。更为关键的是，AdapAction 能够持续规避基于多原则（评估指令一致性、视觉连贯性与安全性）的 LLM 防御，而传统的固定动作攻击则容易被检测。这种鲁棒性源于 AdapAction 具有上下文依据的恶意动作，这些动作在语义与视觉上与合法操作难以区分。因此，AdapAction 展现出极高的隐蔽性，并对由 LLM 驱动的 GUI 智能体构成显著更大的现实威胁。
英文原文摘要 1660 字符
Autonomous Graphical User Interface (GUI) agents powered by Multimodal Large Language Models (MLLMs) are increasingly vital for complex task automation. However, their capacity for self-driven decision-making introduces significant, yet underexplored, security risks, among which backdoor attacks pose a particularly stealthy and high-impact threat. Prior work has shown GUI agents vulnerable to such attacks, but existing methods rely on static trigger-action mappings that execute fixed, context-agnostic behaviors, making them highly detectable. To address this limitation, we introduce **AdapAction**, a novel backdoor attack that subverts the agent's decision-making by embedding an **adaptive, context-aware policy**. Unlike traditional approaches, AdapAction enables the agent to autonomously select environmentally coherent malicious actions based on the current GUI state and user instruction, thereby evading detection while preserving functional utility. Extensive experiments on the Android-In-The-Zoo (AitZ) and AndroidControl benchmarks show that AdapAction achieves up to 100% Attack Success Rate (ASR) while preserving benign task utility. More critically, AdapAction consistently evades a multi-principle-based LLM defense evaluating instruction alignment, visual coherence, and safety, whereas traditional fixed-action attacks are easily detected. This resilience stems from AdapAction's contextually grounded malicious actions, which are semantically and visually indistinguishable from legitimate operations. As a result, AdapAction exhibits exceptional stealth and poses a significantly greater real-world threat to LLM-powered GUI agents.