ARGUS：通过引导指令遵循行为防御多模态间接提示词注入
ARGUS: Defending Against Multimodal Indirect Prompt Injection via Steering Instruction-Following Behavior
对做多模态大模型视觉侧安全的人，这篇给出了一个与模态无关的表征空间干预手段：在指令遵循子空间内搜索与效用退化方向解耦的引导方向，并配合自适应强度与按需触发的轻量检测，可直接迁移到图像/视频输入下的越狱与注入防御。
原文依据 we discover that the instruction-following behavior of MLLMs is encoded in a subspace
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多模态大语言模型（MLLM）日益容易受到多模态间接提示词注入（IPI）攻击，这类攻击将恶意指令嵌入图像、视频或音频中以劫持模型行为。现有防御主要面向纯文本大语言模型设计，难以应对这些多模态威胁，因为它们容易被绕过、依赖特定模态，或泛化能力较差。受激活引导研究的启发，我们假设：通过在表征空间中引导模型行为，可以实现一种与模态无关的鲁棒通用防御。通过大量实验，我们发现 MLLM 的指令遵循行为被编码在一个子空间中。沿该子空间内的方向进行引导，能够促使模型遵循用户指令，从而构成防御的基础。然而，我们也发现朴素的防御方向可能与导致效用退化的方向相耦合，且过强的干预强度会损害模型性能。为解决这一问题，我们提出 ARGUS，它在安全子空间内搜索与效用退化方向解耦的最优防御方向，并进一步结合自适应强度引导，以实现更好的安全-效用权衡。ARGUS 还引入了一个轻量的注入检测阶段以按需激活防御，以及一个后置过滤阶段以验证防御是否成功。实验结果表明，ARGUS 能够在多模态 IPI 下实现鲁棒防御，同时最大限度地保留 MLLM 的效用。我们的代码将发布于 https://github.com/ZeroNLP/ARGUS。
英文原文摘要 1601 字符
Multimodal Large Language Models (MLLMs) are increasingly vulnerable to multimodal Indirect Prompt Injection (IPI) attacks, which embed malicious instructions in images, videos, or audio to hijack model behavior. Existing defenses, designed primarily for text-only LLMs, are unsuitable for countering these multimodal threats, as they are easily bypassed, modality-dependent, or generalize poorly. Inspired by activation steering research, we hypothesize that a robust, general defense independent of modality can be achieved by steering the model's behavior in the representation space. Through extensive experiments, we discover that the instruction-following behavior of MLLMs is encoded in a subspace. Steering along directions within this subspace can enforce adherence to user instructions, forming the basis of a defense. However, we also found that a naive defense direction could be coupled with a utility-degrading direction, and excessive intervention strength harms model performance. To address this, we propose ARGUS, which searches for an optimal defense direction within the safety subspace that decouples from the utility degradation direction, further combining adaptive strength steering to achieve a better safety-utility trade-off. ARGUS also introduces a lightweight injection detection stage to activate the defense on-demand, and a post-filtering stage to verify defense success. Experimental results show that ARGUS can achieve robust defense against multimodal IPI while maximally preserving the MLLM's utility. Our code will be available at https://github.com/ZeroNLP/ARGUS.