模型作为乐高搭建者:通过语义蓝图从良性积木组装恶意
Models as Lego Builders: Assembling Malice from Benign Blocks via Semantic Blueprints
提出一种基于结构化视觉提示(思维导图、表格、旭日图)的语义槽填充越狱方法,为研究LVLM多模态安全对齐与防御提供了可复用的攻击范式。
原文依据 StructAttack decomposes a harmful query into a central topic and a set of benign-looking slot types, then embeds them as structured visual prompts
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论尽管大型视觉语言模型(LVLM)取得了快速进展,视觉模态的引入带来了新的安全漏洞,对手可利用这些漏洞诱导出有偏见或恶意的输出。本文通过语义槽填充展示了一个未被充分探索的漏洞:即使槽类型被刻意设计为看似良性,LVLM也会用不安全内容填充缺失的槽值。基于这一发现,我们提出了StructAttack,一个简单而有效的黑盒设置下的单查询越狱框架。StructAttack将有害查询分解为一个中心主题和一组看似良性的槽类型,然后将它们嵌入为带有小随机扰动的结构化视觉提示(例如思维导图、表格或旭日图)。配合补全引导指令,LVLM自动重组隐藏的语义并生成不安全输出,而不会触发安全机制。尽管每个槽在孤立情况下看似良性(局部良性),StructAttack利用LVLM的推理能力将这些槽组装成连贯的有害语义。在多个模型和基准上的大量实验表明了所提出的StructAttack的有效性。
英文原文摘要 1220 字符
Despite the rapid progress of Large Vision-Language Models (LVLMs), the integration of visual modalities introduces new safety vulnerabilities that adversaries can exploit to elicit biased or malicious outputs. In this paper, we demonstrate an underexplored vulnerability via semantic slot filling, where LVLMs complete missing slot values with unsafe content even when the slot types are deliberately crafted to appear benign. Building on this finding, we propose StructAttack, a simple yet effective single-query jailbreak framework under black-box settings. StructAttack decomposes a harmful query into a central topic and a set of benign-looking slot types, then embeds them as structured visual prompts (e.g., mind maps, tables, or sunburst diagrams) with small random perturbations. Paired with a completion-guided instruction, LVLMs automatically recompose the concealed semantics and generate unsafe outputs without triggering safety mechanisms. Although each slot appears benign in isolation (local benignness), StructAttack exploits LVLMs' reasoning to assemble these slots into coherent harmful semantics. Extensive experiments on multiple models and benchmarks show the efficacy of our proposed StructAttack.