当CLIP看到更多时，它反击得更狠：面向测试时对抗鲁棒性的多视图引导自适应反击
When CLIP Sees More, It Fights Back Harder: Multi-View Guided Adaptive Counterattacks for Test-Time Adversarial Robustness
提出一种免调优的测试时多视图自适应反击框架，通过估计每个视图的损坏程度来软加权反击强度，可直接用于提升CLIP等视觉语言模型在推理阶段对抗攻击的鲁棒性。
原文依据 MAC adaptively scales the counterattack intensity for each view based on its estimated corruption degree.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论诸如CLIP的视觉语言模型已实现卓越的零样本识别能力，但其对抗扰动的鲁棒性仍然有限。最近提出的测试时反击（TTC）通过在推理期间扰动输入图像以使其偏离损坏状态，从而提升CLIP的鲁棒性。然而，TTC在强攻击下仍然脆弱，因为其反击依赖于直接损坏的原始视图，并采用噪声驱动的硬门控方案，无法适应变化的损坏严重程度。为解决这些局限，我们引入多视图引导自适应反击（MAC），它通过损坏感知的软加权对多视图执行反击。具体而言，MAC首先构建输入图像的增强视图以获得多样化嵌入。然后执行反击以细化视图中损坏的嵌入。接着，MAC根据每个视图估计的损坏程度自适应地缩放其反击强度。最后，对自适应反击后的视图进行聚合以产生鲁棒的最终预测。在20个数据集和多种攻击场景上的大量实验表明，MAC在保持高推理速度和内存效率的同时显著提升了鲁棒性，且具有免调优设计。我们的代码可在https://github.com/sunoh-kim/MAC获取。
英文原文摘要 1381 字符
Vision-language models such as CLIP have achieved remarkable zero-shot recognition capabilities, yet their robustness against adversarial perturbations remains limited. Test-time counterattack (TTC) was recently proposed to improve CLIP's robustness by perturbing an input image to steer it away from a corrupted state during inference. However, TTC remains fragile under strong attacks because its counterattack relies on a directly corrupted original view and employs a noise-driven hard-gating scheme that cannot adapt to varying corruption severity. To address these limitations, we introduce Multi-view guided Adaptive Counterattack (MAC), which performs counterattacks for multi-view with corruption-aware soft weighting. Specifically, MAC first constructs augmented views of an input image to obtain diverse embeddings. It then performs counterattacks to refine corrupted embeddings of views. Next, MAC adaptively scales the counterattack intensity for each view based on its estimated corruption degree. Finally, the adaptively counterattacked views are aggregated to yield a robust final prediction. Extensive experiments across 20 datasets and diverse attack scenarios demonstrate that MAC substantially improves robustness while preserving high inference speed and memory efficiency with its tuning-free design. Our code is available at https://github.com/sunoh-kim/MAC.