迈向鲁棒的 Vision Transformer：路径依赖分析与一种简单的两阶段对抗训练
Towards Robust Vision Transformers: Path Dependency Analysis and a Simple Two-Stage Adversarial Training
论文提出的梯度路径掩码（GPM）分析工具可用于定位 ViT 中向对抗样本泄露信息的残差路径，而两阶段对抗训练方案可直接套用到任意 ViT 变体上，在提升鲁棒性的同时不牺牲泛化；此外，论文关于对抗训练后的 ViT 在浅层更依赖全局而非局部关系的发现，为设计 ViT 专用（而非 CNN 混合）架构提供了依据。
原文依据 the residual path serves as a major bottleneck that provides advantageous information to adversaries
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论Vision Transformer（ViT）在性能上已超越卷积神经网络（CNN），成为现代计算机视觉中事实上的主流架构。然而，尽管其表征能力更强，针对 ViT 对抗鲁棒性的研究仍然有限，多数研究仍偏向基于 CNN 的模型。本工作旨在纠正这一架构偏见，并深入分析 ViT 与对抗训练（AT）之间的相互作用。我们首先表明，ViT 能够通过其类别注意力图识别物体的语义部件，这说明经过对抗训练的 ViT 本身已编码了较强的语义先验。接着，我们利用所提出的梯度路径掩码（GPM）分析考察 ViT 的内部信息流，并验证残差路径是向对抗者提供有利信息的主要瓶颈。此外，我们的块间关系分析揭示，经过对抗训练的 ViT 在浅层倾向于更多依赖全局关系而非局部关系——这一新观察表明，ViT 与注入 CNN 式归纳偏置的混合架构之间可能存在根本性的不兼容。基于这些发现，我们设计了一种简单而有效的两阶段对抗训练方案，以缓解这一结构不兼容性，并在多种 ViT 变体和训练方法上同时提升了鲁棒性与泛化能力。所提方法可兼容广泛的对抗训练框架与模型。
英文原文摘要 1531 字符
The Vision Transformer (ViT) has surpassed Convolutional Neural Networks (CNNs) in performance, becoming the de facto architecture in modern computer vision. However, despite its superior representational capacity, research on the adversarial robustness of ViTs remains limited, with most studies still biased toward CNN-based models. This work aims to address this architectural bias and conduct an in-depth analysis of the interaction between ViTs and adversarial training (AT).We first show that ViTs can identify semantic components of objects through their class attention maps, indicating that adversarially trained ViTs inherently encode strong semantic priors. Next, using the proposed Gradient Path Masking (GPM) analysis, we examine the internal information flow of ViTs and verify that the residual path serves as a major bottleneck that provides advantageous information to adversaries. Furthermore, our inter-patch relation analysis reveals that adversarially trained ViTs tend to rely more on global than local relationships in early layers--a novel observation suggesting a fundamental potential incompatibility between ViTs and hybrid architectures that inject CNN-style inductive biases.Building upon these findings, we design a simple yet effective two-stage AT scheme to mitigate this structural incompatibility, achieving simultaneous improvements in robustness and generalization across various ViT variants and training methods. The proposed method is compatible with a wide range of AT frameworks and models.