鲁棒性的几何：优化损失景观曲率与特征流形对齐以实现视觉语言模型的鲁棒微调
The Geometry of Robustness: Optimizing Loss Landscape Curvature and Feature Manifold Alignment for Robust Finetuning of Vision-Language Models
对做 VLM 微调的人，这篇给出了一个可复用的正则化组合：用局部曲率自适应的权重扰动把优化推向平坦极小值，再加一个跨干净/对抗/OOD 输入的特征对齐损失，从而在同一个微调流程里同时保住 ID、OOD 和对抗三个轴，可直接移植到 CLIP 类模型的鲁棒微调。
原文依据 GRACE employs adaptive weight perturbations scaled by local curvature to promote flatter minima, combined with a feature alignment loss
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉语言模型（VLM）的微调方法面临分布内（ID）准确率、分布外（OOD）泛化与对抗鲁棒性之间的关键三元权衡。现有的鲁棒微调策略至多只能解决该权衡中的两个维度。保持泛化的方法能维持 ID/OOD 性能，却使模型易受对抗攻击；而对抗训练提升了对定向攻击的鲁棒性，却降低了 ID/OOD 准确率。我们的关键洞见是：鲁棒性权衡源于两类几何失效——参数空间中尖锐且各向异性的极小值，以及在扰动下发生形变的不稳定特征表示。为此，我们提出 GRACE（基于自适应曲率估计的 Gram 对齐鲁棒性），这是一个统一的微调框架，对 VLM 同时正则化参数空间曲率与特征空间不变性。GRACE 以鲁棒 PAC-Bayes 理论为基础，采用按局部曲率缩放的自适应权重扰动以促进更平坦的极小值，并结合特征对齐损失，在干净、对抗与 OOD 输入之间保持表示一致性。在 CLIP 模型的 ImageNet 微调上，GRACE 将 ID 准确率提升 10.8%、对抗准确率提升 13.5%，同时保持 57.0% 的 OOD 准确率（零样本基线为 57.4%）。几何分析证实，GRACE 在分布偏移下收敛到更平坦的极小值且不产生特征畸变，为基座 VLM 的广义鲁棒性迈出了有原理依据的一步。
英文原文摘要 1565 字符
Fine-tuning approaches for Vision-Language Models (VLMs) face a critical three-way trade-off between In-Distribution (ID) accuracy, Out-of-Distribution (OOD) generalization, and adversarial robustness. Existing robust fine-tuning strategies resolve at most two axes of this trade-off. Generalization-preserving methods retain ID/OOD performance but leave models vulnerable to adversarial attacks, while adversarial training improves robustness to targeted attacks but degrades ID/OOD accuracy. Our key insight is that the robustness trade-off stems from two geometric failures: sharp, anisotropic minima in parameter space and unstable feature representations that deform under perturbation. To address this, we propose GRACE (Gram-aligned Robustness via Adaptive Curvature Estimation), a unified fine-tuning framework that jointly regularizes the parameter-space curvature and feature-space invariance for VLMs. Grounded in Robust PAC-Bayes theory, GRACE employs adaptive weight perturbations scaled by local curvature to promote flatter minima, combined with a feature alignment loss that maintains representation consistency across clean, adversarial, and OOD inputs. On ImageNet fine-tuning of CLIP models, GRACE simultaneously improves ID accuracy by 10.8%, and adversarial accuracy by 13.5% while maintaining 57.0% OOD accuracy (vs. 57.4% zero-shot baseline). Geometric analysis confirms that GRACE converges to flatter minima without feature distortion across distribution shifts, providing a principled step toward generalized robustness in foundation VLMs.