HeSS：用于 VGGT 稀疏性重分配的检测头敏感度分数
HeSS: Head Sensitivity Score for Sparsity Redistribution in VGGT
对做 3D 重建/前馈几何模型的人，这篇给出了一个可直接套用的检测头级稀疏预算再分配方案：用小型校准集上的 Hessian 近似给每个注意力检测头打敏感度分，再按分数非均匀分配注意力预算，从而在 VGGT 这类全局注意力主干上以训练-free 方式换取高稀疏下的精度保持。
原文依据 we perform HeSS-Guided Sparsification, leveraging the pre-computed HeSS to reallocate the total attention budget
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉几何基础 Transformer（VGGT）推动了三维视觉的发展，但其全局注意力层存在二次计算开销，制约了可扩展性。已有若干基于稀疏化的加速技术被提出以缓解该问题，但往往伴随显著的精度下降。我们假设精度下降源于检测头间稀疏化敏感性的异质性，因为现有方法对所有检测头施加统一的稀疏模式。基于这一假设，我们提出一个两阶段稀疏化流程，能够有效量化并利用检测头级的稀疏化敏感性。在第一阶段，我们使用一个新指标——检测头敏感度分数（HeSS）——来度量检测头级稀疏化敏感性，该指标在一个小型校准集上针对两个不同的误差项近似 Hessian。在推理阶段，我们执行 HeSS 引导的稀疏化，利用预先计算的 HeSS 重新分配总注意力预算——为敏感检测头分配更稠密的注意力，为更鲁棒的检测头分配更稀疏的注意力。我们证明 HeSS 能有效刻画检测头级稀疏化敏感性，并从经验上确认全局注意力层中的注意力检测头表现出异质的敏感性特征。大量实验进一步表明，我们的方法在高稀疏度下有效缓解性能退化，在不同稀疏化水平上展现出强鲁棒性。代码见 https://github.com/libary753/HeSS。
英文原文摘要 1574 字符
Visual Geometry Grounded Transformer (VGGT) has advanced 3D vision, yet its global attention layers suffer from quadratic computational costs that hinder scalability. Several sparsification-based acceleration techniques have been proposed to alleviate this issue, but they often suffer from substantial accuracy degradation. We hypothesize that the accuracy degradation stems from the heterogeneity in head-wise sparsification sensitivity, as the existing methods apply a uniform sparsity pattern across all heads. Motivated by this hypothesis, we present a two-stage sparsification pipeline that effectively quantifies and exploits headwise sparsification sensitivity. In the first stage, we measure head-wise sparsification sensitivity using a novel metric, the Head Sensitivity Score (HeSS), which approximates the Hessian with respect to two distinct error terms on a small calibration set. In the inference stage, we perform HeSS-Guided Sparsification, leveraging the pre-computed HeSS to reallocate the total attention budget-assigning denser attention to sensitive heads and sparser attention to more robust ones. We demonstrate that HeSS effectively captures head-wise sparsification sensitivity and empirically confirm that attention heads in the global attention layers exhibit heterogeneous sensitivity characteristics. Extensive experiments further show that our method effectively mitigates performance degradation under high sparsity, demonstrating strong robustness across varying sparsification levels. Code is available at https://github.com/libary753/HeSS.