NuWa：为边缘设备推导轻量级类别特定视觉 Transformer
NuWa: Deriving Lightweight Class-Specific Vision Transformers for Edge Devices
对需要在无人机、智能车等边缘设备上部署 ViT 的视觉研究者，NuWa 提供了一种免重训练的闭式剪枝流程：用类别特定数据做自知识纯化，剔除对目标类别有害的权重，再一次性闭式求解得到紧凑子网，可直接复用于任意类别子集的定制化部署。
原文依据 NuWa performs self-knowledge purification to prune class-detrimental weights and efficiently derives compact ViTs through closed-form optimization.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉 Transformer（ViT）通常需要经过压缩才能部署到无人机、智能车等资源受限的边缘设备上。然而，现有模型压缩方法忽略了一点：许多边缘设备在其应用中只需要特定类别的知识。因此，由此得到的全类别 ViT 会保留冗余知识，并在这些类别上表现欠佳。我们发现，仅仅把校准数据集替换为类别特定数据并不足以解决该问题，因为这类方法面临两个根本性局限。第一，它们忽视了类别有害权重的存在，这些权重会干扰特化过程，而移除它们可以提升类别特定性能。第二，目标类别的多样性以及边缘设备上的资源约束要求大量定制化模型，现有方法耗时且计算开销大，因而难以扩展。在本工作中，我们提出 NuWa，一种低成本方法，通过从基础 ViT 中为具有特定类别需求的边缘设备推导小型 ViT 来应对上述挑战。NuWa 执行自知识纯化以剪除类别有害权重，并通过闭式优化高效地推导出紧凑 ViT。无需剪枝后重训练，所推导的边缘 ViT 在类别特定准确率上即超过基础 ViT，并加速推理。大量实验表明，在类别特定任务上，NuWa 相比最优的免训练剪枝方法准确率最高提升 29.00%。与表现最好的依赖训练的剪枝方法相比，NuWa 实现 33.69 倍的剪枝加速，并将剪枝成本最多降低 99.83%，平均准确率损失仅为 0.61%。
英文原文摘要 1735 字符
Vision Transformers (ViTs) often need to be compressed for deployment on resource-constrained edge devices like drones and smart vehicles. However, existing model compression methods ignore that many edge devices only require the knowledge of specific classes for their applications. As a result, the derived all-class ViTs retain redundant knowledge and perform suboptimally on these classes. We discovered that simply replacing the calibration dataset with class-specific data does not suffice to address this issue, as these methods face two fundamental limitations. First, they overlook the existence of class-detrimental weights, which interfere with specialization, while removing them can improve class-specific performance. Second, the diversity of target classes and resource constraints on edge devices demand numerous customized models. Existing methods are time-consuming and computationally expensive, thus unscalable. In this work, we present NuWa, a cost-efficient method that addresses these challenges by deriving small ViTs from base ViTs for edge devices with specific class requirements. NuWa performs self-knowledge purification to prune class-detrimental weights and efficiently derives compact ViTs through closed-form optimization. Without post-pruning retraining, the derived edge ViTs surpass the base ViT in class-specific accuracy and accelerate inference. Comprehensive experiments demonstrate that NuWa outperforms state-of-the-art training-free pruning methods on class-specific tasks by up to 29.00% in accuracy. Compared with the best-performing training-dependent pruning method, NuWa achieves a 33.69x pruning speedup and reduces pruning cost by up to 99.83%, with only a 0.61% average accuracy loss.