AdaSVD：面向大型多模态模型的自适应奇异值分解
AdaSVD: Singular Value Decomposition with Adaptive Mechanisms for Large Multimodal Models
对需要在有限显存下部署视觉语言模型的研究者，adaComp 的交替更新奇异矩阵补偿截断误差、adaCR 的逐层自适应压缩比分配，可直接套用到任意 Transformer 主干（如 ViT、LLM 解码器）的低秩压缩流程中。
原文依据 AdaSVD introduces adaComp, which adaptively compensates for SVD truncation errors by alternately updating the singular matrices.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论大型多模态模型（LMM）在多模态处理任务中取得了令人瞩目的成果，但其庞大的内存需求为在资源受限设备上的部署带来了重大障碍。奇异值分解（SVD）已成为一种有前景的 LMM 压缩技术，能够大幅降低内存开销。然而，现有基于 SVD 的方法往往难以有效缓解 SVD 截断所导致的误差，与原始模型相比存在明显的性能差距。此外，在所有 Transformer 层采用统一的压缩比，未能考虑不同层之间重要性的差异。为应对这些挑战，我们提出 AdaSVD，一种基于 SVD 的自适应 LMM 压缩方法。具体而言，AdaSVD 引入 adaComp，通过交替更新奇异矩阵来自适应地补偿 SVD 截断误差。此外，AdaSVD 引入 adaCR，根据每一层的相对重要性自适应地分配层特定的压缩比。在多个 LMM 系列上的综合实验证明了 AdaSVD 的有效性，在显著降低内存需求的同时取得了更优的性能。我们将公开 AdaSVD 的全部代码与模型。
英文原文摘要 1300 字符
Large Multimodal Models (LMMs) have attained impressive achievements in multimodal processing tasks, yet their massive memory demands pose major obstacles to deployment on resource-limited devices. Singular Value Decomposition (SVD) has emerged as a promising compression technique for LMMs, delivering substantial reductions in memory overhead. However, existing SVD-based methods often struggle to effectively alleviate the errors caused by SVD truncation, resulting in a noticeable performance gap when compared to the original models. Moreover, adopting a uniform compression ratio across all transformer layers fails to consider the varying importance of different layers. To tackle these challenges, we propose AdaSVD, an adaptive SVD-based LMM compression approach. Specifically, AdaSVD introduces adaComp, which adaptively compensates for SVD truncation errors by alternately updating the singular matrices. Additionally, AdaSVD introduces adaCR, which adaptively assigns layer-specific compression ratios according to the relative importance of each layer. Comprehensive experiments across multiple LMM families show the effectiveness of AdaSVD, achieving better performance while significantly reducing memory requirements. We will make all the code and models of AdaSVD publicly available.