MASQuant：面向多模态大模型的模态感知平滑量化
MASQuant: Modality-Aware Smoothing Quantization for Multimodal Large Language Models
为多模态大模型（视觉-语言）提供可直接套用的训练后量化方案：按模态分别学习平滑因子，并用 SVD 白化把跨模态激活差异压成低秩，从而在视觉 token 与文本 token 分布差异大时仍能统一量化，适合需要把 VLM 部署到低比特推理的场景。
原文依据 using SVD whitening to transform multi-modal activation differences into low-rank forms, enabling unified quantization across modalities
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论具备计算等价性的训练后量化（PTQ）在大语言模型（LLM）上已取得显著进展，但将其应用于多模态大语言模型（MLLM）时仍面临重大挑战。本文以 SmoothQuant 为案例进行分析，识别出两个关键问题：平滑失配与跨模态计算不变性。为解决这些问题，我们提出模态感知平滑量化（MASQuant），这一新框架引入：（1）模态感知平滑（MAS），学习各自独立的、模态专属的平滑因子，以避免平滑失配；（2）跨模态补偿（CMC），通过 SVD 白化将多模态激活差异变换为低秩形式，从而实现跨模态的统一量化，以解决跨模态计算不变性问题。MASQuant 在双模态与三模态 MLLM 上均展现出稳定的量化性能。实验结果表明，MASQuant 在最优的 PTQ 算法中具有竞争力。
英文原文摘要 1072 字符
Post-training quantization (PTQ) with computational equivalence for Large Language Models (LLMs) have demonstrated remarkable advances, however, their application to Multimodal Large Language Models (MLLMs) presents substantial challenges. In this paper, we analyze SmoothQuant as a case study and identify two critical issues: Smoothing Misalignment and Cross-Modal Computational Invariance. To address these issues, we propose Modality-Aware Smoothing Quantization (MASQuant), a novel framework that introduces (1) Modality-Aware Smoothing (MAS), which learns separate, modality-specific smoothing factors to prevent Smoothing Misalignment, and (2) Cross-Modal Compensation (CMC), which addresses Cross-modal Computational Invariance by using SVD whitening to transform multi-modal activation differences into low-rank forms, enabling unified quantization across modalities. MASQuant demonstrates stable quantization performance across both dual-modal and tri-modal MLLMs. Experimental results show that MASQuant is competitive among the state-of-the-art PTQ algorithms.