Quant Experts：面向大型视觉语言模型量化的 token 感知自适应误差重建与混合专家
Quant Experts: Token-aware Adaptive Error Reconstruction with Mixture of Experts for Large Vision-Language Models Quantization
对需要在有限显存下部署 VLM 的视觉研究者，这篇论文给出了一个可直接套用的 PTQ 误差补偿范式：按 token 与模态区分重要通道，用共享低秩适配器加路由低秩适配器（MoE）分别补偿全局与局部量化误差，可迁移到任意 VLM 的量化部署流程中。
原文依据 we propose Quant Experts (QE), a token-aware adaptive error compensation with mixture-of-experts for VLMs quantization
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论训练后量化（PTQ）通过压缩权重与激活、无需重新训练完整模型，已成为缓解视觉语言模型（VLM）庞大计算与内存开销的有效技术。现有 PTQ 方法主要依赖对敏感通道或离群通道的静态识别与全局补偿，却往往忽视这些重要通道在不同输入之间的分布差异，导致量化效果不理想。在本工作中，我们观察到重要通道的分布与出现频率在不同模态之间、以及同一模态内的不同 token 之间都存在显著差异。据此，我们提出 Quant Experts（QE），一种面向 VLM 量化的、基于混合专家的 token 感知自适应误差补偿方法。QE 将重要通道划分为 token 无关组与 token 相关组。对于前者，设计一个共享专家，为大多数 token 使用低秩适配器补偿全局量化误差；对于后者，构建由多个路由低秩适配器组成的路由专家，以补偿与特定 token 相关的局部量化误差。大量实验表明，QE 在从 2B 到 70B 参数的各种量化设置与模型规模下均能稳定提升任务准确率，同时保持与全精度模型相当的性能。
英文原文摘要 1438 字符
Post-Training Quantization (PTQ) has emerged as an effective technique for alleviating the substantial computational and memory overheads of Vision-Language Models (VLMs) by compressing both weights and activations without retraining the full model. Existing PTQ methods primarily rely on static identification and global compensation of sensitive or outlier channels, yet they often overlook the distributional differences of these important channels across inputs, leading to unsatisfactory quantization. In this work, we observe that the distributions and occurrence frequencies of important channels vary significantly both across modalities and among tokens, even within the same modality. Accordingly, we propose Quant Experts (QE), a token-aware adaptive error compensation with mixture-of-experts for VLMs quantization. QE divides the important channels into token-independent and token-dependent groups. For the former, a shared expert is designed for most tokens to compensate for global quantization error using a low-rank adapter. For the latter, routed experts including multiple routed low-rank adapters are elaborated to compensate for local quantization error related to specific tokens. Extensive experiments demonstrate that QE consistently enhances task accuracy across various quantization settings and model scales, ranging from 2B to 70B parameters, while maintaining performance comparable to full-precision models.