VLM-PTQ：面向大型视觉语言模型的高效训练后量化
VLM-PTQ: Efficient Post-Training Quantization for Large Vision-Language Models
为视觉-语言大模型提供了一种非对称训练后量化框架，通过闭式修正项和模态感知的通道重要性区分，在低比特下显著提升量化性能，可直接用于压缩部署VLM。
原文依据 we introduce VLM-PTQ, an asymmetric post-training quantization framework for VLMs
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论训练后量化（PTQ）是高效压缩大规模模型的关键技术，其中GPTQ（对称校准）和GPTAQ（非对称校准）等权重补偿方法已取得显著成功。然而，将这些方法直接应用于视觉语言模型（VLM）会暴露两个明显缺陷：1）标准的最远舍入（RTN）方法对于非对称目标而言并非最优，未能考虑残差引起的最优量化目标偏移；2）所有输入通道在不同模态间被统一处理，忽略了信息密度的差异。本文提出VLM-PTQ，一种面向VLM的非对称训练后量化框架。首先，我们推导出一个闭式修正项来移动量化目标，该修正项显式地考虑了输出残差及相应的逆Hessian列，从而获得比RTN更好的局部最优解。其次，我们提出模态感知量化，区分视觉和语言令牌之间的通道重要性，使量化器能够通过轻量级搜索预先计算更好的量化参数。我们的方法以极小的开销扩展了权重补偿方法，同时在低比特场景下实现了显著的性能提升。大量实验表明，VLM-PTQ与现有方法相比取得了有竞争力的结果，能够在单块GPU上有效压缩从1B到72B参数的模型。
英文原文摘要 1509 字符
Post-training quantization (PTQ) serves as a vital technique for efficiently compressing large-scale models, with weight-compensation methods such as GPTQ (symmetric calibration) and GPTAQ (asymmetric calibration) showing remarkable success. However, directly applying these methods to Vision-Language Models (VLMs) exposes two notable shortcomings: 1) the standard rounding-to-nearest (RTN) method is suboptimal for the asymmetric objective, failing to account for residual-induced shifts in the optimal quantization target; and 2) all input channels are processed uniformly across modalities, overlooking the distinct information densities. In this paper, we introduce VLM-PTQ, an asymmetric post-training quantization framework for VLMs. First, we derive a closed-form correction term that shifts the quantization target, which explicitly accounts for the output residual and the corresponding inverse Hessian column, yielding a better local optimum than RTN. Second, we propose a modality-aware quantization that differentiates channel importance between vision and language tokens, allowing the quantizer to pre-compute better quantization parameters through a lightweight search. Our method extends weight-compensation methods with minimal overhead while achieving significant performance improvements in low-bit scenarios. Extensive experiments demonstrate that VLM-PTQ achieves competitive results compared to existing methods, effectively compressing models from 1B to 72B parameters on a single GPU.