面向大型视觉语言模型的细粒度训练后量化：基于量化感知积分梯度
Fine-Grained Post-Training Quantization for Large Vision Language Models with Quantization-Aware Integrated Gradients
把量化敏感度的粒度从模态级下推到词元级，用积分梯度定量衡量每个词元对量化误差的贡献，可直接迁移到任意 LVLM 的 W4A8/W3A16 部署流程中，作为校准阶段的即插即用模块。
原文依据 we introduce a fine-grained quantization strategy based on Quantization-aware Integrated Gradients (QIG), which leverages integrated gradients to quantitatively evaluate token sensitivity
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论大型视觉语言模型（LVLM）在需要多模态交互的广泛下游任务中取得了显著成功，但其强大的能力伴随着可观的计算与内存开销，阻碍了实际部署。在众多加速技术中，训练后量化是降低内存成本、加速推理的一种流行且有效的策略。然而，现有的 LVLM 量化方法通常在模态层面衡量词元敏感度，无法捕捉丰富而复杂的跨词元交互，也难以在词元层面定量衡量量化误差。随着词元在模型内部交互，模态之间的区分逐渐减弱，这表明需要更细粒度的校准。受机制可解释性中公理化归因的启发，我们提出了一种基于量化感知积分梯度（QIG）的细粒度量化策略，利用积分梯度定量评估词元敏感度，并将粒度从模态级推进到词元级，从而同时反映模态间与模态内的动态特性。在 W4A8 与 W3A16 两种设置下对多个 LVLM 开展的大量实验表明，我们的方法在模型与基准上均能一致地提升精度，且延迟开销可忽略不计。例如，在 3 比特仅权重量化下，我们的方法将 LLaVA-onevision-7B 的平均精度提升 1.60%，使其与全精度对应模型的差距缩小至仅 1.33%。代码见 https://github.com/ucas-xiang/QIG。
英文原文摘要 1642 字符
Large Vision Language Models (LVLMs) have achieved remarkable success in a wide range of downstream tasks that require multimodal interaction, but their powerful capabilities come with substantial computational and memory overhead, which hinders practical deployment. Among numerous acceleration techniques, post-training quantization is a popular and effective strategy for reducing memory cost and accelerating inference. However, existing LVLM quantization methods typically measure token sensitivity at the modality level, which fails to capture the rich and complex cross-token interactions and falls short in quantitatively measuring the quantization error at the token level. As tokens interact within the model, the distinction between modalities gradually diminishes, suggesting the need for fine-grained calibration. Inspired by axiomatic attribution in mechanistic interpretability, we introduce a fine-grained quantization strategy based on Quantization-aware Integrated Gradients (QIG), which leverages integrated gradients to quantitatively evaluate token sensitivity and push the granularity from modality level to token level, reflecting both inter-modality and intra-modality dynamics. Extensive experiments on multiple LVLMs under both W4A8 and W3A16 settings show that our method consistently improves accuracy across models and benchmarks with negligible latency overhead. For example, under 3-bit weight-only quantization, our method improves the average accuracy of LLaVA-onevision-7B by 1.60%, reducing the gap to its full-precision counterpart to only 1.33%. The code is available at https://github.com/ucas-xiang/QIG.