ParallelVLM：基于视觉对齐感知并行推测解码的无损视频大模型加速
ParallelVLM: Lossless Video-LLM Acceleration with Visual Alignment Aware Parallel Speculative Decoding
提出免训练的并行推测解码框架，通过无偏验证器引导剪枝对齐草稿与目标模型，将视频词元剪枝与推测解码结合，为视频大模型的长视频推理提供可复用的加速方案。
原文依据 ParallelVLM, a training-free draft-then-verify speculative decoding framework that overcomes both mutual waiting and limited speedup-ratio problems
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论尽管当前视频大模型在视频理解任务中取得了令人瞩目的性能，但其自回归解码效率仍受限于海量视频词元。视觉词元剪枝可以部分缓解这一瓶颈，但现有方法仍存在信息损失，且解码加速幅度有限。本文提出 ParallelVLM，一种免训练的“草稿-验证”推测解码框架，克服了长视频场景下草稿模型与目标模型之间的相互等待和加速比受限问题。ParallelVLM 采用两个并行化阶段以最大化硬件利用率，并引入无偏验证器引导剪枝策略，通过消除注意力引导剪枝中的位置偏差，使草稿模型与目标模型更好对齐。大量实验表明，ParallelVLM 有效将草稿窗口扩展 1.6-1.8 倍并保持高接受长度，在 LLaVA-Onevision-72B 上相比原始自回归解码加速 3.36 倍，在 Qwen2.5-VL-32B 上加速 2.42 倍，覆盖多种视频理解基准。
英文原文摘要 1122 字符
Although current Video-LLMs achieve impressive performance in video understanding tasks, their autoregressive decoding efficiency remains constrained by the massive number of video tokens. Visual token pruning can partially ease this bottleneck, yet existing approaches still suffer from information loss and yield only modest acceleration in decoding. In this paper, we propose ParallelVLM, a training-free draft-then-verify speculative decoding framework that overcomes both mutual waiting and limited speedup-ratio problems between draft and target models in long-video settings. ParallelVLM features two parallelized stages that maximize hardware utilization and incorporates an Unbiased Verifier-Guided Pruning strategy to better align the draft and target models by eliminating the positional bias in attention-guided pruning. Extensive experiments demonstrate that ParallelVLM effectively expands the draft window by 1.6-1.8x with high accepted lengths, and accelerates various video understanding benchmarks by 3.36x on LLaVA-Onevision-72B and 2.42x on Qwen2.5-VL-32B compared with vanilla autoregressive decoding.