Prune2Drive：面向自动驾驶视觉语言模型加速的即插即用框架
Prune2Drive: A Plug-and-Play Framework for Accelerating Vision-Language Models in Autonomous Driving
面向多视图 VLM 的即插即用视觉词元剪枝：用多样性感知选择保证跨视图语义/空间覆盖，并用视图自适应控制器按相机重要性学习剪枝比例，且不依赖注意力图、无需重训练，可直接套到自动驾驶多相机 VLM 的预填充阶段省算力。
原文依据 a plug-and-play visual token pruning framework for multi-view VLMs in AD
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉语言模型（VLM）已成为自动驾驶（AD）中一种有前景的范式，为感知与决策提供了统一框架。然而，在处理高分辨率、多视图图像时，其显著的计算开销阻碍了实际部署。这种复杂性源于海量的视觉词元：由于自注意力的二次复杂度，这些词元会增加推理延迟与内存消耗。为应对这些挑战，我们提出 Prune2Drive，一个面向自动驾驶中多视图 VLM 的即插即用视觉词元剪枝框架。Prune2Drive 引入两项核心创新：（i）多样性感知的词元选择机制，优先保证跨视图的语义与空间覆盖；（ii）视图自适应剪枝控制器，根据各相机对下游任务的重要性自动学习最优剪枝比例。与先前方法不同，Prune2Drive 无需重新训练模型，也无需访问注意力图，从而确保与现代高效注意力实现兼容。在 DriveLM 与 DriveLMM-o1 基准上的大量实验表明，Prune2Drive 在性能损失极小的情况下实现了显著的加速与内存节省。当仅保留 10% 的视觉词元时，我们的方法在预填充阶段实现 6.40 倍加速，仅消耗原始 FLOPs 的 13.4%，在 DriveLM 基准上的平均性能下降仅为 3%。代码见：https://github.com/MinhaoXiong/Prune2Drive.git
英文原文摘要 1515 字符
Vision-Language Models (VLMs) have emerged as a promising paradigm in autonomous driving (AD), providing a unified framework for perception and decisionmaking. However, their real-world deployment is hindered by significant computational overhead when processing high-resolution, multi-view images. This complexity stems from the massive number of visual tokens, which increases inference latency and memory consumption due to the quadratic complexity of self-attention. To address these challenges, we propose Prune2Drive, a plug-and-play visual token pruning framework for multi-view VLMs in AD. Prune2Drive introduces two core innovations: (i) a diversity-aware token selection mechanism that prioritizes semantic and spatial coverage across views, and (ii) a view-adaptive pruning controller that automatically learns optimal pruning ratios based on camera importance to downstream tasks. Unlike prior methods, Prune2Drive requires no model retraining or access to attention maps, ensuring compatibility with modern efficient attention implementations. Extensive experiments on the DriveLM and DriveLMM-o1 benchmarks demonstrate that Prune2Drive achieves significant speedups and memory savings with minimal performance impact. When retaining only 10% of visual tokens, our method achieves a 6.40x speedup in the prefilling phase and consumes only 13.4% of the original FLOPs, with a mere 3% average performance drop on the DriveLM benchmark. Code is available at: https://github.com/MinhaoXiong/Prune2Drive.git