聚焦而非剪枝：为信息密集图像理解定位指令相关区域
Focus, Don't Prune: Identifying Instruction-Relevant Regions for Information-Rich Image Understanding
对做 LVLM 高效推理的人，这篇给出了「先用指令定位相关区域、再只对区域提细粒度特征」的两阶段视觉 token 削减范式，并附带 InfographicVQA/MultiPageDocVQA/SinglePageDocVQA 上的区域级真值标注，可直接用于训练或评测区域选择模块。
原文依据 first identifies instruction-relevant image regions and then refines them to extract fine-grained visual features
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论大型视觉语言模型（LVLM）借助大型语言模型（LLM）的推理能力，在多模态任务上表现出强劲性能。然而，处理视觉上复杂、信息密集的图像（如信息图或文档版面）时，这类模型需要生成大量视觉 token，带来显著的计算开销。为此，我们提出 PinPoint，一种新颖的两阶段框架：先定位与指令相关的图像区域，再对这些区域进行细化以提取细粒度视觉特征，从而提升推理效果与效率。本方法的核心是指令-区域对齐（Instruction-Region Alignment），它同时利用视觉输入与文本指令来定位相关区域。我们进一步引入了新的标注，在具有挑战性的 VQA 基准 InfographicVQA、MultiPageDocVQA 和 SinglePageDocVQA 上为指令相关区域提供更丰富的真值监督。实验结果表明，PinPoint 不仅取得了优于现有方法的准确率，还通过减少无关视觉 token 降低了计算开销。
英文原文摘要 1152 字符
Large Vision-Language Models (LVLMs) have shown strong performance across various multimodal tasks by leveraging the reasoning capabilities of Large Language Models (LLMs). However, processing visually complex and information-rich images, such as infographics or document layouts, requires these models to generate a large number of visual tokens, leading to significant computational overhead. To address this, we propose PinPoint, a novel two-stage framework that first identifies instruction-relevant image regions and then refines them to extract fine-grained visual features for improved reasoning and efficiency. Central to our approach is the Instruction-Region Alignment, which localizes relevant regions using both visual input and textual instructions. We further introduce new annotations that provide richer ground-truth supervision for instruction-relevant regions across challenging VQA benchmarks: InfographicVQA, MultiPageDocVQA, and SinglePageDocVQA. Experimental results show that PinPoint not only achieves superior accuracy compared to existing methods but also reduces computational overhead by minimizing irrelevant visual tokens.