思深则视弱：理解与缓解多模态大语言模型推理过程中的感知损伤
Deeper Thought, Weaker Aim: Understanding and Mitigating Perceptual Impairment during Reasoning in Multimodal Large Language Models
提出免训练的视觉区域引导注意力（VRGA）框架，通过熵-聚焦准则选择视觉头并重加权注意力，缓解多模态大模型在推理时的注意力分散问题，可直接用于提升VQA和视觉定位任务的性能。
原文依据 we propose a training-free Visual Region-Guided Attention (VRGA) framework that selects visual heads based on an entropy-focus criterion and reweights their attention
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多模态大语言模型（MLLMs）在扩展推理模式下常出现感知损伤，尤其是在视觉问答（VQA）任务中。我们将注意力分散确定为根本原因：在多步推理过程中，模型的视觉注意力变得分散并偏离与问题相关的区域，实际上对视觉输入“失去了焦点”。为了更好地理解这一现象，我们分析了MLLMs的注意力图，并观察到推理提示显著降低了对回答问题的关键区域的注意力。我们进一步发现，模型对图像令牌的整体注意力与模型在图像内注意力的空间分散性之间存在强相关性。利用这一见解，我们提出了一个免训练的视觉区域引导注意力（VRGA）框架，该框架基于熵-聚焦准则选择视觉头并重新加权其注意力，有效引导模型在推理过程中关注与问题相关的区域。在视觉-语言基准上的大量实验表明，我们的方法有效缓解了感知退化，提升了视觉定位和推理准确性，同时为MLLMs如何处理视觉信息提供了可解释的见解。
英文原文摘要 1297 字符
Multimodal large language models (MLLMs) often suffer from perceptual impairments under extended reasoning modes, particularly in visual question answering (VQA) tasks. We identify attention dispersion as the underlying cause: during multi-step reasoning, model's visual attention becomes scattered and drifts away from question-relevant regions, effectively "losing focus" on the visual input. To better understand this phenomenon, we analyze the attention maps of MLLMs and observe that reasoning prompts significantly reduce attention to regions critical for answering the question. We further find a strong correlation between model's overall attention on image tokens and the spatial dispersiveness of model's attention within the image. Leveraging this insight, we propose a training-free Visual Region-Guided Attention (VRGA) framework that selects visual heads based on an entropy-focus criterion and reweights their attention, effectively guiding the model to focus on question-relevant regions during reasoning. Extensive experiments on vision-language benchmarks demonstrate that our method effectively alleviates perceptual degradation, leading to improvements in visual grounding and reasoning accuracy, while offering interpretable insights into how MLLMs process visual information.