DeepScan:面向大型视觉语言模型视觉定位推理的免训练框架
DeepScan: A Training-Free Framework for Visually Grounded Reasoning in Large Vision-Language Models
提出免训练的分层扫描与重聚焦机制,可即插即用地增强LVLM的视觉定位与细粒度理解能力,为视觉证据提取与推理提供可复用框架。
原文依据 DeepScan, a training-free framework that combines Hierarchical Scanning, Refocusing, and Evidence-Enhanced Reasoning for visually grounded reasoning
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论人类即使在嘈杂环境中也能稳健地定位视觉证据并给出有依据的答案,其方式是先识别关键线索,再以自底向上的方式将其与完整上下文关联。受此启发,我们提出DeepScan,一个免训练框架,结合分层扫描、重聚焦和证据增强推理,用于大型视觉语言模型(LVLM)中的视觉定位推理。与追求一次性定位完整证据的现有方法不同,分层扫描执行局部线索探索和多尺度证据提取,以自底向上的方式恢复证据,有效减轻干扰性上下文的影响。随后,重聚焦通过LVLM与视觉专家的协作优化定位到的证据视图。最后,证据增强推理通过混合证据记忆聚合多粒度视图,产生准确且可解释的答案。实验结果表明,DeepScan显著提升了LVLM在多种视觉任务中的表现,尤其是在细粒度视觉理解方面。当与Qwen2.5-VL-7B集成时,在V*上达到90.6%的整体准确率。此外,DeepScan为不同架构和模型规模的LVLM提供一致的改进,且无需额外的适配成本。代码即将开源。
英文原文摘要 1339 字符
Humans can robustly localize visual evidence and provide grounded answers even in noisy environments by identifying critical cues and then relating them to the full context in a bottom-up manner. Inspired by this, we propose DeepScan, a training-free framework that combines Hierarchical Scanning, Refocusing, and Evidence-Enhanced Reasoning for visually grounded reasoning in Large Vision-Language Models (LVLMs). Unlike existing methods that pursue one-shot localization of complete evidence, Hierarchical Scanning performs local cue exploration and multi-scale evidence extraction to recover evidence in a bottom-up manner, effectively mitigating the impacts of distractive context. Refocusing then optimizes the localized evidence view through collaboration of LVLMs and visual experts. Finally, Evidence-Enhanced Reasoning aggregates multi-granular views via a hybrid evidence memory and yields accurate and interpretable answers. Experimental results demonstrate that DeepScan significantly boosts LVLMs in diverse visual tasks, especially in fine-grained visual understanding. It achieves 90.6% overall accuracy on V* when integrated with Qwen2.5-VL-7B. Moreover, DeepScan provides consistent improvements for LVLMs across various architectures and model scales without additional adaptation cost. The code will be open-source soon.