Blink：面向增强多模态理解的动态视觉词元分辨率
Blink: Dynamic Visual Token Resolution for Enhanced Multimodal Understanding
提出基于注意力图的逐层显著性估计与 TokenSR 超分扩展机制，可在单次前向中动态为显著视觉词元分配更多计算，是一种可插拔的视觉词元分辨率调度思路，可迁移到其他 MLLM 的视觉编码与推理加速场景。
原文依据 we propose Blink, a dynamic visual token resolution framework that emulates the human-inspired process within a single forward pass
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多模态大语言模型（MLLMs）在各类视觉语言任务上已取得显著进展，但其视觉感知能力仍然有限。相比之下，人类通过一种类似“眨眼”的序列化过程，动态扫描并聚焦于显著区域，从而高效地感知复杂场景。受这一策略启发，我们首先探究 MLLMs 是否表现出类似行为。我们的初步分析表明，MLLMs 在不同层之间会自然地关注不同的视觉区域，并且有选择地为显著词元分配更多计算能够增强视觉感知。基于这一发现，我们提出 Blink，一个动态视觉词元分辨率框架，在单次前向传播中模拟这种受人类启发的处理过程。具体而言，Blink 包含两个模块：显著性引导扫描与动态词元分辨率。它首先基于注意力图估计每一层中视觉词元的显著性，并通过一个即插即用的词元超分辨率（TokenSR）模块对重要词元进行扩展。在下一层中，当这些扩展词元失去关注时将其丢弃。这一动态机制在广域探索与细粒度聚焦之间取得平衡，从而自适应且高效地增强视觉感知。大量实验验证了 Blink，证明其在增强视觉感知与多模态理解方面的有效性。
英文原文摘要 1370 字符
Multimodal large language models (MLLMs) have achieved remarkable progress on various vision-language tasks, yet their visual perception remains limited. Humans, in comparison, perceive complex scenes efficiently by dynamically scanning and focusing on salient regions in a sequential "blink-like" process. Motivated by this strategy, we first investigate whether MLLMs exhibit similar behavior. Our pilot analysis reveals that MLLMs naturally attend to different visual regions across layers and that selectively allocating more computation to salient tokens can enhance visual perception. Building on this insight, we propose Blink, a dynamic visual token resolution framework that emulates the human-inspired process within a single forward pass. Specifically, Blink includes two modules: saliency-guided scanning and dynamic token resolution. It first estimates the saliency of visual tokens in each layer based on the attention map, and extends important tokens through a plug-and-play token super-resolution (TokenSR) module. In the next layer, it drops the extended tokens when they lose focus. This dynamic mechanism balances broad exploration and fine-grained focus, thereby enhancing visual perception adaptively and efficiently. Extensive experiments validate Blink, demonstrating its effectiveness in enhancing visual perception and multimodal understanding.