LLMind：受生物启发的免训练自适应视觉表征用于视觉语言模型
LLMind: Bio-inspired Training-free Adaptive Visual Representations for Vision-Language Models
在紧像素预算下，用莫比乌斯参数化的非均匀采样替代 VLM 的均匀视觉编码，并通过测试时自适应的闭环语义反馈把采样密度对齐到文本查询的显著区域；这套即插即用的前端可直接复用于任何冻结 VLM 的高分辨率或长视频输入压缩。
原文依据 a novel training-free framework that mimics foveated encoding and cortical magnification in human vision
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉语言模型（VLM）通常假设视觉输入在整个视野内具有均匀的空间保真度，对即使无信息的区域也分配同等精度。相比之下，人类视觉既非均匀也非静态，而是自适应的、选择性的且资源高效的。有鉴于此，我们首次对受生物启发的视觉表征方法进行系统分析，为更高效、更自适应的 VLM 提供洞见。我们提出 LLMind（Looking Like the Mind），一个新颖的免训练框架，模仿人类视觉中的注视点编码与皮层放大机制，在紧像素预算下为 VLM 实现自适应、高效的表征。我们的核心思想是探索一种受生物启发的自适应采样策略（BASS）。这使我们能够设计一个莫比乌斯参数化模块，在保持全局场景结构的同时执行非均匀采样。在 BASS 之上，我们通过测试时自适应引入闭环语义反馈（CSF），使感知显著性同冻结 VLM 的文本信息对齐。我们在多样的场景级与区域引导视觉问答（VQA）基准上，将 LLMind 与均匀采样及其他采样基线进行比较。结果表明，在紧像素预算下，与均匀采样相比，我们的方法取得显著提升：VQAv2 平均提升 +20%，Seed-Bench 提升 +38%，A-OKVQA 提升 +37%。更令人惊讶的是，结果揭示 LLMind 仅用 1%、3% 和 5% 的像素，即可分别保留全分辨率性能的 82%、92% 和 97%。此外，LLMind 轻量、即插即用，且无需改变架构即可与现有 VLM 兼容。
英文原文摘要 1706 字符
Vision-Language Models (VLMs) typically assume a uniform spatial fidelity across the entire field of view of visual inputs, dedicating equal precision to even the uninformative regions. By contrast, human vision is neither uniform nor static. It is adaptive, selective, and resource-efficient. In light of this, we present the first systematic analysis of bio-inspired visual representation methods, providing insights for more efficient and adaptive VLMs. We propose LLMind (Looking Like the Mind), a novel training-free framework that mimics foveated encoding and cortical magnification in human vision to achieve adaptive, efficient representations for VLMs under tight pixel budgets. Our key idea is to explore a Bio-inspired Adaptive Sampling Strategy (BASS). This empowers us to design a Mobius-parameterized module that performs non-uniform sampling while preserving global scene structure. On top of BASS, we introduce closed-loop semantic feedback (CSF) via test-time adaptation to align the perceptual saliency with the textual information from the frozen VLM. We evaluate LLMind against uniform and other sampling baselines across diverse scene-level and region-guided visual question answering (VQA) benchmarks. The results show that ours achieves dramatic gains, with average improvements by +20% on VQAv2, +38% on Seed-Bench, and +37% on A-OKVQA compared to uniform sampling under tight pixel budgets. More surprisingly, results reveal that LLMind can retain up to 82%, 92% and 97% of the full-resolution performance with only 1%, 3% and 5% of the pixels, respectively. Moreover, LLMind is lightweight, plug-and-play, and compatible with existing VLMs without requiring architectural changes.