POINTS-Long：多模态大模型中的自适应双模式视觉推理
POINTS-Long: Adaptive Dual-Mode Visual Reasoning in MLLMs
对做长视频/流式多模态理解的视觉研究者，这篇提供了可动态缩放的视觉词元方案：focus/standby 双模式在推理时按需在精度与效率间切换，配合可动态分离的 KV-cache 维持超长视觉记忆，可直接借鉴到视觉编码器到 LLM 的词元预算分配与流式记忆管理环节。
原文依据 the standby mode retains 97.7-99.7% of the original accuracy using only 1/40-1/10th of the visual tokens
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多模态大语言模型（MLLMs）近期在跨模态理解与生成方面展现出显著能力。然而，视觉词元序列的快速增长——尤其是在长视频与流式场景中——对其可扩展性与实际部署构成了重大挑战。为此，我们提出 POINTS-Long，一个受人类视觉系统启发、具备动态视觉词元缩放能力的原生双模式 MLLM。该模型支持两种互补的感知模式：聚焦模式与待机模式，使用户能够在推理过程中动态权衡效率与精度。在细粒度视觉任务上，聚焦模式保持最优性能；而在长篇幅通用视觉理解上，待机模式仅使用 1/40 至 1/10 的视觉词元，即可保留原始精度的 97.7%–99.7%。此外，POINTS-Long 通过可动态分离的 KV-cache 设计原生支持流式视觉理解，从而高效维持超长视觉记忆。我们的工作为未来 MLLM 的设计提供了新见解，并为自适应、高效的长篇幅视觉理解奠定了基础。模型与代码见 https://anakin-skywalker-joseph.github.io/POINTS-Long-Webpage。
英文原文摘要 1260 字符
Multimodal Large Language Models (MLLMs) have recently demonstrated remarkable capabilities in cross-modal understanding and generation. However, the rapid growth of visual token sequences--especially in long-video and streaming scenarios--poses a major challenge to their scalability and real-world deployment. Thus, we introduce POINTS-Long, a native dual-mode MLLM featuring dynamic visual token scaling inspired by the human visual system. The model supports two complementary perception modes: focus mode and standby mode, enabling users to dynamically trade off efficiency and accuracy during inference. On fine-grained visual tasks, the focus mode retains the optimal performance, while on long-form general visual understanding, the standby mode retains 97.7-99.7% of the original accuracy using only 1/40-1/10th of the visual tokens. Moreover, POINTS-Long natively supports streaming visual understanding via a dynamically detachable KV-cache design, allowing efficient maintenance of ultra-long visual memory. Our work provides new insights into the design of future MLLMs and lays the foundation for adaptive and efficient long-form visual understanding. Model and code are available at https://anakin-skywalker-joseph.github.io/POINTS-Long-Webpage.