iSHIFT：带自适应感知的轻量级慢快 GUI 智能体
iSHIFT: Lightweight Slow-Fast GUI Agent with Adaptive Perception
把「慢/快」双模式推理与感知 token 结合，让 2.5B 的 MLLM 按任务难度自适应决定推理深度与注视区域；做视觉定位或高分辨率界面理解时，可借鉴其用特殊 token 引导注意力到局部区域、以隐式思维链替代显式长链推理来省算力的做法。
原文依据 Special perception tokens guide attention to relevant screen regions, allowing the model to decide both how to reason and where to focus.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多模态大语言模型（MLLM）在理解并交互复杂、像素密集的图形用户界面（GUI）环境方面展现出强大潜力。然而，构建既对高层任务高效、又对细粒度交互精确的智能体仍具挑战。GUI 智能体必须高效执行常规操作，同时还要处理需要精确视觉定位的任务，而现有方法在准确性依赖于识别特定界面元素时往往表现不佳。这些 MLLM 规模依然庞大，且无法根据手头任务调整其推理深度。在本工作中，我们提出 iSHIFT：带灵活 token 的隐式慢快混合推理，这是一种轻量级智能体，将潜变量思考（隐式思维链）与感知控制模块相结合。iSHIFT 使 MLLM 能够在慢模式与快模式之间切换：慢模式利用详细的视觉定位以获得高精度，快模式利用全局线索以提升效率。特殊的感知 token 引导注意力指向相关屏幕区域，使模型能够同时决定如何推理以及聚焦何处。尽管规模仅为紧凑的 2.5B，iSHIFT 在多个基准数据集上达到了最优性能。
英文原文摘要 1222 字符
Multimodal Large Language Models (MLLMs) show strong potential for interpreting and interacting with complex, pixel-rich Graphical User Interface (GUI) environments. However, building agents that are both efficient for high-level tasks and precise for fine-grained interactions remains challenging. GUI agents must perform routine actions efficiently while also handling tasks that demand exact visual grounding, yet existing approaches struggle when accuracy depends on identifying specific interface elements. These MLLMs also remain large and cannot adapt their reasoning depth to the task at hand. In this work, we introduce iSHIFT: Implicit Slow-fast Hybrid Inference with Flexible Tokens, a lightweight agent that integrates latent thinking (implicit chain-of-thought) with a perception control module. iSHIFT enables an MLLM to switch between a slow mode, which leverages detailed visual grounding for high precision and a fast mode that uses global cues for efficiency. Special perception tokens guide attention to relevant screen regions, allowing the model to decide both how to reason and where to focus. Despite its compact 2.5B size, iSHIFT matches state-of-the-art performance on multiple benchmark datasets.