DRS-GUI：面向免训练 GUI 视觉定位的动态区域搜索
DRS-GUI: Dynamic Region Search for Training-Free GUI Grounding
把 MCTS 驱动的 Focus/Shift/Scatter 区域搜索作为免训练插件接到现成 MLLM 上，用于高分辨率截图中的小目标定位，可直接迁移到任意需要从大图中裁剪出相关区域再交给 VLM 的视觉定位任务。
原文依据 we propose DRS-GUI, a training-free dynamic region search framework for GUI grounding that can be seamlessly integrated into existing MLLMs
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论由多模态大语言模型（MLLM）驱动的 GUI 智能体在理解并执行用户指令方面已展现出令人印象深刻的能力。然而，对于现有方法而言，从布满无关 UI 组件的高分辨率截图中准确视觉定位与指令相关的元素仍然具有挑战性。受人类在复杂屏幕上动态调整感知范围以定位任务相关区域这一行为的启发，我们提出 DRS-GUI，一个用于 GUI 视觉定位的免训练动态区域搜索框架，可无缝集成到现有 MLLM 中。DRS-GUI 引入一个轻量级 UI 感知器，执行三种类人感知动作（聚焦、平移与散布）来逐步探索界面并生成区域提议。为了动态调度这些动作，我们进一步设计了一个基于蒙特卡洛树搜索（MCTS）的动作规划器。我们采用区域质量奖励来评估并选择与指令高度相关的区域，从而高效剪枝冗余 UI 元素。实验表明，对于通用与 GUI 专用的 MLLM（Qwen2.5-VL-7B 与 UGround-V1-7B），DRS-GUI 在 ScreenSpot-Pro 上带来 14% 的提升，显著增强了视觉定位性能与泛化能力。
英文原文摘要 1255 字符
GUI agents powered by Multimodal Large Language Models (MLLMs) have demonstrated impressive capability in understanding and executing user instructions. However, accurately grounding instruction-relevant elements from high-resolution screenshots cluttered with irrelevant UI components remains challenging for existing approaches. Inspired by how humans dynamically adjust their perceptual scope to locate task-related regions on complex screens, we propose DRS-GUI, a training-free dynamic region search framework for GUI grounding that can be seamlessly integrated into existing MLLMs. DRS-GUI introduces a lightweight UI Perceptor that performs three human-like perceptual actions (Focus, Shift, and Scatter) to progressively explore the interface and generate region proposals. To dynamically schedule these actions, we further design an Action Planner based on Monte Carlo Tree Search (MCTS). A region quality reward is employed to evaluate and select the highly instruction-relevant region, efficiently pruning redundant UI elements. Experiments demonstrate that DRS-GUI yields a 14% improvement on ScreenSpot-Pro for general and GUI-specific MLLMs (Qwen2.5-VL-7B and UGround-V1-7B), significantly enhancing grounding performance and generalization.