BAMI：GUI 视觉定位中的免训练偏差缓解
BAMI: Training-Free Bias Mitigation in GUI Grounding
针对高分辨率 GUI 截图上视觉定位的精度偏差与歧义偏差，提出免训练的推理期干预（由粗到细聚焦 + 候选选择），可直接套用到任意 VLM 的坐标预测环节，无需重训即可提升定位精度。
原文依据 we introduce Bias-Aware Manipulation Inference (BAMI), which incorporates two key manipulations, coarse-to-fine focus and candidate selection
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论GUI 视觉定位是使 GUI 智能体能够执行点击、拖拽等任务的关键能力。然而，在 ScreenSpot-Pro 基准等复杂场景下，现有模型往往表现欠佳。借助所提出的掩码预测分布（Masked Prediction Distribution, MPD）归因方法，我们发现错误主要来自两方面：高图像分辨率（导致精度偏差）与复杂界面元素（导致歧义偏差）。为应对这些挑战，我们提出偏差感知操作推理（Bias-Aware Manipulation Inference, BAMI），其中包含由粗到细聚焦与候选选择两项关键操作，以有效缓解上述偏差。大量实验结果表明，BAMI 在免训练设定下显著提升了多种 GUI 视觉定位模型的准确率。例如，将本方法应用于 TianXi-Action-7B 模型后，其在 ScreenSpot-Pro 基准上的准确率从 51.9% 提升至 57.8%。此外，消融实验证实 BAMI 方法在不同参数配置下均具有鲁棒性，凸显了其稳定性与有效性。
英文原文摘要 1148 字符
GUI grounding is a critical capability for enabling GUI agents to execute tasks such as clicking and dragging. However, in complex scenarios like the ScreenSpot-Pro benchmark, existing models often suffer from suboptimal performance. Utilizing the proposed Masked Prediction Distribution (MPD) attribution method, we identify that the primary sources of errors are twofold: high image resolution (leading to precision bias) and intricate interface elements (resulting in ambiguity bias). To address these challenges, we introduce Bias-Aware Manipulation Inference (BAMI), which incorporates two key manipulations, coarse-to-fine focus and candidate selection, to effectively mitigate these biases. Our extensive experimental results demonstrate that BAMI significantly enhances the accuracy of various GUI grounding models in a training-free setting. For instance, applying our method to the TianXi-Action-7B model boosts its accuracy on the ScreenSpot-Pro benchmark from 51.9% to 57.8%. Furthermore, ablation studies confirm the robustness of the BAMI approach across diverse parameter configurations, highlighting its stability and effectiveness.