强化学习与具身智能 · CVPR2026 ·2026.06.03

GUI-SAGE:以自解释学习增强 GUI 自动化

GUI-SAGE: Enhancing GUI Automation with Self-Explanatory Learning

Fei Tang、Zhangxuan Gu、Zhengxi Lu、Shangzhan Zhang、Zhengwen Zeng、Shuheng Shen 等 12 人

与你有关的部分

对做视觉智能体(GUI/具身)的人,这篇给出一个可复用思路:用真值动作条件化生成同分布推理轨迹,替代分布外专家示范,避免熵坍缩;配合熵调制的信用分配,把置信度与奖励联合加权,可直接迁移到视觉-语言-动作策略的 RL 微调。

原文依据 generates in-distribution reasoning trajectories for GUI automation
可迁移到 GUI 自动化视觉-语言-动作策略学习多模态智能体

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

带可验证奖励的强化学习(RLVR)在 GUI 自动化上展现出潜力,使智能体能够从二值化的任务完成信号中学习。然而,当任务难度超出模型能力时,同策略探索无法发现正确动作,形成零优势陷阱,从而消除学习信号。引入离策略专家示范看似直观,但会因分布不匹配导致持续的高熵状态,破坏有效学习。我们提出 GUI-SAGE,一个自解释框架,为 GUI 自动化生成同分布的推理轨迹。通过以真值动作为条件,本方法产生同分布的引导,避免分布外专家示范带来的混乱。我们进一步引入熵调制信用分配,联合考虑预测置信度与奖励信号来重新校准学习权重,从而放大对高置信度正确动作的更新、衰减对不确定探索的更新。在 AndroidControl 与 GUI-Odyssey 上的大量实验表明,GUI-SAGE-3B 以 81.1% 的成功率取得有竞争力的性能,显著优于现有方法。我们的分析验证了自解释能维持稳定的学习动态,而专家示范会导致熵坍缩,并且熵调制在同分布样本上带来最大幅度的提升。

英文原文摘要 1466 字符

Reinforcement learning with verifiable rewards (RLVR) has shown promise for GUI automation, enabling agents to learn from binary task completion signals. However, when task difficulty exceeds model capacity, on-policy exploration fails to discover correct actions, creating zero-advantage traps that eliminate learning signals. While incorporating off-policy expert demonstrations seems intuitive, it causes persistent high-entropy states due to distribution mismatch, disrupting effective learning. We propose GUI-SAGE, a self-explanation framework that generates in-distribution reasoning trajectories for GUI automation. By conditioning on ground-truth actions, our method produces in-distribution guidance that avoids the confusion caused by out-of-distribution expert demonstrations. We further introduce Entropy-Modulated Credit Assignment, which recalibrates learning weights by jointly considering prediction confidence and reward signals, enabling amplified updates for confident correct actions and attenuated updates for uncertain explorations. Extensive experiments on AndroidControl and GUI-Odyssey demonstrate that GUI-SAGE-3B achieves competitive performance with 81.1% success rate, substantially outperforming existing methods. Our analysis validates that self-explanations maintain stable learning dynamics while expert demonstrations cause entropy collapse, and that entropy modulation provides the largest improvements on in-distribution samples.

标签

会议 CVPR2026
页码 13007-13016
Bibkey
来源 CVF Open Access