基于执行反馈强化学习训练高层调度器用于长程GUI自动化
Training High-Level Schedulers with Execution-Feedback Reinforcement Learning for Long-Horizon GUI Automation
提出了一种基于执行反馈强化学习训练高层调度器（Coordinator和State Tracker）的方法，可即插即用地增强任意低层执行器在长程GUI自动化任务中的规划与状态管理能力，为视觉语言模型在复杂交互任务中的多智能体协同提供了可复用的训练范式。
原文依据 we propose a staged execution-feedback reinforcement learning algorithm
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论大型视觉语言模型（VLM）的快速发展极大地推动了GUI智能体的研究。然而，GUI智能体在处理长程任务时仍面临重大挑战。首先，单智能体模型难以平衡高层能力与低层执行能力，普遍存在职责耦合和能力冲突的问题。其次，智能体缺乏对任务状态的感知，导致长程任务中的进度丢失。为了解决这些挑战，我们提出了一种分阶段的执行反馈强化学习算法。与训练统一的策略模型不同，我们专注于训练高层调度模型。具体而言，我们提出并训练了两个智能体：一个协调器，负责战略规划和任务分解；一个状态跟踪器，负责上下文压缩和信息管理，以维护任务的状态和连贯性。在此基础上，我们构建了协调器-执行器-状态跟踪器（CES）多智能体框架，该框架可以与任何低层执行器模型集成，通过任务调度和状态管理帮助执行器解决长程任务。在长程任务基准上的实验表明，CES显著增强了系统的规划和状态管理能力。此外，分析证实，我们训练的高层调度模块是一个可泛化、即插即用的模块，能够显著增强各种执行器的长程能力。
英文原文摘要 1512 字符
The rapid development of large vision-language model (VLM) has greatly promoted the research of GUI agent. However, GUI agents still face significant challenges in handling long-horizon tasks. First, single-agent models struggle to balance high-level capabilities and low-level execution capability, facing prevalent issues of responsibility coupling and capability conflicts. Second, agents lack awareness of the task state, leading to progress loss in long-horizon tasks. To address these challenges, we propose a staged execution-feedback reinforcement learning algorithm. Unlike training a unified policy model, we focus on training high-level scheduling models. Specifically, we propose and train two agents: a Coordinator, responsible for the strategic planning and task decomposition; and a State Tracker, responsible for context compression and information management to maintain the task's state and coherence. Based on this, we built the Coordinator-Executor-State Tracker (CES) multi-agent framework, which can be integrated with any low-level Executor model, assisting the Executor in solving long-horizon tasks through task scheduling and state management. Experiments on long-horizon task benchmarks demonstrate that CES significantly enhances the system's planning and state management capabilities. Furthermore, analysis confirms that our trained high-level scheduling module is a generalizable, plug-and-play module that significantly enhances the long-horizon capabilities of various Executors.