观看并学习：从在线视频中学习使用计算机
Watch and Learn: Learning to Use Computers from Online Videos
把「从连续屏幕状态反推用户动作」的逆动力学建模思路，可直接迁移到视觉示教学习与视频到动作的标注流程中，为从无标注视频自动构造可执行轨迹提供了一套可复用的检索+标注管线。
原文依据 we cast trajectory annotation as an inverse dynamics problem that predicts user actions from consecutive screen states
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论计算机使用智能体（CUA）必须跨多样且不断演进的应用规划任务工作流，然而大规模、高质量训练数据的缺乏限制了进展。现有数据集范围狭窄、静态且标注成本高昂，而合成数据往往产生过于简化或与真实情况错位的行为。本文提出 Watch & Learn（W&L）框架，将互联网上易得的人类使用计算机的视频大规模转换为可执行的 UI 轨迹。该方法不直接生成动作，也不依赖手工设计的启发式规则，而是将轨迹标注建模为逆动力学问题，即从连续的屏幕状态预测用户动作，这简化了学习并具备跨领域泛化能力。通过任务感知的检索与标注流程，W&L 产出超过 53K 条高质量轨迹，这些轨迹既能作为上下文示例，也能作为监督训练数据来增强 CUA。在 OSWorld 上，它持续提升通用型与专用型 CUA 的表现；在 WindowsAgentArena 上，它在 15 步限制下于 7B 规模模型中达到最优性能。这些结果表明，网络规模的人类演示视频可作为推进真实世界 CUA 的实用且可扩展的基础。
英文原文摘要 1233 字符
Computer-using agents (CUAs) must plan task workflows across diverse and evolving applications, yet progress is limited by the lack of large-scale, high-quality training data. Existing datasets are narrow, static, and costly to annotate, while synthetic data often yields oversimplified or misaligned behaviors. We present Watch & Learn (W&L), a framework that converts readily available Internet videos of human computer use into executable UI trajectories at scale. Instead of directly generating actions or relying on handcrafted heuristics, we cast trajectory annotation as an inverse dynamics problem that predicts user actions from consecutive screen states, which simplifies learning and generalizes across domains. Through a task-aware retrieval and labeling pipeline, W&L yields over 53K high-quality trajectories that enhance CUAs both as in-context exemplars and as supervised training data. On OSWorld, it consistently improves general-purpose and specialized CUAs, while on WindowsAgentArena it achieves state-of-the-art performance among 7B-scale models under the 15-step limit. These results show that web-scale human demonstration videos can serve as a practical and scalable foundation for advancing real-world CUAs.