扩展智能体强化学习以提升视觉语言模型中的工具集成推理
Scaling Agentic Reinforcement Learning for Tool-Integrated Reasoning in VLMs
提供了一个可扩展的视觉智能体强化学习训练环境与标准化视觉工具接口，视觉研究者可复用其多轮轨迹采样与可验证反馈机制，训练 VLM 学会在推理中自主调用 grounding、parsing 等视觉工具，从而把「看图」变成多步交互式推理。
原文依据 VISTA-Gym unifies diverse real-world multimodal reasoning tasks (7 tasks from 13 datasets in total) with a standardized interface for visual tools
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论尽管近期的视觉语言模型（VLM）展现出强大的图像理解能力，但其「以图像思考」的能力，即通过多步视觉交互进行推理的能力，仍然有限。我们提出 VISTA-Gym，一个可扩展的训练环境，用于激励 VLM 的工具集成视觉推理能力。VISTA-Gym 将多样的真实世界多模态推理任务（共 13 个数据集中的 7 项任务）与视觉工具（如定位、解析）的标准化接口、可执行的交互循环、可验证的反馈信号以及高效的轨迹记录统一起来，从而支持大规模视觉智能体强化学习。尽管近期的 VLM 展现出强大的纯文本推理能力，但无论是闭源还是开源模型，在工具选择、调用与协调方面仍存在困难。借助 VISTA-Gym，我们通过多轮轨迹采样与端到端强化学习训练 VISTA-R1，使其将工具使用与智能体推理交错进行。在 11 个公开的推理密集型 VQA 基准上的大量实验表明，VISTA-R1-8B 以 9.51%–18.72% 的优势超越同等规模的最优基线，证明 VISTA-Gym 是释放 VLM 工具集成推理能力的有效训练场。
英文原文摘要 1251 字符
While recent vision-language models (VLMs) demonstrate strong image understanding, their ability to "think with images," i.e., to reason through multi-step visual interactions, remains limited. We introduce VISTA-Gym, a scalable training environment for incentivizing tool-integrated visual reasoning capabilities in VLMs. VISTA-Gym unifies diverse real-world multimodal reasoning tasks (7 tasks from 13 datasets in total) with a standardized interface for visual tools (e.g., grounding, parsing), executable interaction loops, verifiable feedback signals, and efficient trajectory logging, enabling visual agentic reinforcement learning at scale. While recent VLMs exhibit strong text-only reasoning, both proprietary and open-source models still struggle with tool selection, invocation, and coordination. With VISTA-Gym, we train VISTA-R1 to interleave tool-use with agentic reasoning via multi-turn trajectory sampling and end-to-end reinforcement learning. Extensive experiments across 11 public reasoning-intensive VQA benchmarks show that VISTA-R1-8B outperforms state-of-the-art baselines with similar sizes by 9.51%-18.72%, demonstrating VISTA-Gym as an effective training ground to unlock the tool-integrated reasoning capabilities for VLMs.