从经验中学习选择视觉工具
Learning to Select Visual Tools from Experience
VisTA 将工具选择建模为强化学习问题，利用可验证奖励（RLVR）端到端训练视觉智能体，使其能根据查询动态组合工具库中的工具。对视觉研究者而言，这提供了一种无需人工标注的、经验驱动的工具编排范式，可迁移到需要多工具协同的复杂视觉推理任务（如图表理解、几何推理）。
原文依据 We introduce VisualToolAgent (VisTA), a new reinforcement learning framework that empowers visual agents to dynamically explore, select, and compose tools from a diverse library based on empirical performance.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论我们提出了 VisualToolAgent（VisTA），一种新的强化学习框架，使视觉智能体能够基于经验性能从多样化的工具库中动态探索、选择和组合工具。现有的工具增强视觉推理方法要么依赖免训练提示，要么依赖大规模监督微调；两者都缺乏主动的工具探索，并且通常假设工具多样性有限，而微调方法还需要大量的人工监督。相比之下，VisTA 利用端到端强化学习，仅以任务结果为引导，迭代地优化针对特定查询的复杂工具选择策略。借助可验证奖励的强化学习（RLVR），我们的框架使智能体能够自主发现有效的工具选择路径，而无需显式的推理监督。在 ChartQA、Geometry3K、MathVerse 和 BlindTest 基准上的实验表明，VisTA 相较于免训练和微调基线取得了显著的性能提升，尤其是在分布外样本上。这些结果凸显了 VisTA 增强泛化能力、自适应利用多样化工具的能力，并为灵活、经验驱动的视觉推理系统开辟了道路。
英文原文摘要 1276 字符
We introduce VisualToolAgent (VisTA), a new reinforcement learning framework that empowers visual agents to dynamically explore, select, and compose tools from a diverse library based on empirical performance. Existing methods for tool-augmented visual reasoning either rely on training-free prompting or large-scale supervised fine-tuning; both lack active tool exploration and typically assume limited tool diversity, and fine-tuning methods additionally demand extensive human supervision. In contrast, VisTA leverages end-to-end reinforcement learning to iteratively refine sophisticated, query-specific tool selection strategies, guided solely by task outcomes. Leveraging reinforcement learning with verifiable rewards (RLVR), our framework enables an agent to autonomously discover effective tool-selection pathways without requiring explicit reasoning supervision. Experiments on the ChartQA, Geometry3K, MathVerse, and BlindTest benchmarks demonstrate that VisTA achieves significant performance gains over training-free and fine-tuning baselines, especially on out-of-distribution examples. These results highlight VisTA's ability to enhance generalization, adaptively utilize diverse tools, and pave the way for flexible, experience-driven visual reasoning systems.