SpaceTools：通过双重交互式强化学习实现工具增强的空间推理
SpaceTools: Tool-Augmented Spatial Reasoning via Double Interactive RL
提出双阶段交互式强化学习框架，让视觉语言模型通过探索和反馈学习协调多种视觉工具（深度估计、分割、姿态估计），为视觉任务中的工具增强推理提供了可复用的训练范式。
原文依据 We introduce Double Interactive Reinforcement Learning (DIRL), a two-phase training framework where VLMs learn to coordinate multiple tools through interactive exploration and feedback.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉语言模型（VLM）展现出强大的定性视觉理解能力，但在具身应用所需的度量精确空间推理方面仍存在困难。智能体范式承诺 VLM 可以使用多种工具来增强这些能力，例如深度估计器、分割模型和姿态估计器。然而，如何实现这一愿景仍是一个开放挑战，不能仅依赖手工设计的提示策略或强制使用固定的预定义工具流水线，因为这会限制 VLM 发现最优工具使用模式的能力。强化学习有望弥合这一差距，但由于多工具推理的搜索空间巨大，迄今为止仅限于使用单一视觉工具进行推理。我们提出了双重交互式强化学习（DIRL），这是一个两阶段训练框架，VLM 通过交互式探索和反馈学习协调多个工具。在教学阶段，我们将通过交互式强化学习训练的单工具专家演示与前沿模型使用所有工具的轨迹相结合。在探索阶段，模型通过持续强化学习进一步优化多工具协调。我们的模型 SpaceTools 具备工具增强的空间推理能力，在空间理解基准（RoboSpatial-Home、BLINK、内部基准）上取得了最优性能，并展示了使用 7 自由度机器人作为工具的可靠真实世界操作。DIRL 相比原始 SFT（在 RoboSpatial 上 +12%）和 RL（在 RoboSpatial 上 +16%）基线有显著提升。项目页面：https://spacetools.github.io/。
英文原文摘要 1630 字符
Vision Language Models (VLMs) demonstrate strong qualitative visual understanding, but struggle with metrically precise spatial reasoning required for embodied applications. The agentic paradigm promises that VLMs can use a wide variety of tools that could augment these capabilities, such as depth estimators, segmentation models, and pose estimators. Yet it remains an open challenge how to realize this vision without solely relying on handcrafted prompting strategies or enforcing fixed, predefined tool pipelines that limit VLMs' ability to discover optimal tool-use patterns. Reinforcement Learning could overcome this gap, but has so far been limited to reasoning with a single visual tool due to the large search space in multi-tool reasoning. We introduce Double Interactive Reinforcement Learning (DIRL), a two-phase training framework where VLMs learn to coordinate multiple tools through interactive exploration and feedback. In the teaching phase, we combine demonstrations from a single tool specialist trained via interactive RL with traces from a frontier model using all tools. In the exploration phase, the model further refines multi-tool coordination through continued RL. Our model, SpaceTools, with tool-augmented spatial reasoning ability, achieves state-of-the-art performance on spatial understanding benchmarks (RoboSpatial-Home, BLINK, Internal Benchmark) and demonstrates reliable real-world manipulation using a 7-DOF robot as a tool. DIRL provides substantial improvements over the vanilla SFT (+12% on RoboSpatial) and RL (+16% on RoboSpatial) baselines. Project page: https://spacetools.github.io/.