SenseSearch：通过强化学习赋能视觉语言模型的高分辨率智能体搜索推理
SenseSearch: Empowering Vision-Language Models with High-Resolution Agentic Search-Reasoning via Reinforcement Learning
提出通过强化学习让 VLM 自适应协调图像搜索、文本搜索和图像裁剪工具，解决高分辨率细粒度视觉理解问题；其 BN-GSPO 算法和 HR-MMSearch 基准可直接用于多模态智能体训练与评测。
原文依据 we propose SenseSearch, the first agentic VLM for search-reasoning that supports adaptive multi-tool coordination via reinforcement learning (RL).
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉语言模型（VLM）受限于静态知识和不足的细粒度视觉分析能力，阻碍了其在知识密集型与视觉复杂任务上的表现。尽管近期研究探索了使用搜索或裁剪等外部工具的 VLM 以提升模型性能，但这些方法通常孤立地使用工具，缺乏有效协调多种工具的能力。为填补这一空白，我们提出 SenseSearch，这是首个通过强化学习（RL）支持自适应多工具协调的搜索推理智能体 VLM。具体而言，SenseSearch 动态整合图像搜索、文本搜索和图像裁剪工具，以应对细粒度和知识密集型的视觉理解挑战。我们首先构建了一个高质量的冷启动数据集，以灌输基本的工具使用行为。在随后的 RL 阶段，我们引入批归一化组序列策略优化（BN-GSPO）算法来增强工具调用和推理能力。为全面评估智能体 VLM 在复杂视觉任务上的表现，我们推出了 HR-MMSearch 基准，这是首个由高分辨率图像组成的、面向搜索的基准，包含知识密集型和搜索驱动的问题。实验表明，SenseSearch 在开源搜索和细粒度图像理解基准上达到了最优性能，在 HR-MMSearch 上超越基线 19.18%。SenseSearch 为具有有效且鲁棒工具调用能力的智能体 VLM 提供了一条有前景的路径。所有代码和数据已发布于 https://github.com/OpenSenseNova/SenseNova-MARS。
英文原文摘要 1652 字符
Vision-Language Models (VLMs) are limited by static knowledge and insufficient fine-grained visual analysis, hindering their performance on knowledge-intensive and visually complex tasks. While recent research has explored VLMs that employ external tools like search or cropping to enhance model performance, they typically employ tools in isolation and lack the ability to coordinate multiple tools effectively. To address this gap, we propose SenseSearch, the first agentic VLM for search-reasoning that supports adaptive multi-tool coordination via reinforcement learning (RL). Specifically, SenseSearch dynamically integrates the image search, text search, and image crop tools to tackle fine-grained and knowledge-intensive visual understanding challenges. We first construct a high-quality cold-start dataset to instill basic tool-usage behaviors. In the subsequent RL stage, we introduce Batch-Normalized Group Sequence Policy Optimization (BN-GSPO) algorithm to enhance the tool invocation and reasoning ability. To comprehensively evaluate the agentic VLMs on complex visual tasks, we introduce the HR-MMSearch benchmark, the first search-oriented benchmark composed of high-resolution images with knowledge-intensive and search-driven questions. Experiments demonstrate that SenseSearch achieves state-of-the-art performance on open-source search and fine-grained image understanding benchmarks, outperforming baselines by 19.18% on HR-MMSearch. SenseSearch provides a promising path toward agentic VLMs with effective and robust tool invocation capabilities. All code and data are released at https://github.com/OpenSenseNova/SenseNova-MARS.