像专家一样观察：面向开放集细粒度视觉理解的知识增强智能体
Seeing as Experts Do: A Knowledge-Augmented Agent for Open-Set Fine-Grained Visual Understanding
将开放词表检测与网络级检索结合生成类别假设，并通过全局到局部的聚焦机制将检索到的文本知识转化为空间定位证据，为视觉语言模型提供可验证的细粒度推理依据。
原文依据 It first performs open-vocabulary detection and web-scale retrieval to generate category hypotheses.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论细粒度视觉理解正从静态分类转向知识增强推理，模型不仅需要识别，还需要给出理由。现有方法受限于闭集分类体系和单标签预测，在开放集或依赖上下文的条件下性能显著下降。我们提出知识增强细粒度推理智能体（KFRA），一个将细粒度感知转化为证据驱动推理的统一框架。KFRA通过模拟专家分析的三阶段闭环推理运行：首先执行开放词表检测和网络级检索以生成类别假设；然后通过全局到局部的聚焦机制，将文本知识与视觉证据对齐，进行判别性区域定位；最后在大规模多模态模型中整合所有多模态证据以执行可解释推理。与将检索和推理视为独立过程的现有智能体不同，KFRA建立了检索-定位耦合，将检索到的知识转化为空间定位的证据以供验证。该设计使得在多样细粒度场景中实现事实性、可解释且任务无关的推理成为可能。为评估这一能力，我们构建了FGExpertBench，一个旨在评估六个知识维度上推理深度和跨任务泛化能力的基准。大量实验表明，KFRA持续超越独立的大规模多模态模型和当前智能体框架，在推理准确率上最高提升19%，并在开放集细粒度视觉理解中提供基于证据的可解释性。
英文原文摘要 1735 字符
Fine-grained visual understanding is shifting from static classification to knowledge-augmented reasoning, where models must justify as well as recognise. Existing approaches remain limited by closed-set taxonomies and single-label prediction, leading to significant degradation under open-set or context-dependent conditions. We present the Knowledge-Augmented Fine-Grained Reasoning Agent (KFRA), a unified framework that transforms fine-grained perception into evidence-driven reasoning. KFRA operates through a three-stage closed reasoning loop that emulates expert analysis. It first performs open-vocabulary detection and web-scale retrieval to generate category hypotheses. It then conducts discriminative regions localisation by aligning textual knowledge with visual evidence through a global-to-local focusing mechanism. Finally, it integrates all multimodal evidence within a large multimodal model to perform interpretable reasoning. Unlike existing agents that treat retrieval and reasoning as independent processes, KFRA establishes a retrieval-grounding coupling that converts retrieved knowledge into spatially grounded evidence for verification. This design enables factual, interpretable, and task-agnostic reasoning across diverse fine-grained scenarios. To evaluate this capability, we construct FGExpertBench, a benchmark designed to assess reasoning depth and cross-task generalisation across six knowledge dimensions. Extensive experiments demonstrate that KFRA consistently surpasses both standalone large multimodal models and current agent frameworks, achieving up to 19 percent improvement in reasoning accuracy and delivering evidence-grounded interpretability in open-set fine-grained visual understanding.