WeDetect：将快速开放词表目标检测视为检索
WeDetect: Fast Open-Vocabulary Object Detection as Retrieval
将开放词表检测视为检索问题，采用双塔架构避免跨模态融合层，实现实时推理；其冻结检测器并微调对象性提示词生成通用提案的思路，可复用于需要快速回溯历史数据的场景。
原文依据 Methods without cross-modal fusion layers (non-fusion) offer faster inference by treating recognition as a retrieval problem
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论开放词表目标检测旨在通过文本提示词检测任意类别。无跨模态融合层的方法（非融合）将识别视为检索问题，即在共享嵌入空间中将区域与文本查询匹配，从而提供更快的推理速度。在本工作中，我们充分探索了这一检索理念，并通过一个名为 WeDetect 的模型家族展示了其在效率和多功能性方面的独特优势：（1）最优性能。WeDetect 是一个具有双塔架构的实时检测器。我们表明，凭借精心整理的数据和充分的训练，非融合的 WeDetect 超越了其他融合模型，并建立了强大的开放词表基础。（2）历史数据的快速回溯。WeDetect-Uni 是基于 WeDetect 的通用提案生成器。我们冻结整个检测器，仅微调一个对象性提示词，以跨类别检索通用目标提案。重要的是，提案嵌入是类别特定的，并支持一种新应用——目标检索，支持在历史数据中检索目标。（3）与 LMM 集成用于指代表达理解（REC）。我们进一步提出 WeDetect-Ref，一个基于 LMM 的目标分类器，用于处理复杂的指代表达，它从 WeDetect-Uni 提取的提案列表中检索目标。它摒弃了下一词元预测，并在单次前向传播中对目标进行分类。总之，WeDetect 家族在一个连贯的检索框架下统一了检测、提案生成、目标检索和 REC，在 15 个基准上实现了最优性能，并具有高推理效率。我们将开源所有模型。
英文原文摘要 1685 字符
Open-vocabulary object detection aims to detect arbitrary classes via text prompts. Methods without cross-modal fusion layers (non-fusion) offer faster inference by treating recognition as a retrieval problem, i.e., matching regions to text queries in a shared embedding space. In this work, we fully explore this retrieval philosophy and demonstrate its unique advantages in efficiency and versatility through a model family named WeDetect: (1) State-of-the-art performance. WeDetect is a real-time detector with a dual-tower architecture. We show that, with well-curated data and full training, the non-fusion WeDetect surpasses other fusion models and establishes a strong open-vocabulary foundation. (2) Fast backtrack of historical data. WeDetect-Uni is a universal proposal generator based on WeDetect. We freeze the entire detector and only finetune an objectness prompt to retrieve generic object proposals across categories. Importantly, the proposal embeddings are class-specific and enable a new application, object retrieval, supporting retrieval objects in historical data. (3) Integration with LMMs for referring expression comprehension (REC). We further propose WeDetect-Ref, an LMM-based object classifier to handle complex referring expressions, which retrieves target objects from the proposal list extracted by WeDetect-Uni. It discards next-token prediction and classifies objects in a single forward pass. Together, the WeDetect family unifies detection, proposal generation, object retrieval, and REC under a coherent retrieval framework, achieving state-of-the-art performance across 15 benchmarks with high inference efficiency. We will open-source all models.