RMIR：一个面向推理密集型多模态图像检索的基准数据集
RMIR: A Benchmark Dataset for Reasoning-Intensive Multimodal Image Retrieval
为视觉研究者提供了一个需要复杂推理（功能、时序、因果）的多模态图像检索基准，并揭示了生成式嵌入模型在推理感知训练下的优势，可启发设计更强的推理型视觉-语言检索模型。
原文依据 we introduce RMIR, a benchmark dataset of 1634 queries requiring reasoning across three categories: functional (object affordances), temporal (time-based relationships), and causal (cause-effect reasoning).
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论当前的多模态图像检索基准主要关注相对简单的查询，目标图像要么被直接描述，要么通过与输入图像的简单组合来描述。当检索需要复杂推理来确定目标图像时，任务变得极具挑战性，然而针对这一设定尚无标准化的基准。为填补这一空白，我们提出了RMIR，一个包含1634个查询的基准数据集，这些查询需要跨三个类别进行推理：功能性（物体可供性）、时序性（基于时间的关系）和因果性（因果推理）。每个查询结合了视觉和文本输入，要求具备稳健的视觉理解以及逻辑推断，超越表面匹配来识别正确的目标图像。除了数据集本身，我们还提出了一个生成该数据集的流程，可用于大规模生成额外的推理密集型检索数据。在RMIR上对最优模型的评估揭示了显著的性能差距，最佳模型在推理类别上平均仅达到46.53%的recall@20。我们的评估还表明，具有显式推理的生成式嵌入模型显著优于判别式方法，且推理感知训练比模型规模更具影响力。我们的系统分析揭示了当前多模态检索系统的基本局限性，并将RMIR确立为开发多模态、具备推理能力的检索模型的一个具有挑战性的测试平台。我们的数据集和代码可在https://github.com/amazon-science/rmir获取。
英文原文摘要 1597 字符
Current multimodal image retrieval benchmarks focus on relatively simple queries where target images are either described directly or by simple composition with an input image. When retrieval requires complex reasoning to determine the target image, the task becomes significantly more challenging, yet standardized benchmarks for this setting do not exist. To fill this gap, we introduce RMIR, a benchmark dataset of 1634 queries requiring reasoning across three categories: functional (object affordances), temporal (time-based relationships), and causal (cause-effect reasoning). Each query combines visual and textual inputs that demand robust visual understanding together with logical inference, beyond surface-level matching, to identify correct target images. In addition to the dataset itself, we present a pipeline to generate the dataset, which can be used to generate additional reasoning-intensive retrieval data at scale. Evaluation of state-of-the-art models on RMIR reveals significant performance gaps, with the best model achieving only 46.53% recall@20 averaged across reasoning categories. Our evaluation also shows that generative embedding models with explicit reasoning substantially outperform discriminative approaches, with reasoning-aware training proving more impactful than model scale. Our systematic analysis exposes fundamental limitations in current multimodal retrieval systems and establishes RMIR as a challenging testbed for developing multimodal, reasoning-capable retrieval models. Our dataset and code are available at https://github.com/amazon-science/rmir