超越全局相似度：面向细粒度跨模态理解的多条件检索
Beyond Global Similarity: Multi-Conditional Retrieval for Fine-Grained Cross-Modal Understanding
为视觉研究者提供了一个诊断性基准，用于检验多模态检索器与视觉语言重排序器在自然语言多约束查询下的细粒度组合匹配能力；其发现（视觉线索主导早期排序精度、文本元数据稳定长尾排序、pointwise 重排序器通过显式验证查询-候选一致性显著提升匹配）可直接指导检索系统的重排序模块设计与多模态特征融合策略。
原文依据 visual cues dominate early-rank precision, while textual metadata stabilizes long-tail ordering
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多模态大语言模型（MLLM）的最新进展大幅拓展了多模态检索的能力，使系统能够在视觉与文本模态之间对齐并检索信息。然而，现有基准大多聚焦于粗粒度或单条件对齐，忽视了用户查询会跨模态指定多个相互依赖约束的真实场景。为弥补这一空白，我们提出 MCMR（多条件多模态检索）：一个大规模基准，旨在评估自然语言查询下的细粒度、多条件跨模态检索。MCMR 涵盖五个产品领域：上装与下装、珠宝、鞋履和家具，并保留了组合式匹配所必需的长篇富元数据。每个查询整合互补的视觉与文本属性，要求模型同时满足所有指定条件才算相关。我们对多种基于 MLLM 的多模态检索器与视觉语言重排序器进行了基准评测，以考察其条件感知推理能力。实验结果表明：(i) 不同模型之间存在明显的模态不对称性；(ii) 视觉线索主导早期排序精度，而文本元数据稳定长尾排序；(iii) 基于 MLLM 的逐点重排序器通过显式验证查询-候选一致性，显著提升细粒度匹配。总体而言，MCMR 为推进多模态检索走向组合式、约束感知且可解释的理解，建立了一个具有挑战性的诊断性基准。我们的代码与数据集见 https://github.com/EIT-NLP/MCMR
英文原文摘要 1647 字符
Recent advances in multimodal large language models (MLLMs) have substantially expanded the capabilities of multimodal retrieval, enabling systems to align and retrieve information across visual and textual modalities. Yet, existing benchmarks largely focus on coarse-grained or single-condition alignment, overlooking real-world scenarios where user queries specify multiple interdependent constraints across modalities. To bridge this gap, we introduce MCMR (Multi-Conditional Multimodal Retrieval): a large-scale benchmark designed to evaluate fine-grained, multi-condition cross-modal retrieval under natural-language queries. MCMR spans five product domains: upper and bottom clothing, jewelry, shoes, and furniture. It also preserves rich long-form metadata essential for compositional matching. Each query integrates complementary visual and textual attributes, requiring models to jointly satisfy all specified conditions for relevance. We benchmark a diverse suite of MLLM-based multimodal retrievers and vision-language rerankers to assess their condition-aware reasoning abilities. Experimental results reveal: (i) distinct modality asymmetries across models; (ii) visual cues dominate early-rank precision, while textual metadata stabilizes long-tail ordering; and (iii) MLLM-based pointwise rerankers markedly improve fine-grained matching by explicitly verifying query-candidate consistency. Overall, MCMR establishes a challenging and diagnostic benchmark for advancing multimodal retrieval toward compositional, constraint-aware, and interpretable understanding. Our code and dataset is available at https://github.com/EIT-NLP/MCMR