ReCALL：为基于多模态大模型的组合图像检索重新校准能力退化
ReCALL: Recalibrating Capability Degradation for MLLM-based Composed Image Retrieval
把生成式 MLLM 适配为判别式检索器时会出现细粒度推理能力退化，ReCALL 用「自引导难例挖掘 + MLLM 生成纠正指令与三元组 + 分组对比持续训练」的 diagnose-generate-refine 流程把退化能力拉回来；做多模态检索或把 MLLM 蒸馏成嵌入模型的工作可直接复用这套自举数据生成与一致性过滤机制。
原文依据 compressing a generative MLLM into a single-embedding discriminative retriever triggers a paradigm conflict, which leads to Capability Degradation
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论组合图像检索（CIR）旨在根据由参考图像和修改文本构成的混合查询检索目标图像。早期的双塔视觉语言模型（VLM）难以完成该任务所需的跨模态组合推理。将生成式多模态大语言模型（MLLM）适配用于检索是一个有前景的方向，但我们发现该策略忽视了一个根本问题：将生成式 MLLM 压缩为单一嵌入的判别式检索器会引发范式冲突，进而导致能力退化，即检索适配之后原生细粒度推理能力的下降。为应对这一挑战，我们提出 ReCALL，一个与模型无关的框架，遵循「诊断—生成—精炼」的流程：首先，通过自引导的信息性实例挖掘来诊断检索器的认知盲点；接着，通过提示基础 MLLM 生成纠正性指令与三元组，并借助基于 VQA 的一致性过滤进行质量控制；最后，利用分组对比方案在这些三元组上持续训练以精炼检索器，从而内化细粒度的视觉—语义区分能力，并使检索器的判别式嵌入空间与 MLLM 内在的组合推理重新对齐。在 CIRR 和 FashionIQ 上的大量实验表明，ReCALL 能够持续校准退化的能力并达到最优性能。代码已发布于 https://github.com/RemRico/Recall。
英文原文摘要 1547 字符
Composed Image Retrieval (CIR) aims to retrieve target images based on a hybrid query comprising a reference image and a modification text. Early dual-tower Vision-Language Models (VLMs) struggle with cross-modality compositional reasoning required for this task. While adapting generative Multimodal Large Language Models (MLLMs) for retrieval offers a promising direction, we identify that this strategy overlooks a fundamental issue: compressing a generative MLLM into a single-embedding discriminative retriever triggers a paradigm conflict, which leads to Capability Degradation--the deterioration of native fine-grained reasoning after retrieval adaptation. To address this challenge, we propose ReCALL, a model-agnostic framework that follows a diagnose-generate-refine pipeline: First, we diagnose cognitive blind spots of the retriever via self-guided informative instance mining. Next, we generate corrective instructions and triplets by prompting the foundation MLLM and conduct quality control with VQA-based consistency filtering. Finally, we refine the retriever through continual training on these triplets with a grouped contrastive scheme, thereby internalizing fine-grained visual-semantic distinctions and realigning the discriminative embedding space of retriever with intrinsic compositional reasoning within the MLLM. Extensive experiments on CIRR and FashionIQ show that ReCALL consistently recalibrates degraded capabilities and achieves state-of-the-art performance. Code is available at https://github.com/RemRico/Recall.