Air-Know:仲裁器校准的知识内化鲁棒网络用于组合图像检索
Air-Know: Arbiter-Calibrated Knowledge-Internalizing Robust Network for Composed Image Retrieval
用 MLLM 作为离线专家构造高精度锚框数据集,再把判别逻辑内化到轻量代理仲裁器中,为视觉-语言检索中处理噪声对应关系提供了可复用的「专家-代理-分流」解耦范式。
原文依据 utilizes Multimodal Large Language Models (MLLMs) as an offline expert to construct a high precision anchor dataset
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论组合图像检索(CIR)因其灵活的多模态查询方式而受到广泛关注,但其发展受到噪声三元组对应(NTC)问题的严重制约。现有大多数鲁棒学习方法依赖「小损失假设」,然而 NTC 中特有的语义歧义(例如「部分匹配」)使该假设失效,导致噪声识别不可靠。这使模型陷入一种自我依赖的恶性循环:学习器与仲裁器相互纠缠,最终造成灾难性的「表征污染」。为应对这一关键挑战,我们提出一种新颖的「专家-代理-分流」解耦范式,命名为 Air-Know(仲裁器校准的知识内化鲁棒网络)。Air-Know 包含三个核心模块:(1)外部先验仲裁(EPA)模块,利用多模态大语言模型(MLLM)作为离线专家,构建高精度锚框数据集;(2)专家知识内化(EKI)模块,高效引导轻量代理「仲裁器」内化专家的判别逻辑;(3)双流调和(DSR)模块,利用 EKI 的匹配置信度对训练数据进行分流,形成干净对齐流与表征反馈调和流。在多个 CIR 基准数据集上的大量实验表明,Air-Know 在 NTC 设定下显著优于现有 SOTA 方法,同时在传统 CIR 上也展现出很强的竞争力。
英文原文摘要 1563 字符
Composed Image Retrieval (CIR) has attracted significant attention due to its flexible multimodal query method, yet its development is severely constrained by the Noisy Triplet Correspondence (NTC) problem. Most existing robust learning methods rely on the "small loss hypothesis", but the unique semantic ambiguity in NTC, such as "partial matching", invalidates this assumption, leading to unreliable noise identification. This entraps the model in a self dependent vicious cycle where the learner is intertwined with the arbiter, ultimately causing catastrophic "representation pollution". To address this critical challenge, we propose a novel "Expert-Proxy-Diversion" decoupling paradigm, named Air-Know (ArbIteR calibrated Knowledge iNternalizing rObust netWork). Air-Know incorporates three core modules: (1) The External Prior Arbitration (EPA) module, which utilizes Multimodal Large Language Models (MLLMs) as an offline expert to construct a high precision anchor dataset; (2) Expert Knowledge Internalization (EKI) module, which efficiently guides a lightweight proxy "arbiter" to internalize the expert's discriminative logic; (3) Dual Stream Reconciliation (DSR) module, which leverages the EKI's matching confidence to divert the training data, achieving a clean alignment stream and a representation feedback reconciliation stream. Extensive experiments on multiple CIR benchmark datasets demonstrate that Air-Know significantly outperforms existing SOTA methods under the NTC setting, while also showing strong competitiveness in traditional CIR.