Evo-Retriever：面向多模态文档检索的视角—路径协同 LLM 引导课程演化
Evo-Retriever: LLM-Guided Curriculum Evolution with Viewpoint-Pathway Collaboration for Multimodal Document Retrieval
面向视觉文档检索，提出多视角图像对齐与双向对比学习构造难负例，并用 LLM 元控制器按模型状态动态调整训练课程；做跨模态检索或图文匹配时，可借鉴其难例挖掘与课程调度思路来缓解跨模态嵌入混淆。
原文依据 a bidirectional contrastive learning strategy generates "hard queries" and establishes complementary learning paths
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论视觉语言模型（VLM）擅长数据映射，但真实世界文档的异构性与非结构化特性破坏了跨模态嵌入的一致性。近期的晚期交互方法通过多向量表示增强了图文对齐，然而在有限样本与静态策略下的传统训练无法适应模型的动态演化，导致跨模态检索混淆。为克服这一问题，我们提出 Evo-Retriever，一个以新颖的视角—路径协同为基础、具备 LLM 引导课程演化的检索框架。首先，我们采用多视角图像对齐，通过多尺度与多方向视角增强细粒度匹配。随后，双向对比学习策略生成“难查询”，并为视觉与文本消歧建立互补的学习路径，以重新平衡监督信号。最后，上述协同得到的模型状态摘要被送入 LLM 元控制器，由后者利用专家知识自适应地调整训练课程，以促进模型演化。在 ViDoRe V2 与 MMEB（VisDoc）上，Evo-Retriever 取得最优性能，nDCG@5 分数分别为 65.2% 与 77.1%。
英文原文摘要 1228 字符
Visual-language models (VLMs) excel at data mappings, but real-world document heterogeneity and unstructuredness disrupt the consistency of cross-modal embeddings. Recent late-interaction methods enhance image-text alignment through multi-vector representations, yet traditional training with limited samples and static strategies cannot adapt to the model's dynamic evolution, causing cross-modal retrieval confusion. To overcome this, we introduce Evo-Retriever, a retrieval framework featuring an LLM-guided curriculum evolution built upon a novel Viewpoint-Pathway collaboration. First, we employ multi-view image alignment to enhance fine-grained matching via multi-scale and multi-directional perspectives. Then, a bidirectional contrastive learning strategy generates "hard queries" and establishes complementary learning paths for visual and textual disambiguation to rebalance supervision. Finally, the model-state summary from the above collaboration is fed into an LLM meta-controller, which adaptively adjusts the training curriculum using expert knowledge to promote the model's evolution. On ViDoRe V2 and MMEB (VisDoc), Evo-Retriever achieves state-of-the-art performance, with nDCG@5 scores of 65.2 % and 77.1 %.