CodeMMR：统一自然语言、代码与图像的统一检索
CodeMMR: Bridging Natural Language, Code, and Image for Unified Retrieval
把图像（网页界面、数据可视化、SVG、示意图、UML）与自然语言、代码嵌入同一语义空间做跨模态检索，其指令驱动的多模态对齐与统一嵌入思路可直接迁移到视觉侧的图文检索、视觉定位与多模态 RAG 场景。
原文依据 a unified retrieval model that jointly embeds natural language, code, and images into a shared semantic space through instruction-based multimodal alignment
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论代码检索作为信息检索（IR）问题，支撑着现代软件工程，并日益为检索增强生成（RAG）提供动力，从而改善代码发现、复用以及基于大语言模型的编程的可靠性。然而，现有代码信息检索模型在很大程度上仍以文本为中心，往往忽略了编程产物中固有的视觉与结构信息，例如网页界面、数据可视化、SVG、示意图和 UML。为弥合这一差距，我们提出了 MMCoIR，这是首个用于评估跨五个视觉领域的多模态代码信息检索的综合基准，并通过大量评测表明该任务具有挑战性。因此，我们随后提出了 CodeMMR，一个统一检索模型，通过基于指令的多模态对齐将自然语言、代码和图像联合嵌入到共享语义空间中。CodeMMR 在跨模态和跨语言上实现了强泛化能力，在 nDCG@10 上平均超越有竞争力的基线（如 UniIR、GME、VLM2Vec）10 个点。此外，将 CodeMMR 集成到 RAG 中可提升代码生成保真度以及在未见代码生成任务上的视觉定位能力，凸显了多模态检索作为下一代智能编程系统核心使能因素的潜力。
英文原文摘要 1260 字符
Code search, framed as information retrieval (IR), underpins modern software engineering and increasingly powers retrieval-augmented generation (RAG), improving code discovery, reuse, and the reliability of LLM-based coding.Yet existing code IR models remain largely text-centric and often overlook the visual and structural aspects inherent in programming artifacts such as web interfaces, data visualizations, SVGs, schematic diagrams, and UML.To bridge this gap, we introduce MMCoIR, the first comprehensive benchmark for evaluating multimodal code IR across five visual domains, and show through extensive evaluation the task is challenging.Therefore, we then propose CodeMMR, a unified retrieval model that jointly embeds natural language, code, and images into a shared semantic space through instruction-based multimodal alignment.CodeMMR achieves strong generalization across modalities and languages, outperforming competitive baselines (e.g., UniIR, GME, VLM2Vec) by an average of 10 points on nDCG@10.Moreover, integrating CodeMMR into RAG enhances code generation fidelity and visual grounding on unseen code generation tasks, underscoring the potential of multimodal retrieval as a core enabler for next-generation intelligent programming systems.