照亮通用多模态嵌入中的视觉身份
Illuminating Visual Identity in Universal Multimodal Embeddings
提出身份感知采样机制，在通用多模态嵌入中联合优化视觉身份表示，可直接用于实例检索、重识别和生成内容身份保持等视觉任务。
原文依据 we propose a unified formulation for visual identity discrimination (VisID) and introduce MVEB
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论通用多模态嵌入（UMEs）旨在将各种模态和任务统一到一个共享表示空间中。近年来，在多模态大语言模型（MLLMs）发展的推动下，该领域取得了实质性进展。然而，现有 UME 方法仍未充分探索一项关键能力——视觉身份判别，尽管它在实例检索、重识别以及 AI 生成内容中的身份保持等广泛任务中起着至关重要的作用。为弥补这一空白，我们提出了视觉身份判别（VisID）的统一形式化框架，并引入了 MVEB（多模态视觉身份嵌入基准），这是一个从真实世界和合成数据集中整理的大规模基准，用于支持评估和训练。此外，我们提出了一个简单而有效的学习框架，通过精心设计的身份感知采样机制，联合优化通用多模态表示和视觉身份表示。大量实验表明，我们的方法成功赋予了 UME 强大的身份判别能力，并保持了有竞争力的通用多模态性能。我们相信这项工作不仅阐明了一项关键但被忽视的能力，而且朝着更全面的通用多模态嵌入迈出了一步。代码和数据可在 https://chrisclear3.github.io/MVEB 获取。
英文原文摘要 1413 字符
Universal Multimodal Embeddings (UMEs) aim to unify various modalities and tasks into a shared representation space. In recent years, this field has witnessed substantial progress driven by the development of Multimodal Large Language Models (MLLMs). However, a crucial capability, visual identity discrimination, remains underexplored in existing UME methods, despite its critical role in a wide range of tasks, including instance retrieval, re-identification, and identity preservation in AI-generated content. To bridge this gap, we propose a unified formulation for visual identity discrimination (VisID) and introduce MVEB (Multimodal Visual Identity Embedding Benchmark), a large-scale benchmark curated from both real-world and synthetic datasets to support evaluation and training. Furthermore, we present a simple yet effective learning framework that jointly optimizes general multimodal and visual identity representations through a carefully designed identity-aware sampling mechanism. Extensive experiments demonstrate that our approach successfully endows UMEs with strong identity discrimination capability and maintains competitive general multimodal performance. We believe this work not only illuminates a critical yet neglected capability, but also takes a step toward more holistic universal multimodal embeddings. Code and data are available at \href https://chrisclear3.github.io/MVEB MVEB .