Ego：视觉语言模型的嵌入引导个性化
Ego: Embedding-Guided Personalization of Vision-Language Models
利用 VLM 内部注意力提取代表目标概念的视觉 token 作为概念记忆，无需额外训练即可在测试图像中召回并描述个性化概念，为视觉语言模型的免训练个性化与概念绑定提供可复用思路。
原文依据 we extract visual tokens that predominantly represent the target concept by utilizing the model's internal attention mechanisms
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论随着多模态语言模型的快速发展，能够在日常生活中辅助人类的 AI 助手正变得日益可行。一个关键挑战在于克服这些模型通用化的性质，以提供个性化体验。现有的大型视觉语言模型个性化方法往往依赖额外的训练阶段，这限制了通用性与可扩展性；或者依赖由外部预训练模块构建的工程化流水线，这阻碍了部署效率。在本工作中，我们提出一种高效的个性化方法，利用模型自身捕捉个性化概念的固有能力。具体而言，我们借助模型内部的注意力机制，提取主要表征目标概念的视觉 token。这些 token 充当该特定概念的记忆，使模型在测试图像中出现该概念时能够将其召回并加以描述。我们在包括单概念、多概念和视频个性化在内的多种个性化设定下，对本方法与当前最优方法进行了全面而统一的评测，结果表明本方法以极小的个性化开销取得了显著的性能提升。
英文原文摘要 1185 字符
AI assistants that support humans in daily life are becoming increasingly feasible, driven by the rapid advancements in multimodal language models. A key challenge lies in overcoming the generic nature of these models to deliver personalized experiences. Existing approaches to personalizing large vision language models often rely on additional training stages, which limit generality and scalability, or on engineered pipelines with external pre-trained modules, which hinder deployment efficiency. In this work, we propose an efficient personalization method that leverages the model's inherent ability to capture personalized concepts. Specifically, we extract visual tokens that predominantly represent the target concept by utilizing the model's internal attention mechanisms. These tokens serve as a memory of that specific concept, enabling the model to recall and describe it when it appears in test images. We conduct a comprehensive and unified evaluation of our approach and SOTA methods across various personalization settings including single-concept, multi-concept, and video personalization, demonstrating strong performance gains with minimal personalization overhead.