PersonaVLM：长期个性化多模态大语言模型
PersonaVLM: Long-Term Personalized Multimodal LLMs
为视觉语言模型引入可检索的多模态长期记忆库与用户人格演化建模，可作为个性化 VLM 助手、多轮视觉对话与用户偏好对齐的可复用框架；其 Persona-MME 基准也可用于评测视觉对话中的长期个性化能力。
原文依据 It proactively extracts and summarizes chronological multimodal memories from interactions, consolidating them into a personalized database.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多模态大语言模型（MLLM）已成为数百万用户的日常助手。然而，它们生成符合个体偏好的响应的能力仍然有限。已有方法仅通过输入增强或输出对齐实现静态、单轮个性化，因而无法捕捉用户随时间演化的偏好与人格特征（见图 img1）。本文提出 PersonaVLM，一种面向长期个性化的创新性个性化多模态智能体框架。它通过整合三项关键能力，将通用 MLLM 转变为个性化助手：(a) 记忆：主动从交互中提取并总结按时间顺序排列的多模态记忆，将其整合为个性化数据库；(b) 推理：通过从数据库中检索并整合相关记忆进行多轮推理；(c) 响应对齐：在长期交互中推断用户不断演化的人格特征，以确保输出始终与其独特特征保持一致。在评测方面，我们构建了 Persona-MME，一个包含 2000 余个精选交互案例的综合基准，旨在从七个关键方面与 14 项细粒度任务评估 MLLM 的长期个性化能力。大量实验验证了本方法的有效性：在 128k 上下文下，其在 Persona-MME 与 PERSONAMEM 上分别较基线提升 22.4% 与 9.8%，并分别以 5.2% 与 2.0% 的优势超越 GPT-4o。项目主页：https://PersonaVLM.github.io。
英文原文摘要 1538 字符
Multimodal Large Language Models (MLLMs) serve as daily assistants for millions. However, their ability to generate responses aligned with individual preferences remains limited. Prior approaches enable only static, single-turn personalization through input augmentation or output alignment, and thus fail to capture users' evolving preferences and personality over time (see Fig.img1). In this paper, we introduce PersonaVLM, an innovative personalized multimodal agent framework designed for long-term personalization. It transforms a general-purpose MLLM into a personalized assistant by integrating three key capabilities: (a) Remembering: It proactively extracts and summarizes chronological multimodal memories from interactions, consolidating them into a personalized database. (b) Reasoning: It conducts multi-turn reasoning by retrieving and integrating relevant memories from the database. (c) Response Alignment: It infers the user's evolving personality throughout long-term interactions to ensure outputs remain aligned with their unique characteristics. For evaluation, we establish Persona-MME, a comprehensive benchmark comprising over 2,000 curated interaction cases, designed to assess long-term MLLM personalization across seven key aspects and 14 fine-grained tasks. Extensive experiments validate our method's effectiveness, improving the baseline by 22.4% (Persona-MME) and 9.8% (PERSONAMEM) under a 128k context, while outperforming GPT-4o by 5.2% and 2.0%, respectively. Project page: https://PersonaVLM.github.io.