视觉语言模型 · CVPR2026 ·2026.06.03

Nano-EmoX:从感知到共情的多模态情感智能统一

Nano-EmoX: Unifying Multimodal Emotional Intelligence from Perception to Empathy

Jiahao Huang、Fengyan Lin、Xuechao Yang、Chen Feng、Kexin Zhu、Xu Yang 等 7 人

与你有关的部分

对做视觉研究的人而言,可复用点在于用异构适配器把面部编码器、融合编码器等全模态编码器输出投影进统一语言空间,并以课程式训练(先对齐快速感知、再引入思维链推理)让轻量语言模型同时处理多任务;这套「编码器+适配器+课程」的范式可直接迁移到面部表情、情感识别等视觉任务的多任务统一建模。

原文依据 Nano-EmoX integrates a suite of omni-modal encoders, including an enhanced facial encoder and a fusion encoder
可迁移到 面部表情识别多模态情感分析情感原因分析视觉情感描述生成

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

情感多模态语言模型(MLM)的发展长期受制于低层感知与高层交互之间的鸿沟,导致情感能力碎片化、泛化能力有限。为弥合这一鸿沟,我们提出一种受认知启发的三级层次结构,按认知深度将情感任务组织为感知、理解与交互三个层级,为推进情感建模提供统一的概念基础。在该层次结构的引导下,我们提出小规模多任务 MLM——Nano-EmoX,以及基于课程的学习框架 P2E(Perception-to-Empathy,从感知到共情)。Nano-EmoX 集成了一套全模态编码器,包括增强的面部编码器与融合编码器,以捕捉关键的多模态情感线索并提升跨任务可迁移性。其输出通过异构适配器投影到统一的语言空间,使轻量级语言模型能够处理多样化的情感任务。同时,P2E 通过将快速感知与思维链驱动的共情对齐,逐步培养情感智能。据我们所知,Nano-EmoX 是首个在全部三个层次上统一六项核心情感任务的紧凑型 MLM(2.2B),在多个基准上取得最优或极具竞争力的性能,展现出优异的效率与泛化能力。

英文原文摘要 1414 字符

The development of affective multimodal language models (MLMs) has long been constrained by a gap between low-level perception and high-level interaction, leading to fragmented affective capabilities and limited generalization. To bridge this gap, we propose a cognitively inspired three-level hierarchy that organizes affective tasks according to their cognitive depth--perception, understanding, and interaction--and provides a unified conceptual foundation for advancing affective modeling. Guided by this hierarchy, we introduce Nano-EmoX, a small-scale multitask MLM, and P2E (Perception-to-Empathy), a curriculum-based training framework. Nano-EmoX integrates a suite of omni-modal encoders, including an enhanced facial encoder and a fusion encoder, to capture key multimodal affective cues and improve cross-task transferability. The outputs are projected into a unified language space via heterogeneous adapters, empowering a lightweight language model to tackle diverse affective tasks. Concurrently, P2E progressively cultivates emotional intelligence by aligning rapid perception with chain-of-thought-driven empathy. To the best of our knowledge, Nano-EmoX is the first compact MLM (2.2B) to unify six core affective tasks across all three hierarchy levels, achieving state-of-the-art or highly competitive performance across multiple benchmarks, demonstrating excellent efficiency and generalization.

标签

会议 CVPR2026
页码 22986-22997
Bibkey
来源 CVF Open Access