视觉语言模型 · CVPR2026 ·2026.06.03

面向多模态情感分析的三子空间解耦

Tri-Subspaces Disentanglement for Multimodal Sentiment Analysis

Chunlei Meng、Jiabin Luo、Zhenglin Yan、Zhenyu Yu、Rong Fu、Zhongxue Gan 等 7 人

与你有关的部分

把多模态特征显式拆成全局共享、模态对共享、模态私有三类子空间,并用解耦监督加结构化正则保持纯度,这套「按共享范围分层解耦」的思路可直接迁移到视觉-语言任务(如VQA、图文检索)的表征融合环节,替代笼统的共享/私有二分。

原文依据 we propose a Tri-Subspace Disentanglement (TSD) framework that explicitly factorizes features into three complementary subspaces
可迁移到 多模态情感分析多模态意图识别视觉-语言表征融合

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

多模态情感分析(MSA)融合语言、视觉与声学模态来推断人的情感。现有方法大多关注全局共享表征或模态特有特征,而忽视了仅由特定模态对共享的信号,这限制了多模态表征的表达能力与判别力。为解决这一局限,我们提出三子空间解耦(TSD)框架,将特征显式分解为三个互补子空间:捕捉全局一致性的公共子空间、建模成对跨模态协同的子模态共享子空间,以及保留模态特有线索的私有子空间。为保持这些子空间的纯净与独立,我们引入解耦监督器以及结构化正则化损失。我们进一步设计了子空间感知交叉注意力(SACA)融合模块,自适应地建模并整合来自三个子空间的信息,以获得更丰富、更鲁棒的表征。在 CMU-MOSI 与 CMU-MOSEI 上的实验表明,TSD 在所有关键指标上均达到最优性能,在非对齐设定下于 CMU-MOSI 取得 0.691 的 MAE、于 CMU-MOSEI 取得 54.6% 的 Acc-7,并且能很好地迁移到多模态意图识别任务。消融研究证实,三子空间解耦与 SACA 共同增强了对多粒度跨模态情感线索的建模。

英文原文摘要 1454 字符

Multimodal Sentiment Analysis (MSA) integrates language, visual, and acoustic modalities to infer human sentiment. Most existing methods either focus on globally shared representations or modality-specific features, while overlooking signals that are shared only by certain modality pairs. This limits the expressiveness and discriminative power of multimodal representations. To address this limitation, we propose a Tri-Subspace Disentanglement (TSD) framework that explicitly factorizes features into three complementary subspaces: a common subspace capturing global consistency, submodally-shared subspaces modeling pairwise cross-modal synergies, and private subspaces preserving modality-specific cues. To keep these subspaces pure and independent, we introduce a decoupling supervisor together with structured regularization losses. We further design a Subspace-Aware Cross-Attention (SACA) fusion module that adaptively models and integrates information from the three subspaces to obtain richer and more robust representations. Experiments on CMU-MOSI and CMU-MOSEI demonstrate that TSD achieves state-of-the-art performance across all key metrics, reaching 0.691 MAE on CMU-MOSI and 54.6% Acc-7 on CMU-MOSEI under the unaligned setting, and also transfers well to multimodal intent recognition tasks. Ablation studies confirm that tri-subspaces disentanglement and SACA jointly enhance the modeling of multi-granular cross-modal sentiment cues.

标签

会议 CVPR2026
页码 8791-8800
Bibkey
来源 CVF Open Access