超越缺失模态：超图条件扩散用于不确定性感知的多模态情感识别
Beyond Missing Modalities: Hypergraph Conditioned Diffusion for Uncertainty-Aware Multimodal Emotion Recognition
提出用掩码超图注意力条件扩散模型在潜空间恢复缺失模态特征，并用双通道证据融合量化不确定性，可迁移到视觉-语言多模态任务中处理模态缺失与不确定性建模。
原文依据 we propose Masked Hypergraph Attention (MHGAT) conditioned diffusion model to recover latent features of missing modalities in the latent space.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论对话中的多模态情感识别（MERC）旨在通过有效整合音频、文本和视觉模态来理解每句话中表达的情感。然而，在现实场景中，不可避免的模态缺失往往会降低多模态解释性能。为解决这一问题，我们提出了基于超图扩散和证据融合的情感识别框架（HyperEF），该新颖框架旨在缓解MERC中由不完整模态引起的挑战。具体而言，为减轻模态缺失导致的性能下降，我们提出了掩码超图注意力（MHGAT）条件扩散模型，在潜空间中恢复缺失模态的潜变量特征。为确保同一句话中恢复模态与可用模态之间的语义一致性，我们引入了MHGAT，从可用模态中捕获高阶语义信息以指导扩散模型的去噪过程。此外，为解耦和建模MERC中固有的复杂不确定性，我们提出了双通道证据融合（DCEF），在特征来源级和判别级同时估计不确定性，从而实现自适应证据融合。大量的对比实验和可解释性分析证明了我们的模型在情感识别方面的优越性能，以及模型中每个模块的贡献。
英文原文摘要 1403 字符
Multimodal Emotion Recognition in Conversations (MERC) aims to understand emotions expressed in each utterance by effectively integrating audio, text, and visual modalities. However, in real-world scenarios, unavoidable missing modalities often degrade multimodal interpretation performance. To address this, we propose Hypergraph Diffusion and Evidence Fusion based Emotion Recognition (HyperEF), a novel framework designed to mitigate challenges arising from incomplete modalities in MERC. Specifically, to mitigate performance degradation caused by modality absence, we propose Masked Hypergraph Attention (MHGAT) conditioned diffusion model to recover latent features of missing modalities in the latent space. To ensure semantic consistency between recovered and available modalities within the same utterance, we introduce MHGAT that captures high-order semantic information from available modalities to guide the diffusion model's denoising process. Furthermore, to disentangle and model the complex uncertainties inherent in MERC, we propose Dual Channel Evidence Fusion (DCEF), which estimates uncertainty at both feature source level and discriminative level, thereby achieving adaptive evidence fusion. Extensive comparative experiments and interpretability demonstrate the superior performance of our model in emotion recognition, as well as the contribution of each module within the model.