视觉语言模型 · CVPR2026 ·2026.06.03

主动感知推理:一种受皮层-丘脑启发的动态嵌套循环网络用于不完整数据下的多模态情感分析

Active Perceptual Inference: A Corticothalamic-Inspired Dynamic Nested Recurrent Network for Multimodal Sentiment Analysis with Incomplete Data

Yujuan Zhang、Qing Li、Ziyu Li、Xiuxing Li、Zhuo Wang、Mengrui Xu 等 7 人

与你有关的部分

提出一种受皮层-丘脑启发的动态嵌套循环网络,通过局部循环补全和全局循环置信度加权,实现从被动补全到主动感知推理的范式转变,可复用于视觉-语言任务中的缺失模态补全与动态融合。

原文依据 It is the first to introduce recurrent inference into the data completion task, achieving a paradigm shift from passive completion to active perceptual inference.
可迁移到 多模态情感分析缺失数据补全跨模态融合

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

随机帧级数据缺失是多模态情感分析中的一个关键挑战。现有方法大多局限于通过单次前馈连接和静态跨模态融合进行被动补全,难以生成高质量的补全特征。然而,大脑并非外部信息的被动接收者,而是一个进行主动感知推理的动态系统。其核心在于由皮层内循环补全机制和皮层-丘脑回路形成的动态嵌套循环,迭代地执行感知推理。受此启发,我们提出了动态嵌套循环网络(DNRNet)。它首次将循环推理引入数据补全任务,实现了从被动补全到主动感知推理的范式转变。其局部循环回路模拟皮层内循环模式补全以执行感知推理并生成局部校正特征。全局循环回路模拟丘脑的调节功能,计算模态置信度以动态加权和整合跨模态信息,生成全局校正特征。局部和全局校正特征被融合以获得补全信号,然后与当前迭代的输入特征结合,作为下一次迭代的输入。在MOSI、MOSEI和SIMS数据集上的实验表明,DNRNet在所有缺失率下相比基线模型平均准确率提升1.5%-2.0%,验证了类脑方法在复杂缺失数据场景下的优越性。

英文原文摘要 1654 字符

Random frame-level data missing is a critical challenge in multimodal sentiment analysis. Existing methods are largely limited to passive completion via single-pass feedforward connections and static cross-modal fusion, which struggle to generate high-quality completed features. However, the brain is not a passive recipient of external information but a dynamic system for active perceptual inference. Its core lies in the dynamic nested recurrents formed by intra-cortical recurrent completion mechanisms and corticothalamic circuits, which iteratively perform perceptual inference. Inspired by this, we propose the Dynamic Nested Recurrent Network (DNRNet). It is the first to introduce recurrent inference into the data completion task, achieving a paradigm shift from passive completion to active perceptual inference. Its local recurrent loop simulates intra-cortical recurrent pattern completion to perform perceptual inference and generate local correction features. The global recurrent loop simulates the modulatory function of the thalamus, calculating modality confidence to dynamically weight and integrate cross-modal information, generating global correction features. The local and global correction features are fused to obtain the completion signal, which is then combined with the input features of the current iteration to serve as the input for the next iteration. Experiments on the MOSI, MOSEI, and SIMS datasets demonstrate that DNRNet achieves an average accuracy improvement of 1.5%-2.0% over baseline models across all missing rates, validating the superiority of the brain-inspired approach in complex missing data scenarios.

标签

会议 CVPR2026
页码 1787-1797
Bibkey
来源 CVF Open Access