BALM：面向不均衡缺失率下均衡多模态学习的模型无关框架
BALM: A Model-Agnostic Framework for Balanced Multimodal Learning under Imbalanced Missing Rates
针对各模态缺失概率不一致（IMR）的场景，BALM 的特征校准模块用全局上下文重建单模态特征、梯度再平衡模块从分布与空间两个角度调节梯度幅值与方向，可作为即插即用模块嵌入多模态骨干，直接用于视觉-音频-文本多模态识别中缺失鲁棒性的提升。
原文依据 a model-agnostic plug-in framework to achieve balanced multimodal learning under IMR
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多模态学习常面临不均衡问题：信息丰富的模态主导优化过程，而较弱或部分缺失的模态贡献较小。在存在不均衡缺失模态（IMR）的现实设定中，各模态以不同概率缺失，这种不均衡会进一步加剧，从而扭曲表征学习与梯度动态。本文从训练过程视角重新审视该问题，提出 BALM——一个模型无关的即插即用框架，用于在 IMR 下实现均衡的多模态学习。该框架由两个互补模块组成。特征校准模块（FCM）作用于表征层面，通过全局上下文信息对单模态特征进行重新校准，以在异构缺失模式之间构建共享的表征基。梯度再平衡模块（GRM）作用于优化层面，从分布与空间两个视角调制梯度幅值与方向，从而均衡各模态间的学习动态。BALM 可无缝集成到多种骨干网络中，包括多模态情感识别（MER）模型，且无需改变其架构。在多个 MER 基准上的实验结果证实，BALM 在不同缺失与不均衡设定下均能持续增强鲁棒性并提升性能。
英文原文摘要 1369 字符
Learning from multiple modalities often suffers from imbalance, where information-rich modalities dominate optimization while weaker or partially missing modalities contribute less. This imbalance becomes severe in realistic settings with imbalanced missing modalities (IMR), where each modality is absent with different probabilities, distorting representation learning and gradient dynamics. We revisit this issue from a training-process perspective and propose BALM, a model-agnostic plug-in framework to achieve balanced multimodal learning under IMR. The framework consists of two complementary modules. The Feature Calibration Module (FCM) operates at the representation level, recalibrating unimodal features through global contextual information to build a shared representation basis across heterogeneous missing patterns. The Gradient Rebalancing Module (GRM) works at the optimization level, equalizing learning dynamics across modalities by modulating gradient magnitudes and directions from distributional and spatial perspectives. BALM can be seamlessly integrated into diverse backbones, including multimodal emotion recognition (MER) models, without altering their architectures. Experimental results across multiple MER benchmarks confirm that BALM consistently enhances robustness and improves performance under diverse missing and imbalance settings.