视频理解与时序 · CVPR2026 ·2026.06.03

BriMA:面向多模态持续动作质量评估的桥接模态适应

BriMA: Bridged Modality Adaptation for Multi-Modal Continual Action Quality Assessment

Kanglei Zhou、Chang Li、Qingyi Pan、Liyuan Wang

与你有关的部分

针对多模态持续动作质量评估中模态缺失问题,提出记忆引导的桥接插补模块和模态感知回放机制,可复用于视频理解中模态不完整或动态变化的持续学习场景。

原文依据 BriMA consists of a memory-guided bridging imputation module that reconstructs missing modalities using both task-agnostic and task-specific representations
可迁移到 动作质量评估多模态学习持续学习

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

动作质量评估(AQA)旨在评估动作完成的质量,广泛应用于体育分析、康复评估和人类技能评价。多模态AQA通过利用互补的视觉和运动学线索近期取得了显著进展,然而实际部署中常面临非平稳模态不平衡问题,即某些模态因传感器故障或标注缺失而变得缺失或间歇性可用。现有的持续AQA方法忽视了这一问题,假设训练过程中所有模态始终完整稳定,这限制了其实用性。为解决这一挑战,我们提出了桥接模态适应(BriMA),一种针对模态缺失条件下多模态持续AQA的创新方法。BriMA包含一个记忆引导的桥接插补模块,利用任务无关和任务特定的表示重建缺失模态,以及一个模态感知回放机制,根据模态失真和分布漂移优先选择信息量大的样本。在三个代表性多模态AQA数据集(RG、Fis-V和FS1000)上的实验表明,BriMA在不同模态缺失条件下持续提升性能,平均相关性提高6-8%,误差降低12-15%。这些结果展示了在现实部署约束下迈向鲁棒多模态AQA系统的一步。我们的代码可在https://github.com/ZhouKanglei/BriMA获取。

英文原文摘要 1513 字符

Action Quality Assessment (AQA) aims to score how well an action is performed and is widely used in sports analysis, rehabilitation assessment, and human skill evaluation. Multi-modal AQA has recently achieved strong progress by leveraging complementary visual and kinematic cues, yet real-world deployments often suffer from non-stationary modality imbalance, where certain modalities become missing or intermittently available due to sensor failures or annotation gaps. Existing continual AQA methods overlook this issue and assume that all modalities remain complete and stable throughout training, which restricts their practicality. To address this challenge, we introduce Bridged Modality Adaptation (BriMA), an innovative approach to multi-modal continual AQA under modality-missing conditions. BriMA consists of a memory-guided bridging imputation module that reconstructs missing modalities using both task-agnostic and task-specific representations, and a modality-aware replay mechanism that prioritizes informative samples based on modality distortion and distribution drift. Experiments on three representative multi-modal AQA datasets (RG, Fis-V, and FS1000) show that BriMA consistently improves performance under different modality-missing conditions, achieving 6--8% higher correlation and 12--15% lower error on average. These results demonstrate a step toward robust multi-modal AQA systems under real-world deployment constraints. Our code is available at https://github.com/ZhouKanglei/BriMA.

标签

会议 CVPR2026
页码 38904-38914
Bibkey
来源 CVF Open Access