面向多模态持续学习中的动态模态对齐
Towards Dynamic Modality Alignment in Multimodal Continual Learning
把跨模态对齐看作随任务与网络层演化而变化的动态过程，并用多层图正则化约束它，这个思路可直接迁移到视觉-语言模型的持续预训练或增量微调中，用来抑制跨模态表征漂移。
原文依据 we argue that modality alignment is inherently dynamic, evolving with task learning and feature propagation across layers
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论多模态持续学习（MMCL）旨在使模型能够在多个任务和模态上持续积累知识，同时不遗忘先验信息。相比单模态持续学习，MMCL 面临更多挑战，因为它需要模态之间有效的协作与互补。现有方法往往将模态对齐视为一个静态过程，假设对齐一旦建立便保持不变。然而，我们认为模态对齐本质上是动态的，会随着任务学习以及特征在层间的传播而演化。为此，我们提出动态对齐图正则化（DAGR），这是一种显式建模跨层演化对齐的新方法。通过引入多层级图正则化，我们的方法稳定了对齐过程并缓解了灾难性遗忘。在 MTIL 等基准上的大量实验表明，DAGR 优于基于静态对齐的方法以及其他持续学习技术，取得了更优的稳定性。
英文原文摘要 1043 字符
Multimodal Continual Learning (MMCL) aims to enable models to continuously accumulate knowledge across multiple tasks and modalities without forgetting prior information. MMCL presents more challenges than single-modal continual learning, as it requires effective cooperation and complementarity between modalities. Existing methods often treat modality alignment as a static process, assuming once alignment is established, it remains fixed. However, we argue that modality alignment is inherently dynamic, evolving with task learning and feature propagation across layers. To address this, we introduce Dynamic Alignment Graph Regularization (DAGR), a novel approach that explicitly models the evolving alignment across layers. By incorporating multi-level graph regularization, our method stabilizes the alignment process and mitigates catastrophic forgetting. Extensive experiments on benchmarks, such as MTIL, show that DAGR outperforms static alignment-based methods and other continual learning techniques, achieving superior stability.