解耦稳定性与可塑性以实现多模态测试时自适应
Decoupling Stability and Plasticity for Multi-Modal Test-Time Adaptation
针对多模态模型在测试时分布漂移下的适配，提出用特征维度间冗余度诊断偏置模态，并对各模态适配器做稳定/可塑解耦的非对称更新；做视觉-语言模型在线部署或跨域适配时，可直接复用该诊断信号与解耦适配器结构来抑制负迁移与灾难性遗忘。
原文依据 the biased modality exhibits substantially higher interdimensional redundancy (i.e., strong correlations across feature dimensions)
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论将预训练多模态模型适配到不断变化的测试时分布，即多模态测试时自适应，是一项重大挑战。现有方法常常在无偏模态上遭遇负迁移，在有偏模态上遭遇灾难性遗忘。为应对这些挑战，我们提出解耦稳定性与可塑性适配（DASP），一个新颖的「先诊断、后缓解」框架。我们的分析揭示了统一潜变量空间中的一个关键差异：与无偏模态相比，有偏模态表现出显著更高的维度间冗余（即特征维度之间的强相关性）。基于这一发现，DASP 识别出有偏模态并实施非对称适配策略。该策略采用解耦架构，将每个模态专属适配器划分为稳定组件与可塑组件。非对称机制的工作方式如下：对于需要可塑性的有偏模态，激活并更新可塑组件以捕获域特定信息，同时保持稳定组件固定；相反，对于需要稳定性的无偏模态，绕过可塑组件，并使用 KL 正则化更新稳定组件，以防止负迁移。这种非对称设计使模型能够灵活适配新域，同时保留可泛化的知识。在多种多模态基准上的全面评估表明，DASP 显著优于最优方法。
英文原文摘要 1576 字符
Adapting pretrained multi-modal models to evolving test-time distributions, known as multi-modal test-time adaptation, presents a significant challenge. Existing methods frequently encounter negative transfer in the unbiased modality and catastrophic forgetting in the biased modality. To address these challenges, we propose Decoupling Adaptation for Stability and Plasticity (DASP), a novel diagnose-then-mitigate framework. Our analysis reveals a critical discrepancy within the unified latent space: the biased modality exhibits substantially higher interdimensional redundancy (i.e., strong correlations across feature dimensions) compared to the unbiased modality. Leveraging this insight, DASP identifies the biased modality and implements an asymmetric adaptation strategy. This strategy employs a decoupled architecture where each modality-specific adapter is divided into stable and plastic components. The asymmetric mechanism works as follows: for the biased modality, which requires plasticity, the plastic component is activated and updated to capture domain-specific information, while the stable component remains fixed. Conversely, for the unbiased modality, which requires stability, the plastic component is bypassed, and the stable component is updated using KL regularization to prevent negative transfer. This asymmetric design enables the model to adapt flexibly to new domains while preserving generalizable knowledge. Comprehensive evaluations on diverse multi-modal benchmarks demonstrate that DASP significantly outperforms state-of-the-art methods.