自监督与表征学习 · CVPR2026 ·2026.06.03

面向预训练模型类增量学习的量子门控任务交互知识蒸馏

Quantum-Gated Task-interaction Knowledge Distillation for Pre-trained Model-based Class-Incremental Learning

Linjie Li、Huiyu Xiao、Jiarui Cao、Zhenyu Wu、Yang Ji

与你有关的部分

提出量子门控任务调制机制,动态建模任务嵌入间的依赖关系,并以此引导跨任务适配器的知识蒸馏,为视觉类增量学习提供了一种缓解灾难性遗忘的可复用技术路径。

原文依据 we propose a novel Quantum-Gated Task-interaction Knowledge Distillation (QKD) framework that leverages quantum gating to guide inter-task knowledge transfer.
可迁移到 类增量学习图像分类知识蒸馏

摘要译文

由模型翻译 · 术语统一 · 请对照原文核对关键结论

类增量学习(CIL)旨在从任务流中持续积累知识,并构建覆盖所有已见类别的统一分类器。尽管预训练模型(PTM)在CIL中展现出良好性能,但仍受多任务子空间纠缠的困扰,当任务路由参数校准不佳或任务级表示被刚性固定时,会导致灾难性遗忘。为解决该问题,我们提出一种新颖的量子门控任务交互知识蒸馏(QKD)框架,利用量子门控引导任务间知识迁移。具体而言,我们引入量子门控任务调制门控机制来建模任务嵌入之间的关系依赖,动态捕获流式任务中联合训练与推理时的样本到任务相关性。在量子门控输出的引导下,我们利用这些任务嵌入级相关性权重,从旧适配器到新适配器执行任务交互知识蒸馏,使模型能够弥合独立任务子空间之间的表示差距。大量实验表明,QKD有效缓解遗忘并达到最优性能。代码见:https://github.com/Frank-lilinjie/CVPR26-QKD

英文原文摘要 1309 字符

Class-incremental learning (CIL) aims to continuously accumulate knowledge from a stream of tasks and construct a unified classifier over all seen classes. Although pretrained models (PTMs) have shown promising performance in CIL, they still struggle with the entanglement of multi-task subspaces, leading to catastrophic forgetting when task routing parameters are poorly calibrated or task-level representations are rigidly fixed. To address this issue, we propose a novel Quantum-Gated Task-interaction Knowledge Distillation (QKD) framework that leverages quantum gating to guide inter-task knowledge transfer. Specifically, we introduce a quantum-gated task modulation gating mechanism to model the relational dependencies among task embedding, dynamically capturing the sample-to-task relevance for both joint training and inference across streaming tasks. Guided by the quantum gating outputs, we perform task-interaction knowledge distillation guided by these task-embedding-level correlation weights from old to new adapters, enabling the model to bridge the representation gaps between independent task subspaces. Extensive experiments demonstrate that QKD effectively mitigates forgetting and achieves state-of-the-art performance. Code is available at: https://github.com/Frank-lilinjie/CVPR26-QKD

标签

会议 CVPR2026
页码 3920-3929
Bibkey
来源 CVF Open Access