面向少样本类增量学习的语义引导全局-局部协同提示词学习
Semantic-Guided Global-Local Collaborative Prompt Learning for Few-Shot Class Incremental Learning
利用冻结的视觉语言模型原型与 LLM 生成的语义描述做全局-局部双对齐提示词学习,为视觉研究者提供了一种在少样本增量场景下抑制灾难性遗忘与过拟合的可复用范式,可直接迁移到开放词表识别与持续学习式视觉分类。
原文依据 we propose an innovative Semantic-Guided Global-Local Collaborative Prompt Learning (SGLC) framework
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论少样本类增量学习(FSCIL)在机器学习中提出了一个关键挑战:模型需要在仅有少量样本的情况下持续整合新类别,同时保留对先前已见类别的知识。尽管现有 FSCIL 方法已取得有前景的结果,但由于难以在旧知识保持与新知识获取之间取得平衡,它们仍然受到灾难性遗忘和少样本过拟合的困扰。为应对这些挑战,我们提出了一种创新的语义引导全局-局部协同提示词学习(SGLC)框架。该框架构建于强大的预训练视觉语言模型(VLM)之上,首先引入一种双重对齐机制:在全局上将视觉特征与视觉-文本原型对齐,在局部上将多视角视觉特征与局部文本属性特征对齐,从而在通过先前类别的冻结原型保留已有知识的同时,促进有效的知识学习。此外,为缓解过拟合,我们引入大语言模型(LLM)生成语义丰富的文本描述,并通过知识蒸馏同时指导全局与局部的提示词学习。在 miniImageNet、CIFAR-100 和 CUB200 数据集上的大量实验表明,SGLC 相较于最优方法表现更优。
英文原文摘要 1371 字符
Few-Shot Class-Incremental Learning (FSCIL) poses a critical challenge in machine learning, requiring models to continuously integrate novel classes with limited samples while preserving knowledge of previously seen classes. While existing FSCIL approaches have demonstrated promising results, they still suffer from catastrophic forgetting and few-shot overfitting due to the challenge of balancing old knowledge retention with new knowledge acquisition. To address these challenges, we propose an innovative Semantic-Guided Global-Local Collaborative Prompt Learning (SGLC) framework. Built upon powerful pre-trained Vision-Language Models (VLMs), the framework first introduces a dual-alignment mechanism: globally aligning visual features with visual-textual prototypes and locally aligning multi-view visual features with local textual attribute features, which facilitates effective knowledge learning while preserving existing knowledge via frozen prototypes of previous classes. Furthermore, to alleviate overfitting, we incorporate Large Language Models (LLMs) to generate semantically rich textual descriptions, which simultaneously guide both global and local prompt learning through knowledge distillation. Extensive experiments on the miniImageNet, CIFAR-100, and CUB200 datasets demonstrate that SGLC performs favorably against the state-of-the-art methods.