TAPE：音频语言模型中任务自适应原型进化用于完全少样本类增量音频分类
TAPE: Task-Adaptive Prototype Evolution in Audio-Language Models for Fully Few-shot Class-incremental Audio Classification
论文提出的任务适配器（Task-Adapter）和原型进化机制可迁移到视觉中的少样本类增量学习场景，例如利用CLIP等视觉语言模型进行增量目标识别时，通过隔离特征空间和动态原型更新来缓解灾难性遗忘与过拟合。
原文依据 A Task-Adapter that isolates audio features in a metric space to mitigate catastrophic forgetting
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论完全少样本类增量音频分类（FFCAC）具有挑战性，因为增量会话和基础会话中的训练样本都有限。现有的少样本学习方法应用于FFCAC时会出现灾难性遗忘和过拟合。预训练的音频语言模型（ALM）已在许多音频学习任务中取得成功。然而，我们发现直接在FFCAC上使用ALM是不切实际的，因为文本与音频之间的错位会导致更严重的灾难性遗忘和过拟合。我们提出了任务自适应原型进化（TAPE）框架，以促进ALM应对FFCAC的挑战，该框架包含两个关键组件：（1）任务适配器，在度量空间中隔离音频特征，以减轻灾难性遗忘，同时保留跨会话的知识；（2）原型进化机制，在推理过程中利用查询样本动态细化类原型，从而实现自适应学习并减少过拟合。据我们所知，我们是首个在FFCAC任务上使用ALM的工作。我们在三个音频数据集上进行了实验：NSynth-100（乐器识别）、FSC-89（事件检测）和LBS-100（语音识别）。实验结果表明，我们提出的TAPE显著超越了基线。具体而言，在平均准确率（AA ↑）上，它平均将第二优方法从54.93%提升至82.76%；在性能下降率（PD ↓）上，从28.74%降低至12.56%。我们的代码可在https://github.com/YvoGao/TAPE获取。
英文原文摘要 1625 字符
Fully Few-shot Class-incremental Audio Classification (FFCAC) is challenging since the training samples are limited both in the incremental sessions and in the base session. Existing few-shot learning methods suffer from catastrophic forgetting and overfitting when applied to FFCAC.Pre-trained Audio-Language Models (ALMs) have achieved success in many audio learning tasks. However, we find that it is impractical to directly use ALM on FFCAC, since misalignment between text and audio causes even severe catastrophic forgetting and overfitting. We propose a Task-Adaptive Prototype Evolution (TAPE) framework to facilitate ALMs to tackle the challenges of FFCAC, which consists of two key components:(1) A Task-Adapter that isolates audio features in a metric space to mitigate catastrophic forgetting while preserving knowledge across sessions, and (2) A Prototype Evolution mechanism that dynamically refines class prototypes using query samples during inference, thereby enabling adaptive learning and reducing overfitting.To the best of our knowledge, we are the first to use ALMs on the FFCAC task. We conduct experiments on three audio datasets: NSynth-100 (instrument recognition), FSC-89 (event detection), and LBS-100 (voice recognition). The experimental results show that our proposed approach TAPE significantly surpasses the baselines. Specifically, it averagely improves upon the second best from 54.93% to 82.76% in terms of Average Accuracy (AA \uparrow), and from 28.74% to 12.56% in terms of Performance Dropping rate (PD \downarrow). The code for our work is available at https://github.com/YvoGao/TAPE.