为遗忘而设计:用于机器遗忘的深度半参数模型
Designing to Forget: Deep Semi-parametric Models for Unlearning
把「遗忘」从改参数的后处理问题转成架构设计问题:用融合模块聚合每个训练样本的信息,测试时直接删除样本即可完成遗忘,无需重训或微调,为视觉模型的数据删除合规与版权数据移除提供可复用范式。
原文依据 SPMs use a fusion module that aggregates information from each training sample, enabling explicit test-time deletion of selected samples without altering model parameters.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论近期机器遗忘领域的进展主要集中于开发从已训练模型中移除特定训练样本的算法。与之不同,我们观察到并非所有模型都同样易于遗忘。因此,我们提出了一类深度半参数模型(SPM),其在遗忘过程中表现出非参数化行为。SPM 使用一个融合模块聚合来自每个训练样本的信息,从而能够在测试时显式删除所选样本而无需改动模型参数。实验表明,SPM 在图像分类与生成任务上取得了与参数化模型相当的任务性能,同时在遗忘效率上显著更优。值得注意的是,在 ImageNet 分类任务上,SPM 将相对于重训练(oracle)基线的预测差距缩小了 11%,并且相比参数化模型上的现有方法实现了超过 10 倍的遗忘加速。
英文原文摘要 903 字符
Recent advances in machine unlearning have focused on developing algorithms to remove specific training samples from a trained model. In contrast, we observe that not all models are equally easy to unlearn. Hence, we introduce a family of deep semi-parametric models (SPMs) that exhibit non-parametric behavior during unlearning. SPMs use a fusion module that aggregates information from each training sample, enabling explicit test-time deletion of selected samples without altering model parameters. Empirically, we demonstrate that SPMs achieve competitive task performance to parametric models in image classification and generation, while being significantly more efficient for unlearning. Notably, on ImageNet classification, SPMs reduce the prediction gap relative to a retrained (oracle) baseline by 11% and achieve over 10xfaster unlearning compared to existing approaches on parametric models.