基于自适应梯度重加权与多阶段目标优化的机器遗忘
Machine Unlearning via Adaptive Gradient Reweighting and Multi-stage Objective Optimization
为视觉模型提供一种无需重训练的遗忘机制：通过自适应梯度重加权与多阶段目标优化（方向校正、时序稳定、自适应目标组合）缓解遗忘与保留目标间的梯度冲突，可直接用于扩散模型或分类骨干网的概念/样本擦除与隐私合规。
原文依据 the inherent inconsistency between forgetting and retention objectives results in gradient conflict and domination problems during training
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论机器遗忘（MU）旨在从预训练模型中移除训练样本的影响，而无需完全重新训练模型。现有 MU 方法已做出若干努力，以实现彻底遗忘的同时保持模型在剩余数据上的性能。然而，它们通常对不同数据施加相同权重，忽视了相似样本或近似类别之间模糊的决策边界。这导致对浅层记忆样本的不必要消耗，并使近似保留类别出现显著性能下降。此外，遗忘目标与保留目标之间固有的不一致性会在训练过程中引发梯度冲突与支配问题，阻碍模型收敛并降低整体性能。为解决这些问题，我们提出一种新颖的自适应梯度重加权方法，为单个遗忘样本或易受影响的保留类别分配重要性权重，从而实现更高效的遗忘并保持近似类别的性能。随后，我们提出一种多阶段目标优化策略，包含三个优化阶段：方向校正、时序稳定与自适应目标组合。该策略校正冲突梯度的方向，并防止某一任务（遗忘或保留）支配模型更新。全面的分析与在多个公开数据集上的大量实验表明，我们的方法在各种任务与场景中均取得了可观的性能提升。
英文原文摘要 1589 字符
Machine Unlearning (MU) focuses on removing the influence of training samples from pre-trained models without retraining the model entirely. Existing MU methods have made several efforts to enable complete forgetting while preserving the model's performance on remaining data. However, they typically apply equal weights across different data, overlooking the ambiguous decision boundaries between similar samples or approximate classes. This leads to unnecessary consumption of shallowly memorized samples and significant performance degradation for approximate retention classes. Additionally, the inherent inconsistency between forgetting and retention objectives results in gradient conflict and domination problems during training, hindering model convergence and degrading overall performance. To address these, we introduce a novel adaptive gradient reweighting that assigns importance weights to individual forget samples or vulnerable retention classes, thereby enabling more efficient unlearning and preserving the performance of approximate classes. Subsequently, we propose a multi-stage objective optimization strategy, which comprises three optimization stages: Direction Rectification, Temporal Stabilization, and Adaptive Objective Combination. This strategy rectifies the direction of conflicting gradients and prevents one task (forgetting or retention) from dominating the model update. Comprehensive analyses and extensive experiments on multiple public datasets demonstrate that our method achieves considerable performance improvements in various tasks and scenarios.