GDFA:几何驱动且带方向任务向量对齐的联邦遗忘
GDFA: Geometry-Driven Federated Unlearning with Directional Task Vector Alignment
把损失景观几何(平坦极小值)作为提升遗忘鲁棒性的手段,并通过对任务向量做方向一致性过滤来隔离共享知识——这套「平坦化 + 方向对齐的任务向量」思路可迁移到视觉模型的版权数据删除、概念擦除与个性化模型去身份化等场景。
原文依据 We demonstrate that migrating models to flat regions can enhance unlearning robustness in Non-IID FL.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论联邦学习(FL)是一种去中心化框架,既能实现不同客户端之间的协同训练,又能保证各客户端的本地数据隐私。然而,当隐私法规要求提出删除请求时,如何高效地从目标客户端中移除特定客户端数据的贡献仍颇具挑战。现有遗忘方法在非独立同分布(Non-IID)数据分布下尝试对特定目标客户端进行遗忘时存在显著局限。处于尖锐优化区域的模型可能因微小的参数变化而遭受灾难性的知识损失,而 FL 中 Non-IID 数据分布所导致的客户端间参数更新冲突会进一步加剧这种遗忘。经验上,我们观察到 Non-IID 设置下的冲突更新会产生方向不一致的任务向量,从而无法隔离目标知识。因此,我们在遗忘特定目标客户端时利用了损失景观的几何性质。我们证明,将模型迁移到平坦区域可以增强 Non-IID FL 中遗忘的鲁棒性。相应地,我们提出了 GDFA 框架:该框架首先将全局模型迁移到平坦的损失域;随后,相关客户端生成遗忘任务向量,GDFA 对其进行过滤,仅保留方向一致的成分。该过程在通过反向向量聚合进行精确移除之前隔离出共享知识属性,从而最大化知识保留。大量实验表明,GDFA 在多种数据集与架构上于遗忘有效性与效率方面均优于最优方法,且在保留任务上的精度损失极小。
英文原文摘要 1684 字符
Federated Learning (FL) is a decentralized framework that not only enables collaborative training with different clients but also ensures their local data privacy. However, when deletion requests arise under privacy regulations, efficiently removing specific client data contributions from target clients can be challenging. Existing unlearning methods face significant limitations under Non-IID (Non-Independent and Identically Distributed) data distributions when attempting to unlearn specific target clients in FL. Models in sharp optimization regions can suffer catastrophic knowledge loss from minor parameter changes, exacerbating this forgetting due to conflicting parameter updates across clients caused by Non-IID data distributions in FL. Empirically, we observe that conflicting updates under Non-IID settings generate misaligned task vectors that fail to isolate target knowledge. Therefore, we exploit the loss landscape geometry in unlearning specific target clients. We demonstrate that migrating models to flat regions can enhance unlearning robustness in Non-IID FL. Correspondingly, we introduce GDFA, a framework that initially transitions the global model to a flat loss domain. Subsequently, relevant clients generate unlearning task vectors, which GDFA filters to retain only directionally consistent components. This process isolates shared knowledge attributes before precise removal through reverse vector aggregation, maximizing knowledge retention. Extensive experiments demonstrate that GDFA outperforms state-of-the-art methods in unlearning efficacy and efficiency across diverse datasets and architectures, with minimal accuracy loss on retained tasks.