基于最优传输的子模型提取实现高效个性化联邦学习
Submodel Extraction for Efficient and Personalized Federated Learning via Optimal Transport
提出基于最优传输的服务器端个性化剪枝框架,可在不访问原始数据的情况下为异构客户端生成定制子模型,并利用自适应正则化缓解参数发散,为视觉模型在边缘设备上的高效个性化部署提供可复用方案。
原文依据 SubFLOT introduces an Optimal Transport-enhanced Pruning (OTP) module that treats historical client models as proxies for local data distributions
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论联邦学习(FL)能够在保护数据隐私的同时实现协作式模型训练,但其实际部署受到系统异构性和统计异构性的制约。尽管联邦网络剪枝为缓解这些问题提供了一条途径,现有方法却面临一个关键困境:服务器端剪枝缺乏个性化,而客户端剪枝对资源受限设备而言计算代价过高。此外,剪枝过程本身会在异构子模型之间引发显著的参数发散,破坏训练稳定性并阻碍全局收敛。为应对这些挑战,我们提出 SubFLOT,一种用于服务器端个性化联邦剪枝的新框架。SubFLOT 引入最优传输增强剪枝(OTP)模块,将历史客户端模型视为本地数据分布的代理,把剪枝任务建模为 Wasserstein 距离最小化问题,从而在不访问原始数据的情况下生成定制子模型。同时,为抵消参数发散,我们的基于缩放的自适应正则化(SAR)模块自适应地惩罚子模型对全局模型的偏离,惩罚强度按客户端的剪枝率缩放。大量实验表明,SubFLOT 持续且显著优于最优方法,凸显了其在资源受限边缘设备上部署高效且个性化模型的潜力。
英文原文摘要 1464 字符
Federated Learning (FL) enables collaborative model training while preserving data privacy, but its practical deployment is hampered by system and statistical heterogeneity. While federated network pruning offers a path to mitigate these issues, existing methods face a critical dilemma: server-side pruning lacks personalization, whereas client-side pruning is computationally prohibitive for resource-constrained devices. Furthermore, the pruning process itself induces significant parametric divergence among heterogeneous submodels, destabilizing training and hindering global convergence. To address these challenges, we propose SubFLOT, a novel framework for server-side personalized federated pruning. SubFLOT introduces an Optimal Transport-enhanced Pruning (OTP) module that treats historical client models as proxies for local data distributions, formulating the pruning task as a Wasserstein distance minimization problem to generate customized submodels without accessing raw data. Concurrently, to counteract parametric divergence, our Scaling-based Adaptive Regularization (SAR) module adaptively penalizes a submodel's deviation from the global model, with the penalty's strength scaled by the client's pruning rate. Comprehensive experiments demonstrate that SubFLOT consistently and substantially outperforms state-of-the-art methods, underscoring its potential for deploying efficient and personalized models on resource-constrained edge devices.