ProxyFL:一种代理引导的联邦半监督学习框架
ProxyFL: A Proxy-Guided Framework for Federated Semi-Supervised Learning
把分类器可学习权重当作类别代理来同时对齐客户端间与客户端内的分布,这一「代理池」思路可直接迁移到视觉中的半监督伪标签筛选与长尾/域偏移下的分类头校准。
原文依据 we consider the learnable weights of classifier as proxy to simulate the category distribution both locally and globally
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论联邦半监督学习(FSSL)旨在以隐私保护的方式,利用客户端上部分标注的本地数据协作训练一个全局模型。在 FSSL 中,数据异构性是一个具有挑战性的问题,它同时存在于客户端之间与客户端内部。外部异构性指不同客户端之间的数据分布差异,而内部异构性表示客户端内部标注数据与未标注数据之间的不匹配。大多数 FSSL 方法通常设计固定或动态的参数聚合策略,在服务器端收集客户端知识(外部),和/或过滤掉低置信度的未标注样本来减少本地客户端中的错误(内部)。但前者难以通过直接权重精确拟合理想的全局分布,后者则导致参与联邦训练的数据减少。为此,我们提出了一个名为 ProxyFL 的代理引导框架,致力于通过统一的代理同时缓解外部与内部异构性。具体而言,我们将分类器的可学习权重视为代理,用以在本地和全局模拟类别分布。对于外部异构性,我们显式地针对离群点优化全局代理,而非直接优化权重;对于内部异构性,我们通过正负代理池将丢弃的样本重新纳入训练,以缓解潜在错误伪标签的影响。洞察实验与理论分析表明了我们在 FSSL 中显著的性能与收敛表现。
英文原文摘要 1525 字符
Federated Semi-Supervised Learning (FSSL) aims to collaboratively train a global model across clients by leveraging partially-annotated local data in a privacy-preserving manner. In FSSL, data heterogeneity is a challenging issue, which exists both across clients and within clients. External heterogeneity refers to the data distribution discrepancy across different clients, while internal heterogeneity represents the mismatch between labeled and unlabeled data within clients. Most FSSL methods typically design fixed or dynamic parameter aggregation strategies to collect client knowledge on the server (external) and / or filter out low-confidence unlabeled samples to reduce mistakes in local client (internal). But, the former is hard to precisely fit the ideal global distribution via direct weights, and the latter results in fewer data participation into FL training. To this end, we propose a proxy-guided framework called ProxyFL that focuses on simultaneously mitigating external and internal heterogeneity via a unified proxy. I.e., we consider the learnable weights of classifier as proxy to simulate the category distribution both locally and globally. For external, we explicitly optimize global proxy against outliers instead of direct weights; for internal, we re-include the discarded samples into training by a positive-negative proxy pool to mitigate the impact of potentially-incorrect pseudo-labels. Insight experiments & theoretical analysis show our significant performance and convergence in FSSL.