Fed-ADE:分布偏移下联邦后自适应的自适应学习率
Fed-ADE: Adaptive Learning Rate for Federated Post-adaptation under Distribution Shift
为视觉模型部署后的无标签在线自适应提供了一套按客户端、按时间步动态调节学习率的机制:用预测不确定性的变化与特征漂移来估计分布偏移强度,可直接迁移到视觉模型在非平稳数据流上的持续测试时自适应,避免固定学习率导致的欠拟合或发散。
原文依据 Fed-ADE employs uncertainty dynamics estimation to capture changes in predictive uncertainty and representation dynamics estimation to detect covariate-level feature drift
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论部署后场景中的联邦学习必须在无法获取真实标签的条件下,适应跨异构客户端的非平稳数据流。一项主要挑战是在客户端特定且随时间变化的分布偏移下选择学习率,固定学习率往往导致欠拟合或发散。我们提出 Fed-ADE(带分布偏移估计的联邦自适应),这是一个无监督的联邦自适应框架,利用分布动态的轻量级估计器。具体而言,Fed-ADE 采用不确定性动态估计来捕捉预测不确定性的变化,并采用表征动态估计来检测协变量层面的特征漂移,将二者结合为按客户端、按时间步的自适应学习率。我们给出理论分析,表明我们的动态估计能够近似底层分布偏移,并带来动态遗憾与收敛保证。在多种分布偏移(标签偏移、协变量偏移和概念偏移)下的图像与文本基准实验表明,该方法相对强基线取得了一致提升。这些结果凸显出:分布偏移感知的自适应能够在真实世界的非平稳性下实现有效且鲁棒的联邦后自适应。
英文原文摘要 1270 字符
Federated learning (FL) in post-deployment settings must adapt to non-stationary data streams across heterogeneous clients without access to ground-truth labels. A major challenge is learning rate selection under client-specific, time-varying distribution shifts, where fixed learning rates often lead to underfitting or divergence. We propose Fed-ADE (Federated Adaptation with Distribution Shift Estimation), an unsupervised federated adaptation framework that leverages lightweight estimators of distribution dynamics. Specifically, Fed-ADE employs uncertainty dynamics estimation to capture changes in predictive uncertainty and representation dynamics estimation to detect covariate-level feature drift, combining them into a per-client, per-timestep adaptive learning rate. We provide theoretical analyses showing that our dynamics estimation approximates the underlying distribution shift and yields dynamic regret and convergence guarantees. Experiments on image and text benchmarks under diverse distribution shifts (label, covariate, and concept) demonstrate consistent improvements over strong baselines. These results highlight that distribution shift-aware adaptation enables effective and robust federated post-adaptation under real-world non-stationarity.