超越欧氏 gossip：异构且不平衡图像上的 KL 重心共识
Beyond Euclidean Gossip: KL-Barycentric Consensus on Heterogeneous and Imbalanced Images
为视觉模型在去中心化、非独立同分布数据上的协同训练提供几何感知的共识机制：在指数族期望参数空间做线性混合即得前向KL重心，可直接替换联邦/去中心化视觉训练中的权重平均，缓解异构站点间的分布漂移。
原文依据 Clients communicate in the expectation parameter space of an exponential family, where simple linear mixing yields a forward Kullback--Leibler (KL) barycenter consensus.
摘要译文
由模型翻译 · 术语统一 · 请对照原文核对关键结论完全去中心化的深度学习去除了全局服务器并保障本地数据隐私。然而，在数据非独立同分布（non-i.i.d.）且客户端规模不平衡的情形下，对权重、梯度或动量做欧氏平均的共识可能会退化。我们提出一种基于自然梯度变分推断的几何感知方法。客户端在指数族的期望参数空间中进行通信，其中简单的线性混合即可得到前向 Kullback–Leibler（KL）重心共识。该聚合结果是距离所有客户端分布最近的模型，从而对齐异构站点间的更新并缓解分布漂移。我们进一步给出一种轻量级的去中心化 Adam 实现，其中每个客户端维护一个对角高斯后验，并在期望空间中完成优化与共识。我们证明了在连通图上凸损失情形下的收敛性。在 CIFAR-100 与一个医学图像分割基准上，我们的方法在严重非独立同分布与客户端不平衡的情形下显著优于欧氏空间共识基线，在 CIFAR-100 上取得约 20% 的准确率提升，同时保持相同的通信预算并改善训练稳定性。代码见 https://github.com/x-lu/Beyond-euclidean-gossip。
英文原文摘要 1334 字符
Fully decentralized deep learning removes global servers and ensures local data privacy. However, Euclidean consensus, averaging weights, gradients or momentum, may degrade under the situations with non-i.i.d. (non-independent and identically distributed) data and client size imbalance. We propose a geometry-aware approach based on natural gradient variational inference. Clients communicate in the expectation parameter space of an exponential family, where simple linear mixing yields a forward Kullback--Leibler (KL) barycenter consensus. The aggregate is the model closest to all client distributions, aligning updates across heterogeneous sites and mitigating distribution shift. We further provide a lightweight decentralized Adam implementation where each client maintains a diagonal Gaussian posterior and operates entirely in the expectation space for both optimization and consensus. We prove convergence for convex losses on connected graphs. On CIFAR-100 and a medical image segmentation benchmark, our method substantially outperforms Euclidean-space consensus baselines under severe non-i.i.d. and client-imbalance cases, achieving around 20% accuracy gain on CIFAR-100, while matching the communication budget and improving training stability. The code is available at https://github.com/x-lu/Beyond-euclidean-gossip.