AI and the World's Hard Problems · Theoretical computer science · arXiv · 2026-10-21
Federated learning with heavy-tailed gradient noise and communication noise: a variance-reduction based algorithm
Federated learning lets local devices jointly train a global model while keeping data decentralized and private, but heavy-tailed gradient noise and communication noise are prevalent in large-scale machine learning over wireless networks and IoT deployments. The proposed algorithm, VRA-FedSGD, applies momentum variance reduction with a nonlinear mapping to mitigate heavy-tailed gradient noise, and uses a variance-reduced aggregation mechanism to suppress heavy-tailed communication noise. The authors derive convergence rates for nonconvex and strongly convex objectives, and simulated experiments on logistic regression with real-world data verify its effectiveness.