Federated Learning: Strategies for Improving Communication Efficiency

The paper addresses communication efficiency in federated learning, a setting where a high-quality central model must be trained even though the training data remains spread across a large number of clients, often mobile phones, with slow and unreliable connections. In each round, every client independently computes an update to the current model from its local data and transmits it to a central server, which aggregates the client-side updates into a new global model. Because uplink bandwidth is the bottleneck, the authors introduce two complementary compression strategies: structured updates and sketched updates.

Structured updates are learned directly within a restricted space parametrized by fewer variables, for example low-rank matrices or a random mask, while sketched updates learn a full model update and then compress it through a combination of quantization, random rotations, and subsampling before sending it to the server. In experiments on both convolutional and recurrent networks, these techniques reduced the uplink communication cost by two orders of magnitude, addressing a central obstacle to making federated learning practical on bandwidth-constrained devices.