Textbook of AI

Federated Learning is a distributed machine learning approach in which models are trained across many decentralised devices (smartphones, hospitals, IoT devices) without centralising the raw data. Each device trains the model locally on its own data and sends only model updates—gradients or parameter deltas—to a central server, which aggregates them to produce an updated global model. The raw data never leaves the device.

Proposed by McMahan et al. (2017) at Google, federated learning was motivated by the privacy and regulatory challenges of collecting sensitive data at a central location. The canonical example is next-word prediction on mobile keyboards: each phone trains a language model on the user's typing without the user's keystrokes ever being uploaded. Medical federated learning allows hospitals to jointly train diagnostic models without sharing patient records. Cross-silo federated learning (few powerful participants) and cross-device federated learning (many resource-constrained devices) present different engineering challenges.

Federated learning is not automatically private: gradient updates can leak information about individual examples, as demonstrated by gradient inversion attacks. Combining federated learning with differential privacy (adding noise to aggregated updates), secure aggregation (cryptographic protocols ensuring the server sees only the sum of updates), and homomorphic encryption provides stronger privacy guarantees. The approach has limitations: communication costs, handling non-i.i.d. data across devices, dealing with stragglers and dropped clients, and coordinating optimisation across heterogeneous hardware. Despite these challenges, federated learning has become an important tool for privacy-preserving ML in production.