Source-linked AI summary
Differentially Private Federated Learning: A Client Level Perspective
Robin C. Geyer, Tassilo Klein, Moin Nabi
TL;DR
The paper addresses the risk that federated model parameters reveal a client’s participation or data despite decentralized training. It incorporates randomized mechanisms into federated learning to provide client-level differential privacy, finding that sufficiently many participating clients can achieve this with only a minor performance cost.
Problem
Federated model parameters can reveal information about training data or whether a client participated, so privacy protection must extend beyond keeping raw data undisclosed.
Method
The paper incorporates a randomized, Gaussian-mechanism-based privacy procedure into federated learning and adapts the mechanism during decentralized training.
Results
For K = 100 and 1000, accuracy reaches 78% and 92%, while for K = 10000 the private model almost reaches non-private accuracy.
Takeaways & Limitations
Client-level differential privacy is feasible and can preserve high accuracy when sufficiently many parties participate, with privacy budgeting optimized through update-distribution analysis.
Abstract
from arXiv · showhide
Federated learning is a recent advance in privacy protection. In this context, a trusted curator aggregates parameters optimized in decentralized fashion by multiple clients. The resulting model is then distributed back to all clients, ultimately converging to a joint representative model without explicitly having to share the data. However, the protocol is vulnerable to differential attacks, which could originate from any party contributing during federated optimization. In such an attack, a client's contribution during training and information about their data set is revealed through analyzing the distributed model. We tackle this problem and propose an algorithm for client sided differential privacy preserving federated optimization. The aim is to hide clients' contributions during training, balancing the trade-off between privacy loss and model performance. Empirical studies suggest that given a sufficiently large number of participating clients, our proposed procedure can maintain client-level differential privacy at only a minor cost in model performance.
1 Introduction
Federated learning keeps client data decentralized, but trained model parameters can still reveal information about participation or data use. The paper proposes client-level differential privacy to hide whether a client participated while preserving model performance.
- Standard machine learning approaches often overlook privacy, and anonymization alone may not sufficiently protect data used by deployed services.
- Federated learning trains a shared model across decentralized clients while a trusted curator centralizes learned parameters and redistributes the aggregate model.
- Model parameters can reveal information about the data used during training, even when clients do not disclose their raw data.
- The proposed mechanism protects a client’s entire dataset by hiding whether that client participated in decentralized training.
- Client participation can be hidden while model performance remains high, with client-level differential privacy achieved at a minor performance loss.
2 Background
Federated optimization reduces the need to share raw data but must account for communication constraints, heterogeneous client data, and privacy leakage through learned parameters. Differential privacy mechanisms add calibrated randomness, track cumulative privacy loss, and can be adapted to protect an entire client dataset.
- Federated optimization addresses limited communication and non-IID, unbalanced, massively distributed client data through repeated curator–client model updates.
- Centralized private learning can add a randomized mechanism such as the Gaussian mechanism to the training process.
- The paper uses (ε, δ)-differential privacy, where δ represents the probability that pure ε-differential privacy is violated.
- The Gaussian mechanism adds noise calibrated to a function’s dataset sensitivity, defined by the maximum Euclidean distance between outputs on adjacent inputs.
- Privacy accounting tracks cumulative δ across Gaussian-mechanism queries and stops answering once a threshold is reached.
- Unlike data-level private learning, the proposed federated mechanism protects whether an entire client participated while maintaining high model performance.
3 Method
The method protects client-level participation privacy by replacing federated averaging with randomized subsampling, clipping, and Gaussian noise. Privacy loss is tracked during training while noise, sampling, and clipping are tuned to balance privacy against model distortion and learning progress.
- Randomized aggregation: Random sub-sampling selects m_t clients each round, who receive the central model, optimize locally, and return client updates Δw_k.
- Randomized aggregation: Client updates are clipped to an l_2 norm bounded by S, giving the update sum sensitivity at most S.The clipped update is Δ̄w_k = Δw_k/max(1, ∥Δw_k∥_2/S).
- Randomized aggregation: A Gaussian mechanism adds noise to the clipped update sum; dividing by m_t produces an approximate average used to form w_{t+1}.
- Privacy accounting: The moments accountant evaluates δ from ε, σ, and m, and training stops when δ reaches its prescribed threshold.
- Parameter choices: The clipping bound S is set each round to the median norm of unclipped client contributions, although computing this median is not randomized.The authors state that leakage through the median is small and defer a privacy measure for it to future work.
- Parameter choices: The distortion-to-privacy trade-off depends on r = σ^2/m: larger σ and smaller m increase privacy loss, while smaller σ and m can reduce loss at fixed distortion.
4 Experiments
Experiments simulate federated learning on non-IID MNIST with 100, 1,000, or 10,000 clients, evaluating accuracy under fixed ε and client-specific δ stopping thresholds. Hyperparameters are selected by cross-validation to maximize accuracy under each privacy bound.
- Experimental setup: Clients receive two MNIST shards, so most clients observe examples from only two digits and cannot individually achieve high accuracy across all ten digits.
- Experimental setup: The study evaluates K ∈ {100, 1000, 10000} clients, with exactly 600 data points per client; data points are repeated for the two larger settings.
- Experimental setup: Cross-validation searches over client batches, local epochs, participating clients per round m, and Gaussian-mechanism parameter σ.
- Privacy accounting: ε is fixed at 8, while training stops at δ values of e^-3, e^-5, and e^-6 for 100, 1,000, and 10,000 clients, respectively.
- Evaluation: The reported table compares differentially private and non-differentially private models using accuracy, communication rounds, and communication costs.The non-private benchmark is reported for K = 100.
5 Results
The experiments select models that maximize accuracy under the relevant δ bound, preferring fewer communication rounds when accuracies tie. Figure 1 tracks accuracy until privacy-threshold stopping, while Figure 2 jointly tracks accuracy, client-update variance, and update scale.
- Model selection: Models are chosen for highest accuracy below the applicable δ bound, with fewer communication rounds preferred when accuracies are equal.
- Reported results: Table 1 reports ACC, CR, and CC for differentially private settings with 100, 1,000, and 10,000 clients, alongside a non-private K = 100 benchmark.
- Accuracy over training: Figure 1 plots digit-classification accuracy over decentralized training on non-IID MNIST and marks where differentially private runs reach the δ threshold and stop.
- Update statistics: Figure 2 plots accuracy, between-client variance, and update scale over training for 100 non-differentially private clients.
6 Discussion
Model performance improves as more clients participate, approaching non-private accuracy at K = 10000. The discussion also shows that dynamically adapting privacy parameters can reduce privacy loss early while preserving later accuracy gains.
- 78% and 92% accuracy for K ∈{100, 1000} remain substantially better than training on clients’ own data, despite staying below non-private performance.For these client counts, model accuracy does not converge and remains significantly below the non-differentially private baseline.
- For K = 10000, the differentially private model almost reaches the accuracy of the non-differentially private model.This suggests that privacy incurs almost no performance cost when many parties participate, including mobile phones and other consumer devices.
- Raising m_t over training improves model performance.
- Lowering m_t and σ_t together while keeping σ_t^2/m_t constant has almost no impact on early-round accuracy gain but reduces privacy loss.The reduced privacy loss allows more later communication rounds before the privacy budget is drained.
- During label fitting, client updates are similar and V_c is low, whereas U_c is high because updates to randomly initialized weights are large.During data fitting, V_c rises, linking the observed training behavior to two proposed phases of learning.
7 Conclusion
Client-level differential privacy is empirically feasible, with high model accuracies when sufficiently many parties participate. Studying data and update distributions can also improve privacy budgeting.
- Client-level differential privacy is feasible, and high model accuracies can be reached when sufficiently many parties participate.
- Careful investigation of data and update distributions can lead to optimized privacy budgeting.