Source-linked AI summary
Wireless Federated Learning with Local Differential Privacy
Mohamed Seif, Ravi Tandon, Ming Li
TL;DR
The paper studies federated learning over a Gaussian multiple access channel under local differential privacy constraints. It proposes analog wireless aggregation using channel-aligned gradients and artificial Gaussian noise, and analyzes the resulting privacy, convergence, and resource tradeoffs. Privacy leakage per user scales as O(1/sqrt(K)) rather than remaining constant under orthogonal transmission.
Problem
The paper asks whether wireless superposition can improve privacy in federated learning while balancing convergence, wireless resources, and privacy constraints.
Method
The proposed analog aggregation scheme aligns users’ gradients at the parameter server and adds artificial Gaussian noise selected to satisfy local differential privacy.
Results
Privacy leakage per user scales as O(1/sqrt(K)), compared with constant scaling for orthogonal transmission.
Takeaways & Limitations
Wireless superposition provides bandwidth-efficient gradient aggregation together with strong local differential privacy guarantees, while the paper characterizes convergence-resource-privacy tradeoffs.
Takeaways & Limitations
Aligning all users’ gradients makes effective SNR limited by the user with the worst channel conditions.
Abstract
from arXiv · showhide
In this paper, we study the problem of federated learning (FL) over a wireless channel, modeled by a Gaussian multiple access channel (MAC), subject to local differential privacy (LDP) constraints. We show that the superposition nature of the wireless channel provides a dual benefit of bandwidth efficient gradient aggregation, in conjunction with strong LDP guarantees for the users. We propose a private wireless gradient aggregation scheme, which shows that when aggregating gradients from $K$ users, the privacy leakage per user scales as $\mathcal{O}\big(\frac{1}{\sqrt{K}} \big)$ compared to orthogonal transmission in which the privacy leakage scales as a constant. We also present analysis for the convergence rate of the proposed private FL aggregation algorithm and study the tradeoffs between wireless resources, convergence, and privacy.
1 Introduction
Federated learning keeps raw local data at users while exchanging gradients, but raw gradients can still leak information. This motivates private FL and wireless aggregation, culminating in a scheme that combines privacy with bandwidth-efficient training.
- Motivation: Federated learning enables multiple users to jointly train a model while retaining local datasets at each user.Users compute local gradients, which are exchanged for model updates.
- Motivation: Raw gradient exchange can leak information even when users never share their local data directly.
- Privacy: Local differential privacy lets users perturb and disclose data to an untrusted aggregator, but stronger privacy generally reduces utility.
- Wireless FL: Wireless FL uses channel superposition to support gradient aggregation efficiently over a shared wireless medium.
- Contribution: The proposed Gaussian-MAC scheme combines channel-aligned local gradients with artificial Gaussian noise to satisfy user-level LDP constraints.
- Contribution: Privacy leakage per user scales as O(1/sqrt(K)), whereas orthogonal transmission yields privacy leakage that scales as a constant.
2 System Model & Problem Statement
The paper models FL as iterative distributed gradient descent over a Gaussian MAC, with users transmitting local information under power and LDP constraints. It studies how channel conditions and wireless resources affect convergence and privacy.
- System Model: The system has K users and a central parameter server communicating over a single-antenna Gaussian multiple access channel.Users know their local channel gains, while the parameter server has global channel-state information.
- System Model: Each user transmits over a time-invariant channel with a maximum power constraint P_k, and the received signal includes additive Gaussian noise.
- Federated Learning Problem: Users hold private local datasets and communicate with the parameter server to minimize the aggregate loss F(w).The global dataset is the union of the users’ local datasets.
- Federated Learning Problem: At each iteration, the parameter server broadcasts w_t, users compute local gradients, and the server updates the global parameter using distributed gradient descent.The iteration continues until convergence.
- Privacy Constraint: Wireless gradient descent must satisfy local differential privacy constraints for every user.The paper defines both (ϵ, δ)-LDP and pure ϵ-LDP when δ = 0.
- Problem Statement: The study evaluates tradeoffs among gradient-descent convergence, wireless channel conditions, power, SNR, and users’ privacy budgets.
3 Main Results & Discussions
The paper develops an over-the-air private gradient aggregation scheme for wireless FL and analyzes its privacy leakage, convergence, and resource tradeoffs. Gradient alignment and artificial Gaussian noise support unbiased aggregation while enabling privacy guarantees under wireless channel constraints.
- Private wireless gradient aggregation: The proposed scheme transmits local gradients with artificial Gaussian noise and aligns gradient components at the parameter server.Users dedicate transmission power to gradient vectors and privacy noise, while the receiver performs post-processing on the aggregated signal.
- Private wireless gradient aggregation: The aligned-gradient signal is limited by the user with the worst effective SNR, min_j |h_j|^2P_j.The alignment coefficient is selected using the minimum effective channel-and-power product across users.
- Private wireless gradient aggregation: The processed gradient estimate is unbiased because the effective noise is zero mean, with its variance determined by the aggregated channel, artificial, and AWGN terms.The scheme combines local gradient estimates at the parameter server and treats the resulting noise as a perturbation of the true gradient.
- Local differential privacy analysis: The Gaussian mechanism and bounded gradient sensitivity yield per-user (ϵ_k, δ)-LDP guarantees for each training iteration.The privacy analysis applies the Gaussian mechanism to the received signal after bounding the effect of changing one user's local dataset.
- Local differential privacy analysis: O(1/√K) asymptotic per-user privacy leakage decreases with the number of users, whereas orthogonal transmission leakage scales as a constant.Across T iterations, the paper uses advanced composition to analyze total leakage and reports that wireless-FL leakage approaches zero as K increases.
- Convergence rate of private FL: The convergence analysis assumes a λ-strongly convex and µ-smooth loss, and optimizes artificial-noise parameters to maximize convergence under desired LDP constraints.The paper formulates the noise allocation as an optimization problem and studies the resulting privacy-convergence tradeoff.
4 Simulation Results
Simulations evaluate private wireless FL on synthetic linear regression, showing how users, transmission mode, and power affect training loss.
- Experimental setup: The simulations use λ = 10^-3, T = 1000 training iterations, unit channel-noise variance, and per-user privacy level (ϵ, δ) = (1.2, 10^-4)-LDP.Channel coefficients are drawn from CN(0, 1).
- Simulation findings: Increasing the number of users makes the training loss decay faster with T when P_k = 30 dBm for all users.This is shown in Fig. 4(a).
- Simulation findings: For matched iteration budgets and P_k = 30 dBm, non-orthogonal transmission is more efficient in bandwidth and accuracy than the private orthogonal scheme.The comparison uses K T_2 = T_1 = T iterations.
- Simulation findings: Increasing transmit power makes the training error decay faster with T.This is shown in Fig. 4(c).
5 Conclusion & Future Directions
The paper concludes that wireless superposition jointly improves bandwidth efficiency and LDP guarantees, while exposing tradeoffs among privacy, convergence, and wireless resources. It identifies worst-channel alignment as a limitation and smaller-user subsets as a future direction.
- Conclusion: The proposed wireless aggregation scheme provides bandwidth efficiency together with strong LDP guarantees.The conclusion frames these as dual benefits of the wireless channel.
- Conclusion: Privacy leakage under the proposed scheme decreases with the number of users, whereas orthogonal transmission has constant privacy leakage.The supplied conclusion passage states the comparison but does not include the scaling exponent.
- Conclusion: As T increases, variance from local privacy perturbation and noisy-channel effects becomes dominant in the training loss.This behavior is described for the number-of-users, transmission-mode, and transmit-power comparisons.
- Conclusion: The paper analyzes and optimizes convergence while studying tradeoffs between wireless resources, convergence, and privacy.These tradeoffs include effects visible in training-loss behavior as iterations increase.
- Future directions: Aligning all users’ gradients limits effective SNR by the user with the worst channel conditions.Future work could align gradients from a smaller subset of users and generalize to multiple antennas.
Appendix I: Proof of Theorem 2
The proof establishes convergence prerequisites for the wireless gradient estimate and then applies standard SGD convergence results for smooth, strongly convex losses. The resulting bound is obtained by substituting the estimated second-moment bound into the convergence expression.
- Proof strategy: The gradient estimate at the parameter server is unbiased because the total additive noise has zero mean.This is the first property used in the convergence proof.
- Proof strategy: The proof establishes a bounded second moment for the gradient estimate, with the bound denoted G^2.The argument uses Cauchy–Schwarz and the bounded-gradient assumption.
- Assumptions: The bounded-gradient condition is expressed as ∥g_k(w_t)∥_2 ≤ L for every user k.The passage identifies L as the relevant Lipschitz constant.
- Proof strategy: Standard SGD convergence results for μ-smooth and λ-strongly convex losses are invoked after establishing the estimator properties.These results provide the convergence expression used in the theorem.
- Proof strategy: Substituting G^2 from equation (21) into equation (22) yields Theorem 2.This is the final algebraic step described in the proof.