PaperlayerView exact revision on arXiv

Source-linked AI summary

Privacy For Free: Wireless Federated Learning Via Uncoded Transmission With Adaptive Power Control

Dongzhu Liu, Osvaldo Simeone

arXiv:2006.05459v4cs.ITcs.NIeess.SP

TL;DR

Wireless federated learning needs privacy guarantees beyond avoiding raw-data exchange. This paper uses channel noise and adaptive power control with uncoded OMA/NOMA transmission, showing privacy can be free below an SNR-dependent threshold while over-the-air computing benefits DP-constrained learning.

  • Problem

    Federated learning can reduce data leakage through communication, but formal differential-privacy guarantees generally require additional masking mechanisms.

  • Method

    The paper optimizes adaptive transmit power for uncoded OMA and NOMA gradient transmission under privacy and power constraints, using closed-form offline solutions and heuristic online control.

  • Results

    Privacy is obtained without learning-performance loss when the privacy level is below a threshold that decreases with SNR, while over-the-air computing benefits conventional OMA under DP constraints.

  • Takeaways & Limitations

    Dynamic power allocation and over-the-air computing can improve wireless federated learning under differential-privacy constraints relative to static allocation and conventional OMA.

  • Takeaways & Limitations

    The study leaves optimizing the channel-inversion threshold and implementing digital transmission with quantization-induced privacy as future work.

Abstract

from arXiv · show

Federated Learning (FL) refers to distributed protocols that avoid direct raw data exchange among the participating devices while training for a common learning task. This way, FL can potentially reduce the information on the local data sets that is leaked via communications. In order to provide formal privacy guarantees, however, it is generally necessary to put in place additional masking mechanisms. When FL is implemented in wireless systems via uncoded transmission, the channel noise can directly act as a privacy-inducing mechanism. This paper demonstrates that, as long as the privacy constraint level, measured via differential privacy (DP), is below a threshold that decreases with the signal-to-noise ratio (SNR), uncoded transmission achieves privacy "for free", i.e., without affecting the learning performance. More generally, this work studies adaptive power allocation (PA) for decentralized gradient descent in wireless FL with the aim of minimizing the learning optimality gap under privacy and power constraints. Both orthogonal multiple access (OMA) and non-orthogonal multiple access (NOMA) transmission with "over-the-air-computing" are studied, and solutions are obtained in closed form for an offline optimization setting. Furthermore, heuristic online methods are proposed that leverage iterative one-step-ahead optimization. The importance of dynamic PA and the potential benefits of NOMA versus OMA are demonstrated through extensive simulations.

I. INTRODUCTION … A. Learning Protocol

The paper develops differentially private wireless federated learning through direct uncoded gradient transmission, using channel noise and adaptive power control to balance privacy, convergence, and power constraints. It formulates protocols for OMA and NOMA and specifies the distributed gradient-descent learning procedure and privacy setting.

  • I. INTRODUCTION: Federated learning trains shared models without directly exchanging raw data, but communications about local models can still expose information about local datasets.Potential attacks include membership inference and model inversion, motivating formal differential-privacy guarantees.
  • I. INTRODUCTION: The paper exploits channel noise as a privacy mechanism for uncoded gradient transmission and studies adaptive power allocation for privacy-constrained learning.The analysis covers OMA and NOMA, with offline closed-form optimization and heuristic online one-step-ahead methods.
  • A. Wireless Federated Learning: Wireless federated learning can use multiple-access channels for centralized aggregation, with OMA and NOMA providing alternative communication protocols.NOMA aggregates simultaneous analog updates through the superposition property, while prior work also considers sparsification, power control, and device selection.
  • B. Differential Privacy for Federated Learning: Differential privacy quantifies information leakage through sensitivity to changing one data point, while the edge server is modeled as honest-but-curious.The server follows the protocol but may attempt to infer information about device data.
  • B. Differential Privacy for Federated Learning: Static power allocation overlooks that channel noise affects convergence and privacy differently across iterations.Earlier noise tends to impair convergence less than later noise, whereas privacy depends on a weighted sum of inverse noise power across iterations.
  • II. MODELS AND DEFINITIONS: The considered system has a single-antenna edge server and K single-antenna devices connected through a shared noisy channel, with each device holding a local dataset.Devices collaboratively train a common regression or classification model parameterized by w.
  • A. Learning Protocol: The learning objective is a regularized empirical average over the union of local datasets, and it is an unbiased estimate of generalization loss only for i.i.d. samples from a common distribution.The objective is nevertheless routinely used for non-i.i.d. federated-learning data.
  • A. Learning Protocol: At each communication round, the server broadcasts the current model over an ideal downlink, devices compute and transmit local gradients, and the server updates the model by gradient descent.These steps repeat until a convergence condition is met, while gradient transmission motivates the privacy framework.

B. Communication Model … C. Differential Privacy

The paper models uncoded wireless federated learning over block-fading channels using OMA and NOMA, with adaptive power control constrained by differential privacy. Privacy is defined against an honest-but-curious server through the distinguishability of received signals from neighboring datasets.

  • B. Communication Model: Devices upload uncoded gradient information to an edge server over a shared wireless channel, with NOMA enabling over-the-air computing and OMA serving as a reference.The paper focuses on uncoded NOMA while also studying uncoded OMA under the same transmission assumption.
  • B. Communication Model: The channel is block flat-fading: coefficients remain constant within each block, vary across blocks, and each block carries one gradient vector over d channel uses.The model dimension d is assumed limited because on-device machine-learning models are typically small.
  • B. Communication Model: Perfect CSI lets each device compensate its channel phase, yielding a real, non-negative effective channel gain and simplifying power-control design.The paper explicitly makes no optimality claim for this phase-compensation assumption.
  • 1) Orthogonal Multiple Access (OMA):: OMA uses TDMA, scheduling devices in successive orthogonal blocks; with I = KT blocks, the protocol implements T global gradient-descent iterations.In block i = K(t −1) + k, device k transmits gradient information for iteration t.
  • 2) Non-Orthogonal Multiple Access (NOMA):: NOMA synchronizes devices at the symbol level so all devices upload gradients simultaneously in each block, with I = T blocks matching the T iterations.Synchronization can use standard timing-advance procedures, and the received signal is denoted y(t) = y[t].
  • 2) Non-Orthogonal Multiple Access (NOMA):: For both access schemes, devices obey a transmit-power constraint, while the maximum SNR corresponds to P even though privacy requirements may require transmitting below P.The quantity dN0 represents channel-noise power within one communication block.
  • C. Differential Privacy: Differential privacy protects against an honest-but-curious server by bounding distinguishability between received-signal distributions generated by neighboring global datasets differing in one sample.The (ǫ, δ)-DP condition bounds the absolute privacy-loss variable by ǫ with probability at least 1 −δ.

D. Assumptions On the Loss Functions · III. ORTHOGONAL MULTIPLE ACCESS · A. Differential Privacy Analysis

The paper assumes smoothness and the Polyak–Łojasiewicz condition for the global loss, then analyzes OMA with uncoded transmission and adaptive power control under differential privacy constraints. Privacy depends on effective noise relative to useful signal and permits adaptation to changing channel gains when convergence effects are accounted for.

  • D. Assumptions On the Loss Functions: The global loss F(w) has an L-Lipschitz continuous gradient over R^d.This is the paper’s smoothness assumption: ∥∇F(w) − ∇F(w′)∥ ≤ L∥w − w′∥ for all w,w′ ∈ R^d.
  • D. Assumptions On the Loss Functions: The optimization problem has a non-empty solution set, and F(w) satisfies the Polyak–Łojasiewicz condition for some μ > 0.The PL condition is more general than strong convexity; strong convexity with constant μ implies the PL inequality with the same parameter.
  • III. ORTHOGONAL MULTIPLE ACCESS: OMA transmits scaled, noisy gradients, with artificial Gaussian noise added alongside channel noise and α_k^(t) ≥ 0 controlling scaling.The received signal’s effective noise is the sum of channel and artificial noise.
  • III. ORTHOGONAL MULTIPLE ACCESS: The OMA design optimizes parameter sequences to maximize learning performance subject to an (ε, δ)-differential privacy constraint.The section develops privacy and convergence analyses before formulating and solving the optimization problem.
  • III. ORTHOGONAL MULTIPLE ACCESS: OMA assumes each sample-wise gradient is bounded by a prescribed constant γ^(t) at every iteration.The bound applies to every possible training sample, not only samples in the devices’ local datasets.
  • A. Differential Privacy Analysis: For fixed power-control parameters, privacy is determined by the sensitivity of the noiseless received signal after excluding effective noise.The parameters are assumed not to reveal information about local datasets, so the received signal is the disclosed data-dependent function.
  • A. Differential Privacy Analysis: Lemma 1 states that OMA federated gradient averaging guarantees (ε, δ)-DP for any fixed parameter sequence satisfying the stated condition.The privacy condition is derived using advanced composition and depends on the effective-noise standard deviation.
  • A. Differential Privacy Analysis: Privacy depends on per-iteration useful-signal-to-effective-noise power ratios, while effective noise combines channel noise with device-added noise and can adapt to channel gains.Such adaptation is permitted when its effect on convergence is properly included.

B. Convergence Analysis · C. Optimization

The convergence analysis bounds the OMA average optimality gap by separating geometric decay from effective noise, with later noise more damaging than earlier noise. The optimization section minimizes this bound under differential-privacy and power constraints, revealing that strict privacy can make full-power transmission suboptimal.

  • B. Convergence Analysis: The OMA average optimality gap after T iterations is upper bounded under Assumptions 1 and 2 with learning rate η = 1/L.This result is stated in Lemma 2.
  • B. Convergence Analysis: The bound combines geometric decay of the initial optimality gap with a term capturing effective additive noise powers.The initial-gap contribution decreases geometrically as T increases.
  • B. Convergence Analysis: Noise added at earlier iterations is less damaging because its contribution is discounted by (1 − µ/L)^(T − t).Noise introduced later receives less discount and therefore has greater impact on the final optimality gap.
  • C. Optimization: The optimization minimizes the Lemma 2 optimality bound across K devices and T iterations under the (ε, δ)-DP constraint and power constraints.The optimization variables occur only in the bound’s noise-related second term.
  • C. Optimization: Replacing the effective-noise standard deviation with its definition yields the OMA optimization problem and separable OMA local problems for each device.The local problems can be solved in parallel.
  • C. Optimization: Without the DP constraint, the optimal strategy fully uses the power budget P for local-gradient transmission.The passages contrast this unconstrained solution with the privacy-constrained case.
  • C. Optimization: Under sufficiently strict privacy, full-power transmission may not remain optimal because the DP constraint changes the power-allocation solution.Thus, privacy can make a zero-noise-oriented power strategy suboptimal.
  • C. Optimization: To guarantee (ε, δ)-DP by leave-one-out analysis, G(t) must be fixed at each iteration independently of local data sets.The paper revisits this requirement when discussing online methods.

1) Offline Optimization: · 2) Online Optimization:

The offline method converts the privacy- and power-constrained allocation problem into a convex program with a closed-form optimal solution, including a regime where privacy is obtained without learning-performance loss. The online heuristic repeatedly applies this solution using predicted future parameters, channel evolution, and privacy-budget information while avoiding additional data leakage.

  • 1) Offline Optimization:: Known channel parameters enable offline optimization by converting the constrained problem into a convex program with an optimal closed-form solution.The offline formulation assumes the parameter sequence is known beforehand.
  • 1) Offline Optimization:: Under OMA, the offline solution uses full transmission power when channel noise alone satisfies the privacy constraint, yielding privacy “for free.”In this regime, the optimal solution matches the one without the differential-privacy constraint.
  • 1) Offline Optimization:: When the free-privacy condition fails, transmit power is scaled down to leverage channel noise and satisfy (ε, δ)-DP, with a minimum-power optimum obtainable by bisection.The optimal scaling becomes smaller under more stringent differential-privacy constraints, and time-invariant scaling is generally suboptimal.
  • 2) Online Optimization:: The online approach replaces unknown future parameters with predictions and applies the offline solution one step ahead using the residual privacy budget.The procedure repeats at each iteration with updated predictions and accumulated differential-privacy loss.
  • 2) Online Optimization:: Future fading-channel parameters are predicted from an autoregressive Rician model, with MMSE estimation of future channel power from current CSI.The diffuse component follows an AR(1) process with temporal correlation coefficient 0 ≤ρ_k ≤1.
  • 2) Online Optimization:: To estimate data-dependent gradient bounds without additional leakage, devices predict future bounds from an additional server-broadcast signal.The server broadcasts ∥y^(t−1)∥ alongside the model signal, and receiving this signal does not affect privacy.
  • 2) Online Optimization:: The online implementation clips local gradients before transmission to enforce bounded-gradient constraints, while the proposed heuristic performs well in practice despite clipping-related convergence-analysis limitations.The approach is specified for OMA in the cited implementation passage.

IV. NON-ORTHOGONAL MULTIPLE ACCESS

The NOMA design uses uncoded local-gradient transmission with scaling factors chosen for noiseless recovery of a scaled global gradient. Its effective noise combines channel noise with artificial noise from all devices, while optimization jointly considers scaling parameters and added-noise power.

  • NOMA design: NOMA transmits local gradients using the uncoded strategy (16) at each iteration.The design analyzes NOMA under iterative uncoded transmission.
  • NOMA design: Scaling factors α(t)_k are selected so the edge server recovers a scaled version of the global gradient (5) without noise.The selection follows gradient alignment, with h(t) aligned through a constant c(t).
  • Effective noise: The effective NOMA noise sums channel noise and artificial-noise contributions from all devices.Its standard deviation is specified by the resulting aggregate-noise expression.
  • Optimization variables: The section optimizes the parameters (c(1), · · · , c(T)) together with the added-noise power σ(1)_k.This optimization is stated as part of (36).

A. Differential Privacy Analysis

The NOMA differential-privacy analysis defines sensitivity through the received noiseless aggregated signal and derives an (ε, δ)-DP condition using the corresponding effective noise. This effective noise combines channel noise with privacy-inducing noise from all devices.

  • Sensitivity: For NOMA, the disclosed function’s sensitivity is defined by the received noiseless aggregated signal and bounded under Assumption 3.The sensitivity concerns changes to device k’s data set.
  • DP guarantees: Federated gradient averaging via NOMA guarantees (ε, δ)-DP when the condition in Lemma 3 is satisfied.The condition uses m(t), the standard deviation of the effective noise.
  • Effective noise: NOMA’s effective privacy noise equals the sum of channel noise and privacy-inducing noise added by all devices.The passage identifies this aggregate noise as an important advantage of NOMA.

B. Convergence Analysis · C. Optimization · 1) Offline Optimization:

The paper bounds NOMA’s average optimality gap under decentralized gradient descent, then formulates privacy- and power-constrained optimization. Its offline NOMA solution is convex, generally coupled across devices, and can obtain privacy for free under a stated condition.

  • B. Convergence Analysis: At iteration t, the edge server estimates the global gradient using the effective wireless aggregation process.
  • B. Convergence Analysis: Lemma 4 upper bounds the average optimality gap after T iterations for NOMA with learning rate η = 1/L.The bound depends on the standard deviation m(t) of the effective noise.
  • C. Optimization: NOMA optimization minimizes the optimality bound subject to the (ǫ, δ)-DP constraint and power constraints across T iterations.
  • C. Optimization: Without differential privacy, devices with the smallest ratio h(t) determine the optimal solution.The solution must also satisfy the gradient alignment condition (36).
  • C. Optimization: Under sufficiently strict differential privacy constraints, setting c(t)k = 0 is no longer optimal.
  • 1) Offline Optimization:: With known parameters, the offline NOMA problem becomes convex after a variable change but cannot be solved in parallel across devices.Theorem 2 characterizes its optimal offline solution.
  • 1) Offline Optimization:: Under the stated condition, devices need not add transmitted noise, and privacy is obtained “for free” because the DP-constrained optimum matches the unconstrained optimum.Otherwise, multiple optima exist; scaling all transmitted powers down leverages channel noise to ensure (ǫ, δ)-DP, while larger Dk makes the free-privacy condition less restrictive.

2) Online Optimization: · V. NUMERICAL RESULTS

The online NOMA scheme uses predicted channels and estimated parameters to adapt power allocation while enforcing privacy and power constraints, but makes no optimality claim. Numerical experiments evaluate privacy effects and adaptive power allocation benefits under defined synthetic-data, communication, and benchmark settings.

  • 2) Online Optimization:: Online NOMA power allocation uses predicted channels, clipped per-sample gradients, and received-signal estimates averaged over the global dataset.The procedure predicts channels as in (32), sets bγ through (35), and obtains {bG(t)k} from the previous received signal.
  • 2) Online Optimization:: The NOMA transmission rule modifies clipped gradient transmission when estimated parameters would make optimized c(t) violate the power constraint.The passage identifies power-constraint violation as a final issue and introduces a modified clipped gradient transmission rule.
  • 2) Online Optimization:: The proposed online NOMA scheme is evaluated numerically without a claim of optimality.The authors explicitly state that they make no claims of optimality for NOMA and test the scheme in the next section.
  • 2) Online Optimization:: Algorithm 2 repeatedly predicts future quantities, applies Theorem 2 over the remaining interval, broadcasts c(t), and updates privacy loss.The algorithm operates for t = 1, · · · , T and includes predictors, theorem-based optimization, scaling-factor broadcasts, and privacy-loss updates.
  • V. NUMERICAL RESULTS: The numerical study begins with a synthetic dataset of Dtot = 10000 pairs in R10, distributed evenly across K = 10 devices.Covariates are drawn i.i.d. as N(0, I), and labels follow the specified linear-plus-noise construction.
  • V. NUMERICAL RESULTS: The experiments use ridge regression with λ = 5×10−5, an all-zero initial model, and curvature parameters computed from the data Gramian matrix.The local dataset size is Dk = 1000 for all k, with PL and smoothness parameters given by the smallest and largest Gramian eigenvalues.
  • V. NUMERICAL RESULTS: The default configuration sets SNRmax = 30 dB, provides 30 communication blocks, and uses ε = 20 and δ = 0.01.These settings imply T = 3 iterations per device for OMA and T = 30 iterations for NOMA.
  • V. NUMERICAL RESULTS: Performance is measured by the normalized optimality gap and compared against equal-per-iteration privacy allocation and a scheme without the DP constraint.Offline results average 1000 channel realizations, while online results average 100 channel realizations.

A. Offline Optimization … VI. CONCLUSIONS

The paper evaluates adaptive power allocation for privacy-constrained wireless federated learning across offline, online, and MNIST settings. Results show benefits from adaptive PA and NOMA, while identifying privacy–learning tradeoffs and unresolved convergence questions.

  • A. Offline Optimization: NOMA with adaptive or static PA outperforms OMA across the considered privacy levels, while adaptive PA gains most under stringent DP constraints.Adaptive PA’s advantage decreases as ǫ increases, and privacy-for-free thresholds are identified.
  • A. Offline Optimization: Optimality gaps decrease with SNR until DP becomes the bottleneck; OMA with optimal PA can outperform NOMA with static PA at high SNR.This highlights the importance of power-allocation optimization, particularly in the high-SNR regime.
  • A. Offline Optimization: Increasing data-set heterogeneity harms all schemes, affects NOMA more strongly than OMA, and makes NOMA power constraints increasingly performance-limiting.For NOMA, adaptive PA converges first to the unconstrained performance and then to static PA.
  • B. Online Optimization: Under privacy constraints, increasing the communication budget always benefits adaptive PA, whereas conventional static PA has an optimal budget because additional blocks can increase privacy loss.NOMA retains an advantage over OMA at large I under privacy constraints.
  • B. Online Optimization: Increasing the number of users worsens OMA but does not harm NOMA, emphasizing NOMA’s spectral efficiency in wireless edge learning.For OMA, more users imply fewer iterations and less information leakage per user, reducing adaptive PA’s performance gain.
  • B. Online Optimization: Under DP constraints, increasing the clipping threshold beyond a given value degrades performance because privacy requires stronger scaling down of transmitted signals.Without DP constraints, larger thresholds reduce gradient distortion, so threshold selection must balance learning and privacy.
  • C. MNIST Data Set: On MNIST logistic regression, adaptive PA significantly outperforms static PA for OMA in both training cross-entropy loss and test-set error.The experiment uses ǫ = 5, δ = 0.01, SNRmax = 13 dB, and communication budget I as the varied quantity.
  • VI. CONCLUSIONS: The analysis shows privacy can be obtained “for free” below an SNR-dependent threshold, while over-the-air computing provides significant DP benefits over conventional OMA.The paper also finds active noise addition generally suboptimal, notes channel inversion can amplify noise, and leaves online convergence and digital transmission for future work.

APPENDIX · A. Proof of Lemma 1 · B. Proof of Lemma 2

The appendix proves Lemmas 1 and 2 by deriving differential-privacy bounds from Gaussian effective noise and iterating an expected optimization inequality under the stated assumptions.

  • A. Proof of Lemma 1: The analysis models the received-signal uncertainty through an effective-noise standard deviation that includes N0.The supplied derivation identifies the effective noise and its standard deviation in the received signals.
  • A. Proof of Lemma 1: The proof represents the k-th device’s privacy loss after T iterations using the differential-privacy loss definition in (12).This establishes the privacy-loss expression before bounding its violation probability.
  • A. Proof of Lemma 1: The proof bounds privacy violation probability using a Gaussian tail-probability inequality and the referenced privacy-loss bound.The argument applies the Gaussian tail bound and then uses the resulting inequality to obtain the DP condition.
  • A. Proof of Lemma 1: The final step uses the monotonicity of πxex2 to establish the desired result.This completes the inequality manipulation in the proof of Lemma 1.
  • B. Proof of Lemma 2: Under Assumption 1, the proof of Lemma 2 begins from an equality and takes expectation over the additive noise.The expectation is applied to both sides of the preceding inequality.
  • B. Proof of Lemma 2: Using Lemma 1 with η = 1/L, the proof subtracts the optimal value F ∗ and invokes Assumption 2 for the final inequality.The resulting bound is applied repeatedly through T iterations while taking expectation over all additive noises.

C. Proof of Theorem 1

The proof transforms the original variables into an equivalent convex local optimization problem, solves it via KKT conditions, and maps the resulting solution back to the original variables. The optimal transmit-power-minimizing solution is obtained by selecting the maximum feasible transformed parameter.

  • A change of variables converts the original formulation into an equivalent local optimization problem with the included constraints.
  • The resulting problem is convex, so its solution is characterized by necessary and sufficient KKT conditions.The Lagrange multipliers correspond to the DP, transmit-power, and non-negative-parameter constraints.
  • The optimal dual variable ζopt is found by bisection search to satisfy the equality in (58f), with ζopt = 0 when (P_T^k)^2 < R_dp.
  • Reverting to the original variables completes the theorem proof, with minimum transmit power attained by the maximum feasible value of (b(t)_k)opt.
Loading 2006.05459v4…