PaperlayerView exact revision on arXiv

Source-linked AI summary

A Framework for Evaluating Gradient Leakage Attacks in Federated Learning

Wenqi Wei, Ling Liu, Margaret Loper, Ka-Ho Chow, Mehmet Emre Gursoy, Stacey Truex, Yanzhao Wu

arXiv:2004.10397v2cs.LGcs.CRstat.ML

TL;DR

Federated learning’s shared updates can leak the private data that remains on clients, raising the question of how such attacks should be systematically evaluated. The paper develops and applies a framework for comparing attack effectiveness and cost across attack, training, and compression configurations, with preliminary mitigation experiments. Across four benchmark datasets, the framework demonstrates reconstructable client data and supports analysis of mitigation strategies.

  • Problem

    Although federated learning keeps training data on clients, shared updates can still enable gradient-based reconstruction of that data.

  • Method

    The paper builds a principled evaluation framework that characterizes attacks, varies attack and federated-learning configurations, and measures effectiveness, cost, compression effects, and mitigation.

  • Results

    Across four benchmark datasets, experiments show that shared parameter updates can reveal private training data and that attack outcomes vary with configurations, compression, and mitigation settings.

  • Takeaways & Limitations

    Systematic measurement is important for understanding client privacy leakage threats and developing and evaluating mitigation strategies in federated learning.

  • Takeaways & Limitations

    The threat model assumes limited client compromise and an honest-but-curious federated server that may inspect client updates.

Abstract

from arXiv · show

Federated learning (FL) is an emerging distributed machine learning framework for collaborative model training with a network of clients (edge devices). FL offers default client privacy by allowing clients to keep their sensitive data on local devices and to only share local training parameter updates with the federated server. However, recent studies have shown that even sharing local parameter updates from a client to the federated server may be susceptible to gradient leakage attacks and intrude the client privacy regarding its training data. In this paper, we present a principled framework for evaluating and comparing different forms of client privacy leakage attacks. We first provide formal and experimental analysis to show how adversaries can reconstruct the private local training data by simply analyzing the shared parameter update from local training (e.g., local gradient or weight update vector). We then analyze how different hyperparameter configurations in federated learning and different settings of the attack algorithm may impact on both attack effectiveness and attack cost. Our framework also measures, evaluates, and analyzes the effectiveness of client privacy leakage attacks under different gradient compression ratios when using communication efficient FL protocols. Our experiments also include some preliminary mitigation strategies to highlight the importance of providing a systematic attack evaluation framework towards an in-depth understanding of the various forms of client privacy leakage threats in federated learning and developing theoretical foundations for attack mitigation.

1 Introduction

Federated learning keeps training data on client devices while sharing local updates, but those updates can still expose private data. The paper introduces a framework to evaluate attack effectiveness, cost, configuration effects, compression, and mitigation strategies.

  • Federated learning enables decentralized, continuous on-device training without centralizing all training data.
  • Sharing local parameter updates does not guarantee privacy because adversaries can reconstruct client training data from intercepted gradients.
  • The framework evaluates how attack and federated-learning configurations affect privacy-leakage success rates and costs.
  • The study also evaluates leakage under gradient compression and examines preliminary mitigation strategies.

2 Problem Formulation

This section defines federated learning’s training and communication workflow, threat model, and client privacy leakage attack. It frames the framework around evaluating how federated-learning and attack configurations affect leakage success and cost.

  • 2.1 Federated learning: Federated learning distributes training across N clients, with only a selected subset participating in each round and sharing local updates with the server.Clients download the global state, train locally, and choose batch size and local iterations before sharing updates.
  • 2.1 Federated learning: Distributed SGD uploads local gradients for iterative aggregation, whereas federated averaging uploads parameter updates that the server combines through weighted averaging.The notation uses n_k for client k’s data points and n for the total participating data at round t.
  • 2.1 Federated learning: The baseline protocol sends full update vectors, while communication-efficient protocols compress updates to reduce communication with limited value loss.The framework explicitly evaluates leakage against both baseline and sketched-update protocols.
  • 2.2 Threat Model: The threat model allows an adversary to intercept local parameter updates and access a compromised client’s saved model executable, without accessing its private training data.The federated server is honest but curious: it performs aggregation while potentially analyzing client updates for leakage.
  • 2.2 Threat Model: Client privacy leakage is a gradient-based feature-reconstruction attack that uses a shared gradient or update to reconstruct private training data.The attack applies to federated and decentralized settings, including image or video training data.
  • 2.3 The Client Privacy Leakage (CPL) Attack: An Overview: The reconstruction procedure initializes dummy data, compares its gradient with the client gradient using a distance loss, and iteratively optimizes the dummy input.Algorithm 1 takes the model, gradients, model parameters, learning rate, and attack configurations as inputs and outputs reconstructed data.
  • 2.3 The Client Privacy Leakage (CPL) Attack: An Overview: When a weight update is shared, the algorithm first converts it to a gradient; it then obtains a label from the shared gradient and optimizes until the reconstruction converges or fails.Minimizing the L2 distance between reconstructed and actual gradients is described as leading to leakage, with a label-based regularizer improving stability.
  • 2.3 The Client Privacy Leakage (CPL) Attack: An Overview: The framework studies how attack initialization, termination, loss, optimization, and federated-learning hyperparameters affect attack success rate and cost.Figure 1 illustrates the client privacy leakage attack, while the framework compares vulnerabilities in baseline and communication-efficient protocols.

3 Evaluation Framework

This section evaluates how dummy-data initialization, random seeds, and attack termination choices affect client privacy leakage reconstruction. It combines convergence analysis with experiments comparing initialization strategies and seed sensitivity.

  • Attack Initialization: The framework studies how dummy attack seed initialization influences reconstruction quality, confidence, convergence, and attack effectiveness.It contrasts random, patterned, color-based, dark/light, and theoretically optimal initialization methods.
  • Convergence Analysis: Under convexity and Lipschitz-smoothness assumptions, gradient-based reconstruction attack convergence is guaranteed.The convergence analysis motivates studying how the initial dummy data affects attack efficiency and convergence.
  • Geometrical Initialization: Geometrical initialization can speed convergence and improve attack stability by transforming the layer input around a geometrically chosen parameterization.The paper represents this choice using g(x) = σ(w*(x − b*)) rather than direct parameterization.
  • Initialization Comparison: CPL-patterned, CPL-R.G.B., and CPL-dark/light outperform CPL-random in convergence and reconstruction confidence, while approximating CPL-optimal effectiveness.The comparison covers five initialization methods and evaluates reconstruction quality and iteration count.
  • Random Seed Sensitivity: Different random seeds produce diverse convergence processes and reconstruction quality, with patterned initialization more efficient and stable than CPL-random.Label reconstruction can succeed even when private content reconstruction fails.

Attack Termination Condition:

Attack termination depends on both the maximum iteration count and the gradient-loss distance threshold. Experiments show that insufficient iterations cause failures, while a generalizable threshold remains dataset-dependent.

  • Termination Design: The termination condition combines a maximum attack-iteration limit with an L2-distance threshold for the gradient difference.Both factors determine when reconstruction optimization stops.
  • Maximum Iterations: When the maximum attack iterations are 10 or 20, CPL attacks fail on all or most datasets regardless of initialization.At 100 or 300 iterations, initialization quality matters more, and CPL-patterned exceeds CPL-random ASR across three datasets.
  • Maximum Iterations: A sufficiently large iteration limit improves the opportunity for stronger initialization methods to achieve higher attack success rates.CPL-random remains unstable across LFW, CIFAR10, and CIFAR100 when comparing 100 with 300 iterations.
  • L2-Distance Threshold: The gradient-difference L2-distance threshold is dataset-dependent, although 0.0001 performs well in generalization across four benchmark datasets.This threshold compares gradients from reconstructed and private local data.

Gradient Loss (Distance) Function:

The attack uses gradient-distance optimization to reconstruct private data, and the optimizer choice affects convergence behavior. Different optimization methods trade computational simplicity against the ability to escape slow convergence paths.

  • Gradient Loss: The CPL attack uses an L2 distance function as its gradient loss, while cosine similarity and entropy are alternative distance functions.Different loss functions may affect training efficiency.
  • Attack Optimization: First-order optimizers such as Adam and stochastic gradient descent are easier and less time-consuming to compute, whereas second-order methods can escape slow saddle-point convergence paths.The paper compares L-BFGS and Adam for CPL-patterned attacks on LFW.

3.2 Hyperparameter Configurations in Federated learning

Federated learning hyperparameters shape the effectiveness and cost of client privacy leakage attacks. Batch size, data resolution, and activation-function choices affect reconstruction success and attack iterations.

  • Training Data Resolution:: CPL-patterned attacks are more effective than CPL-random across 32×32, 64×64, and 128×128 LFW images, requiring 3 to 4× fewer attack iterations.CPL-random requires substantially more iterations to achieve high reconstruction performance.
  • Training Data Resolution:: The framework evaluates attacks on higher-resolution data, including a 512 × 512 Indiana University Chest X-Rays image.The higher-resolution attack is presented as an example of scaling CPL attacks beyond smaller image inputs.
  • Activation Function:: Activation-function choice is evaluated as a federated-learning hyperparameter because it is highly related to CPL attack performance.The comparison includes Sigmoid, Tanh, and LeakReLU, using attack iterations and success rate.

Activation Function:

Activation functions and model components affect whether gradients preserve information usable for reconstruction. ReLU and dropout can hinder CPL attacks through different gradient mechanisms, while pooling does not prevent them.

  • Activation Function:: ReLU can prevent full training-data reconstruction because zero gradients in its negative region remove gradient information needed by CPL attacks.The dying-ReLU behavior stops affected parameters from responding to error variations during optimization.
  • Activation Function:: Sigmoid and Tanh pass gradients between layers in an almost lossless way, unlike the gradient-blocking behavior described for ReLU.The passage characterizes both functions as differentiable and bijective.
  • Activation Function:: Dropout hampers CPL convergence by producing different gradients on each query, making the uploaded gradients elusive to the attack.This effect is described as preventing the reconstructed gradients from converging to the uploaded gradients.
  • Activation Function:: Pooling cannot prevent CPL attacks because pooling layers do not contain parameters.The stated distinction is that pooling lacks parameters whose gradients could disrupt the attack.
  • Evaluation:: The framework measures attack effects and cost using reconstruction metrics, attack success rates, and attack iterations.ASRc and ASRl denote success rates for reconstructed content and labels, respectively.

SSIM

SSIM evaluates image-reconstruction quality by comparing structural properties of reconstructed and ground-truth images. Values closer to 1 indicate better attack quality.

  • SSIM: SSIM compares image means, variances, and covariance, with stabilizing constants based on pixel-value dynamic range.The symbols are defined for reconstructed and reference images, while k1 and k2 default to 0.01 and 0.03.
  • SSIM: SSIM measures structural similarity between two images using a perception-based model that considers perceived image degradation.The paper uses SSIM for evaluating image datasets and presents it as an improvement over traditional image-similarity measures such as MSE.
  • SSIM: An SSIM value closer to 1 indicates better image-reconstruction quality in the CPL attack evaluation.SSIM is used specifically for image datasets, whereas MSE applies across formats including attributes and text.
  • Evaluation: Attack iterations measure how many reconstruction-learning updates are required to converge and satisfy the attack-success condition.Success is tied to the reconstructed and actual gradients reaching a sufficiently small L2 distance.

4 Experiments and Results

Experiments evaluate CPL attacks across image and attribute datasets, attack configurations, FL hyperparameters, communication compression, and preliminary mitigations. CPL generally achieves strong reconstruction, while attack effectiveness and cost vary with initialization, local training, compression, and noise.

  • Attack comparisons: CPL achieved the highest attack success rate and lowest attack iterations across four image datasets, with high SSIM and low MSE.Compared with deep gradient and gradient inverting attacks, CPL reconstructed content and labels more efficiently.
  • Attack comparisons: On Breast Cancer Wisconsin, CPL was most effective; gradient inverting reached ASRc of 78% and ASRl of 94%, versus 35% and 56% for deep gradient.On UCI Adult, all three attacks had good and similar performance.
  • Training-data inference comparisons: CPL produced the highest SSIM and lowest MSE in comparison with Aono and Melis attacks, indicating the strongest reconstruction quality.The comparison used log-scaled MSE because the magnitudes differed substantially.
  • Initialization: Geometrical initialization improved attack success and convergence over random initialization, while optimal initialization served as a theoretical reference requiring background data information.CIFAR100 required more time and iterations than CIFAR10 and LFW.
  • Local-training hyperparameters: On LFW, CPL attack success decreased to 96%, 89%, 76%, and 13% as batch size increased to 2, 4, 8, and 16.Increasing local training iterations also reduced success to 97%, 85%, and 39% for 5, 7, and 9 iterations.
  • Communication-efficient protocols: Under compression, CPL retained high-confidence reconstructions at 40% for MNIST and CIFAR10 and at 90% for CIFAR100 and LFW.Higher compression reduced successful-attack iterations, while more training labels made attacks more severe.
  • Mitigation strategies: Gaussian or Laplace noise largely mitigated leakage on CIFAR100 and LFW, but incurred an accuracy cost.Controlled local training iterations were also evaluated as a preliminary mitigation strategy.

5 Related work

Related work distinguishes training-phase privacy attacks from prediction-phase attacks. The paper situates CPL among gradient leakage, proportionality-based reconstruction, membership, model inversion, and related inference attacks.

  • Training-phase privacy attacks: Gradient leakage attacks reconstruct private training data during FL training from local parameter updates shared with the server.Clients may be compromised enough for an adversary to intercept updates without directly accessing private training data.
  • Training-phase privacy attacks: Aono et al. partially recover private data from the proportionality between training data and gradient updates in multilayer perceptrons.This attack differs from CPL's broader gradient leakage setting.
  • Prediction-phase privacy attacks: Prediction-phase privacy exploits include model inversion, membership inference, property inference, and GAN-based reconstruction attacks.These attacks use model predictions or trained-model properties rather than only the local update mechanism described for CPL.

6 Conclusion

The conclusion presents a principled framework for comparing client privacy leakage attacks and reports that shared parameter updates can enable reconstruction of private local training data. It emphasizes systematic evaluation for understanding threats and assessing mitigations.

  • Framework contribution: The framework evaluates and compares different forms of client privacy leakage attacks.It analyzes attack configurations and FL hyperparameter settings affecting leakage.
  • Privacy leakage: Adversaries can reconstruct private local training data by analyzing shared local gradient or weight-update vectors.The conclusion frames this result as applying to the shared parameter update itself.
  • Implications: Experiments on four benchmark datasets support systematic evaluation of leakage threats and mitigation strategies in federated learning.The stated purpose is to deepen understanding of attack forms and mitigation evaluation.

7 Appendices

The appendices state convexity and Lipschitz-smoothness assumptions, introduce supporting lemmas, and complete a proof through recursive inequalities, induction, and telescopic cancellation.

  • Assumptions: The appendix assumes convexity for f(x) and defines Lipschitz smoothness using a Lipschitz constant L.These assumptions support the subsequent lemmas and proof steps.
  • Lemmas: The appendix presents lemmas for differentiable convex functions, Lipschitz-smooth functions, and co-coercivity.These lemmas provide intermediate inequalities for the proof.
  • Proof result: The final proof steps establish the bound Tβ ≤ 1/(f(xT) − f(x∗)) − 1/(f(x0) − f(x∗)) ≤ 1/(f(xT) − f(x∗)).The appendix then states that the proof is complete.
Loading 2004.10397v2…