PaperlayerView exact revision on arXiv

Source-linked AI summary

Do Gradient Inversion Attacks Make Federated Learning Unsafe?

Ali Hatamizadeh, Hongxu Yin, Pavlo Molchanov, Andriy Myronenko, Wenqi Li, Prerna Dogra, Andrew Feng, Mona G. Flores, Jan Kautz, Daguang Xu, Holger R. Roth

arXiv:2202.06924v3cs.LGcs.CRcs.CVcs.DC

TL;DR

Shared model gradients have raised concerns that FL may expose training data, particularly where prior attack methods assume fixed BN statistics. This paper develops gradient inversion that handles updated BN statistics and new ways to measure leakage; it concludes that image recovery is often very difficult in common FL scenarios.

  • Problem

    Prior gradient-inversion work often assumes fixed BN statistics, although statistics can vary across batches in practical FL, raising concerns that shared gradients may reveal training data.

    Source: §I · pp. 1–2

  • Method

    The paper extends gradient inversion with an FL-specific BN loss that estimates running BN statistics when clients update them during training.

    Source: §I · p. 3, B. Limitations of Previous Attack… · p. 5

  • Results

    In most common FL scenarios, recovering training images is very difficult, and prior efforts likely exaggerated gradient-inversion risks.

    Source: §IV · p. 12

  • Takeaways & Limitations

    The work provides a basis for reproducible data-leakage measurement and for studying privacy-accuracy tradeoffs using quantifiable metrics.

    Source: §IV · p. 12

    The study covers modern CNNs for image classification and clients using SGD, not architectures such as segmentation or NLP models or optimizers such as FedOpt.

    Source: §IV · p. 12

Abstract

from arXiv · show

Federated learning (FL) allows the collaborative training of AI models without needing to share raw data. This capability makes it especially interesting for healthcare applications where patient and data privacy is of utmost concern. However, recent works on the inversion of deep neural networks from model gradients raised concerns about the security of FL in preventing the leakage of training data. In this work, we show that these attacks presented in the literature are impractical in FL use-cases where the clients' training involves updating the Batch Normalization (BN) statistics and provide a new baseline attack that works for such scenarios. Furthermore, we present new ways to measure and visualize potential data leakage in FL. Our work is a step towards establishing reproducible methods of measuring data leakage in FL and could help determine the optimal tradeoffs between privacy-preserving techniques, such as differential privacy, and model accuracy based on quantifiable metrics. Code is available at https://nvidia.github.io/NVFlare/research/quantifying-data-leakage.

I. INTRODUCTION

Federated learning keeps raw data at client sites, but shared model updates may still expose training data to gradient inversion. This work studies server-side attacks in horizontal FL and addresses the gap between prior fixed-BN assumptions and training with updated BN statistics.

I. INTRODUCTION

  • FL supports collaborative model training without sharing raw data, making it attractive for healthcare applications where patient privacy matters.Healthcare institutions can learn across diverse patient populations and acquisition settings.Source: §I · p. 1
  • The work focuses on server-side gradient inversion in horizontal FL, where clients send locally computed model updates to a central server.The server aggregates updates and returns an updated global model; the paper assumes an honest-but-curious server with access to training configurations and updates.Source: §I · pp. 1–2
  • The study proposes ways to measure and visualize potential data leakage and evaluates attacks across client data amounts, batch sizes, and differential-privacy settings.It uses the findings to recommend more secure and privacy-preserving FL settings.Source: §I · p. 2
  • Earlier attacks match synthesized and real-data gradients; later methods improve image reconstruction, but some prior FL analysis used fixed BN statistics.DLG was limited to low-resolution images and computationally expensive second-order optimization; Geiping et al. used cosine similarity and first-order optimization, while Yin et al. matched fixed BN running statistics.Source: §I · p. 2
  • Prior attacks commonly assume fixed Batch Normalization statistics, but those statistics vary across batches in practical FL and can undermine inversion.The paper attributes accumulated errors to BN-statistic shifts across batches.Source: §I · p. 2
  • The paper introduces an FL-specific BN loss for gradient inversion across multiple batches while clients update BN statistics.The contribution traces BN's momentum mechanism and estimates running statistics to match gradients.Source: §I · pp. 1–3

A. Inversion Attacks

Gradient inversion optimizes candidate inputs to reproduce a client’s model updates. The proposed attack accounts for Batch Normalization statistics that change during training, unlike prior attacks that assumed fixed statistics.

A. Inversion Attacks

  • Gradient inversion optimizes candidate inputs to match observed client updates, using the global model state to compute matching gradients.The attack may initialize the input randomly or from a prior.Source: A. Inversion Attacks · p. 3, Figure 2 · p. 3

B. Practical Gradient Inversion Attack

  • Training-mode BN statistics make prior fixed-statistic attacks poorly suited to practical FL, especially when clients perform multiple local iterations.The proposed attack uses updates generated while BN statistics are updated during training.Source: B. Practical Gradient Inversion Attack · p. 3
  • The attack simulates client training by matching intercepted gradients and BN statistics, initializing its attack network from global weights and client updates.Its BN loss matches statistics from trainable inputs to intercepted client statistics across layers.Source: B. Practical Gradient Inversion Attack · pp. 3–4
  • More local iterations make running-statistic estimates less accurate, making it harder to match gradients.The paper illustrates effects of training-image count, batch size, and local iterations in Figures 4 and 6.Source: B. Practical Gradient Inversion Attack · p. 4
  • Domain-specific image priors initialize candidate images, while trainable labels and image-prior losses support reconstruction.The image-prior loss includes total variation and an l2 norm; labels are optimized with a classification loss such as cross-entropy.Source: B. Practical Gradient Inversion Attack · p. 4

C. Differential Privacy (DP)

The paper considers differential privacy (DP) as a way to reduce leakage from shared updates and also evaluates DP-SGD during local training. It notes the privacy–accuracy tradeoff and the scope of the evaluated noise mechanism.

C. Differential Privacy (DP)

  • DP can offer theoretical guarantees against individual privacy loss, but tuning it to retain acceptable model accuracy is difficult.The paper frames added update noise as reducing inversion success at the cost of model performance.Source: C. Differential Privacy (DP) · p. 4
  • Local DP adds noise to gradients shared between FL participants, protecting outgoing model updates independently of the local optimizer.The paper contrasts this with DP-SGD, which adds noise at each local training iteration.Source: C. Differential Privacy (DP) · p. 4
  • The study uses Gaussian noise on model updates to explore DP and establish an upper bound for successful inversion attacks in FL.Noise is scaled using a chosen percentile of absolute update values; experiments use the 95th percentile to reduce outlier effects.Source: C. Differential Privacy (DP) · p. 4
  • The authors also evaluate leakage when clients use DP-SGD locally and identify alternative DP or gradient-perturbation methods as future work.DP-SGD clips local gradients by a maximum l2 norm and adds noise relative to the clipping norm.Source: C. Differential Privacy (DP) · p. 4

D. Image Identifiability Precision

Image identifiability precision (IIP) measures whether reconstructed images correspond to exact images in a training pool. The paper modifies the metric to account for all clients’ training and validation images in FL.

D. Image Identifiability Precision

  • IIP uses pretrained-network feature embeddings and nearest-neighbor matching to identify exact matches between reconstructions and training images.The original formulation computes the score from a fixed number of randomly selected images.Source: D. Image Identifiability Precision · p. 5
  • The FL-adjusted IIP evaluates all clients’ training and validation images and counts a match only when the closest training image belongs to the attacked client.This allows a diverse population of training images to contribute to leakage quantification.Source: D. Image Identifiability Precision · p. 5

III. RESULTS

The study simulates cross-silo, horizontally partitioned FL for chest X-ray classification with eight clients. It varies client training-set sizes and batch sizes, while noting that the evaluated clients represent higher-risk settings than typical deployments.

III. RESULTS

  • Eight clients train ImageNet-pretrained ResNet-18 models for chest X-ray classification in a 100-round cross-silo FL simulation.Each client trains for one epoch per round using SGD without momentum and a step-wise learning-rate decay every 40 rounds.Source: A. FL Scenario · p. 5, Table II · p. 5
  • The experiments use chest X-ray and color fundus datasets, with dataset-specific mean images as input priors.The training and validation case counts are the same across datasets A and B.Source: A. FL Scenario · p. 5, Table II · p. 5
  • The evaluated clients are considered higher-risk than real-world FL clients with larger training sets and batch sizes.The study intentionally uses relatively small batch sizes to examine clients with varying amounts of data and local iterations.Source: A. FL Scenario · p. 5

B. Limitations of Previous Attack Scenarios

The attack must simulate clients’ updated Batch Normalization statistics and start from the current global checkpoint; reconstruction quality also varies with training conditions. Later FL rounds improve image fidelity, while more local iterations and larger training sets can make inversion harder.

B. Limitations of Previous Attack Scenarios

  • The attack extends GradInversion with a BN loss that updates running means and variances during optimization, enabling inversion when clients update BN statistics.Assuming fixed BN statistics accumulates estimation errors that hinder reconstruction; the attack also needs the global weights used to initialize the client for that FL round.Source: B. Limitations of Previous Attack… · p. 5
  • Both the current global checkpoint and accurate BN-statistic accounting are necessary for successful inversion; without either, the attack fails.The figure also shows that a correct CXR prior and grayscale constraint produce the highest qualitative reconstruction among the displayed conditions.Source: B. Limitations of Previous Attack… · p. 5, Figure 3 · p. 6

C. Measuring Data Leakage by Inverting Gradients

  • Later FL rounds, such as round 90 of 100, yield reconstructions with improved image fidelity and anatomical detail.The paper attributes this observation as possibly reflecting more effective learned features or images causing higher losses later in training.Source: C. Measuring Data Leakage by Inverting… · p. 6
  • More local iterations and larger training sets make reconstructions harder because BN-statistic estimates accumulate errors across different training images.The attack estimates BN statistics approximately; increasing local iterations makes those estimates less accurate.Source: C. Measuring Data Leakage by Inverting… · p. 6
  • SSIM measures pixel-wise similarity between reconstructed and original images to quantify data leakage.The paper describes it as more intuitive and interpretable than RMSE or PSNR.Source: C. Measuring Data Leakage by Inverting… · p. 6

D. Mitigating Inversion Through Differential Privacy

The paper tests Gaussian noise added to client updates as a defense against server-side gradient inversion. Increasing noise reduces reconstruction similarity, with a stated prior-based SSIM threshold illustrated for one client and round.

D. Mitigating Inversion Through Differential Privacy

  • As Gaussian noise on updates increases, reconstruction SSIM decreases and reconstructions move closer to the attack’s prior.The prior’s SSIM with the original image is 0.37, treated as a lower bound for the attack’s metric.Source: D. Mitigating Inversion Through… · p. 6
  • For high-risk client 9 at FL round 90, noise with σ=10^-2 yields reconstructions with SSIM at or below the prior-to-original SSIM, an example of the paper’s proposed “safe” criterion.The criterion compares reconstruction-to-original similarity with the prior-to-original similarity.Source: D. Mitigating Inversion Through… · pp. 6–7

E. Quantification of Data Leakage

The paper introduces RDLV to compare leakage across clients relative to the attack prior. Its results distinguish the high-risk single-image client from clients with larger training sets and show a privacy–accuracy tradeoff under DP-SGD.

E. Quantification of Data Leakage

  • RDLV measures the relative change in SSIM between a training image and its reconstruction against the prior, enabling a more comparable leakage metric across clients.The comparison addresses variation in reference images that makes absolute similarity values across clients difficult to compare.Source: E. Quantification of Data Leakage · p. 7
  • Only high-risk client 9, which sends updates after one iteration on one image, shows positive RDLV up to around σ0≥25 for most FL rounds; clients with multiple images do not leak within the confidence interval.Positive RDLV means reconstructions are more similar to original images than to the prior; global model accuracy drops as added noise increases.Source: E. Quantification of Data Leakage · p. 7
  • DP-SGD reduces leakage for high-risk client 9 to negative RDLV, but harms global model performance more than the simple Gaussian-noise protocol.The authors note that client-specific privacy-parameter tuning might be needed to optimize global performance under DP-SGD.Source: E. Quantification of Data Leakage · p. 8

F. Deep Embedding-Based Visualizations

The visualization uses ImageNet-pretrained ResNet-18 embeddings and t-SNE to show training images and reconstructions. With higher Gaussian noise, the two clusters drift farther apart as similarity decreases.

F. Deep Embedding-Based Visualizations

  • With higher Gaussian noise, t-SNE clusters of original images and reconstructions drift farther apart, alongside a significant decrease in similarity.The embeddings are 512-dimensional features extracted with an ImageNet-pretrained ResNet-18.Source: F. Deep Embedding-Based Visualizations · p. 8

G. Membership Inference Attack Data Leakage

The adjusted Image Identifiability Precision (IIP) measures identifiable leakage by counting exact matches between reconstructed and original training images. Client 9 has the highest score, and both IIP and matched-image cosine similarity decrease as Gaussian noise increases.

G. Membership Inference Attack Data Leakage

IV. DISCUSSION

The discussion finds that leakage risk varies with client training conditions and that differential privacy reduces inversion risk while degrading model performance at higher noise levels. It also outlines attack mitigations, measurement caveats, and study limitations.

IV. DISCUSSION

  • Clients sending updates based on few training images face higher leakage risk; larger training sets, more local iterations, and larger batches make identifiable reconstruction harder.Accumulated errors in estimating BN statistics make the attack harder when clients iterate over several images.Source: §IV · pp. 8–10, Figure 6 · p. 9
  • Later-round reconstructions capture finer anatomical detail, but none were identifiable as images in the original training set.The authors attribute the improved image quality to better feature learning by the global model.Source: §IV · p. 10
  • Withholding the global-model state, architecture, or BN statistics can hinder server-side inversion; keeping client-specific BN statistics trades away a common global model.The text also describes secure aggregation as a way to restrict access to model state.Source: §IV · p. 10
  • Higher Gaussian noise reduces gradient-inversion risk, but degrades model performance; degradation is more pronounced with DP-SGD.The discussion compares a simple calibrated-noise protocol with DP-SGD.Source: Figure 8 · p. 10, Figure 9 · p. 10, §IV · p. 12
  • Secure aggregation, homomorphic encryption, and targeted encryption are presented as additional ways to protect exchanged updates.The text says targeted encryption of vulnerable model parts might provide security with minimal computational overhead.Source: §IV · p. 12
  • Some reconstructions have relatively high SSIM without close image similarity or high IIP, so SSIM alone can suggest leakage where qualitative review does not.The authors suggest deep-feature similarity metrics may provide more plausible comparisons in such cases.Source: Figure 7 · p. 9, §IV · p. 12
  • The study covers ResNet-18 image classification with SGD; segmentation, NLP architectures, other optimizers, and generative attacks remain outside its scope.The authors identify these areas for further study.Source: §IV · p. 12
  • The proposed leakage measures and visualizations are intended to support reproducible assessment and study of privacy–accuracy tradeoffs.The discussion frames this as a foundation for quantifying leakage in FL.Source: §IV · p. 12
Loading 2202.06924v3…