PaperlayerView exact revision on arXiv

Source-linked AI summary

Evaluating Gradient Inversion Attacks and Defenses in Federated Learning

Yangsibo Huang, Samyak Gupta, Zhao Song, Kai Li, Sanjeev Arora

arXiv:2112.00059v1cs.CRcs.LG

TL;DR

Gradient inversion attacks threaten federated-learning privacy by recovering client data from shared gradients, but prior evidence did not systematically characterize risks and defenses. This paper reevaluates attacks under weaker assumptions, measures defense trade-offs and recovery costs, and finds that suitable combined defenses can substantially reduce attack effectiveness with limited utility loss.

  • Problem

    Gradient inversion attacks can recover clients’ private data from shared gradients, while systematic evidence on attack risk, defense security, and their utility, overhead, and privacy trade-offs was lacking.

  • Method

    The paper reevaluates a state-of-the-art attack under unknown BatchNorm statistics and private labels, systematically tests defenses and combinations, and estimates end-to-end single-image recovery costs.

  • Results

    Relaxing strong attack assumptions weakens reconstruction, while InstaHide without public-data mixing combined with gradient pruning defends against the state-of-the-art attack and makes recovery on a 500,000-image client dataset enormous in estimated time.

  • Takeaways & Limitations

    Federated learning can use configurations and combined defenses that improve security against gradient inversion attacks while preserving a favorable privacy–utility trade-off.

  • Takeaways & Limitations

    The evaluation uses the strongest attack and relatively small batch sizes, so invalidating assumptions or increasing batch size may allow weaker defenses with better accuracy.

Abstract

from arXiv · show

Gradient inversion attack (or input recovery from gradient) is an emerging threat to the security and privacy preservation of Federated learning, whereby malicious eavesdroppers or participants in the protocol can recover (partially) the clients' private data. This paper evaluates existing attacks and defenses. We find that some attacks make strong assumptions about the setup. Relaxing such assumptions can substantially weaken these attacks. We then evaluate the benefits of three proposed defense mechanisms against gradient inversion attacks. We show the trade-offs of privacy leakage and data utility of these defense methods, and find that combining them in an appropriate manner makes the attack less effective, even under the original strong assumptions. We also estimate the computation cost of end-to-end recovery of a single image under each evaluated defense. Our findings suggest that the state-of-the-art attacks can currently be defended against with minor data utility loss, as summarized in a list of potential strategies. Our code is available at: https://github.com/Princeton-SysML/GradAttack.

1 Introduction

Federated learning keeps client data local but can still expose private information through shared gradients. This paper systematically evaluates gradient inversion risks, defenses, utility, privacy leakage, and computation costs.

  • Threat: Gradient inversion attacks can reconstruct clients’ private data from gradients observed by eavesdroppers or malicious participants.These participants may include honest-but-curious servers or clients.
  • Scope: Secure aggregation and homomorphic encryption are excluded because their special setups and overhead are outside the evaluation.
  • Research gap: Prior work had not systematically evaluated attack risk, defense security, and trade-offs among test accuracy, computation overhead, and privacy.
  • Attack assumptions: Relaxing assumptions about BatchNorm statistics and private labels significantly weakens a state-of-the-art attack, limiting it to low-resolution images.
  • Evaluation: The paper evaluates proposed defenses for privacy leakage, data utility, computation overhead, and their performance when combined.It also estimates the cost of end-to-end recovery of one image under each defense.

2 Gradient Inversion Attacks

Gradient inversion attacks formulate input recovery as optimization that matches recovered and observed gradients while applying image priors. Success has progressed from small low-resolution batches to larger batches and high-resolution images, with later methods adding normalization-based priors and cross-trial consistency.

  • Attack formulation: Gradient inversion recovers an approximation x of a private batch x* by optimizing against gradients computed from private data.The private batch has size b and image size d.
  • Attack formulation: The objective combines gradient matching Lgrad with an auxiliary regularizer Raux(x) based on image priors.
  • Prior attacks: Accurate pixel-level reconstruction was practical up to batch size 8 for low-resolution CIFAR images, but did not scale to high-resolution images or larger ReLU models.The original formulation used ℓ2 gradient distance without regularization.
  • Improved attacks: Careful choices of cosine gradient distance and total variation regularization enabled recovery of a single high-resolution ImageNet image and low-resolution batches up to 100.
  • Recent variants: Later work added BatchNorm-statistics priors and cross-trial consistency, while a recursive closed-form approach recovered only low-resolution images at batch size 1.

3 Strong Assumptions Made by SOTA Attacks

The evaluated state-of-the-art attack relies on knowing private BatchNorm statistics and labels. Relaxing these assumptions weakens recovery, especially for high-resolution images.

  • Strong assumptions: The analysis identifies knowledge of private BatchNorm statistics and private labels as two strong assumptions in gradient inversion attacks.Geiping et al.'s attack makes both assumptions.
  • Assumption 1: Knowing BatchNorm statistics: Knowing private-batch BatchNorm statistics lets the attacker normalize recovered batches using the same statistics as the client.These statistics are not necessary to share in Federated Learning.
  • Relaxation 1: Not knowing BatchNorm statistics: Without private BatchNorm statistics, BNinfer reconstructs low-resolution batches better than BNproxy but misses details relative to BNexact.For a single high-resolution image, BNinfer fails to produce a recognizable reconstruction.
  • Relaxation 2: Not knowing private labels: Without private labels, reconstruction quality drops for a batch of 16 images, while large CIFAR-10 batches often contain many images from the same class.The attack is re-designed under each relaxed threat model using only the attacker's available knowledge.
  • Assumption 2: Knowing private labels: Private labels can be inferred from gradients, but when multiple batch images share a label, attacks cannot determine their count or image assignments.Nonuniform labels in larger CIFAR-10 batches therefore make label inference harder.

4 Defenses Against the Gradient Inversion Attack

The paper reviews cryptographic, gradient-perturbation, and data-transformation defenses against gradient inversion. Their costs and utility trade-offs motivate evaluating lightweight mechanisms and combinations in the Federated Learning setting.

  • Encrypt gradients: Cryptographic defenses use secure aggregation or homomorphic encryption to protect gradients, but require special setup and can be costly to implement.The paper does not study these approaches because of their special setups and overhead.
  • Encrypt gradients: Secure aggregation can still permit an honest-but-curious server to attack the summed gradient vector.An honest-but-curious client may also attack the server-returned model despite homomorphic encryption.
  • Perturbing gradients: Gradient pruning sets small-magnitude gradients to zero, but the safety of pruning more than 70% was demonstrated only against weaker attacks.The suggested prune ratio may not remain safe against the state-of-the-art attack.
  • Perturbing gradients: Adding Gaussian or Laplacian gradient noise can defend against attacks, but earlier results required accuracy losses exceeding 30% on CIFAR-10.The paper leaves comparison with DPSGD to future work because common implementations use pre-training and fine-tuning.
  • Data transformation: InstaHide's defense and attack had not been evaluated in Federated Learning, where attackers observe gradients rather than encoded images.This gap motivates the paper's systematic evaluation.

5 Evaluation of defenses

The evaluation measures how defenses trade off privacy leakage, test accuracy, computation, and attack effectiveness under strong attacker assumptions. Gradient pruning and MixUp alone have important limitations, while combining gradient pruning with Intra-InstaHide substantially weakens reconstruction with a smaller accuracy loss.

  • Experimental setup: The experiments evaluate defenses on CIFAR-10 with ResNet-18, using attack batch sizes 1, 16, and 32, while granting knowledge of private BatchNorm statistics and labels.The attack uses 50 CIFAR-10 images and, for MixUp and Intra-InstaHide, a decode step requiring eavesdropping across T = 20 training epochs.
  • Gradient pruning: As pruning ratio increases, reconstructions gain artifacts but remain recognizable at p = 0.9; defending the strongest attack may require p ≥0.999.A pruning ratio this high introduces around 10% accuracy loss.
  • MixUp: MixUp with k = 4 causes only ~2% test-accuracy impact but does not sufficiently defend the attack, while increasing k to 6 only slightly reduces leakage.The reconstructions remain highly recognizable, making MixUp alone potentially impractical against the state-of-the-art attack.
  • Intra-InstaHide: Intra-InstaHide with k = 4 loses an extra ~2% accuracy compared with MixUp and produces artifacts and color shifts at batch size 32.Private-image structures remain vaguely identifiable, so Intra-InstaHide alone does not fully defend the attack.
  • Combined defenses: Combining Intra-InstaHide (k = 4) with gradient pruning (p = 0.9) makes batch-size-32 reconstructions almost unrecognizable with ~7% accuracy loss.The combination yields a higher LPIPS score than gradient pruning at p = 0.999 while losing less accuracy.
  • Computation cost: End-to-end recovery under InstaHide takes > 3,000× longer than no defense or GradPrune for N = 5,000, and becomes enormous for N = 500,000.The estimate assumes 32 × 32 × 3 images, a batch size of 128, T = 50, and 10,000 optimization iterations on one RTX 2080 Ti.

6 Conclusions

The paper finds that relaxing attackers’ knowledge of BatchNorm statistics and private labels significantly weakens gradient inversion attacks, while combined defenses can improve privacy with limited utility loss.

  • Relaxing assumptions about known BatchNorm statistics and private labels significantly weakens state-of-the-art gradient inversion attacks.The attack may then work only for low-resolution images.
  • The study evaluates additional defenses, utility-privacy trade-offs, recovery computation costs, and combined mechanisms, while leaving high-resolution defenses and some attacks for future work.
  • Using BatchNorm layers without sharing private-batch statistics weakens the attack and is feasible in federated learning.
  • Larger batch sizes hinder label guessing and image recovery; batches smaller than 32 are not safe against the strongest attack, even with weak defenses.
  • For batch size 32, combining InstaHide (k = 4) with gradient pruning (p = 0.9) makes reconstructions almost unrecognizable at a cost of ~7% accuracy loss.

A Experimental details and more results

The appendix documents the experimental testbed and figure-specific image selections used to illustrate recovery quality and label distributions across CIFAR-subset and ImageNet-subset evaluations.

  • Experiments used Nvidia RTX 2080 Ti and V100 GPUs, with Table 4 summarizing the images used in each main-paper figure or table.
  • Figure 1a uses the first sampled batch of 16 CIFAR-subset images to demonstrate low-resolution recovery when BatchNorm statistics are unknown.
  • Figure 1b uses the best-reconstructed ImageNet-subset image to demonstrate high-resolution recovery when BatchNorm statistics are unknown.
  • Figure 2a evaluates class-label percentages per batch across the entire CIFAR10 dataset using a random seed.
  • Figure 2b uses the first sampled batch of 16 images to demonstrate recovery when labels are not assumed known.

A.1 Hyper-parameters

The appendix reports training and attack hyperparameters, including regularization settings varied in Figures 4 and 5 to identify the best reconstructions.

  • Training: All experiments train ResNet-18 for 200 epochs with SGD momentum 0.9, default learning rate 0.1, and decay by 0.1 every 50 epochs.Gradient pruning with p = 0.99 and p = 0.999 uses initial learning rate 0.02.
  • The attack: The attack evaluates performance under different αTV and αBN values, corresponding to total-variation and BatchNorm regularization coefficients.
  • The attack: αTV = 1e-2 gives the best reconstruction for a single CIFAR-10 image in the BNexact setting.
  • The attack: αTV = 1e-3 gives the best reconstruction for a batch of 16 CIFAR-10 images in the BNinfer setting.

A.2 Details and more results for Section 3

Additional experiments examine ImageNet recovery and a more realistic attack setting where BatchNorm statistics are unknown but private labels are known.

  • Attacking a single ImageNet image: The ImageNet attack runs for 24,000 iterations with Adam, αTV = 0.1, αBN = 0.001, and five reruns, reporting the best LPIPS result.
  • Qualitative and quantitative results for a more realistic attack: With batch size 32 and Intra-InstaHide (k = 4), the realistic attack is no longer effective, requiring only around 3% accuracy loss.The realistic setting assumes known private labels but unknown BatchNorm statistics.
  • Qualitative and quantitative results for a more realistic attack: Table 5 reports LPIPS on 50 CIFAR-10 images to compare utility-security trade-offs when labels are known but BatchNorm statistics are unknown.Lower LPIPS values indicate greater privacy leakage, and the least-leakage defense is marked in green.

A.3 More results for the strongest attack

Additional evaluations examine defenses against the strongest attack across MNIST, defense configurations, and batch sizes. Gradient pruning alone may require extreme pruning, while combining it with InstaHide makes reconstructions almost unrecognizable.

  • Experimental setup: MNIST evaluations use a simple 6-layer ConvNet without BatchNorm against the strongest attack with private labels known.The evaluation varies gradient pruning, MixUp, Intra-InstaHide, and their combinations.
  • Gradient pruning: p ≥0.9999 pruning may be required for gradient pruning to defend the strongest attack on MNIST.This result concerns the simple 6-layer ConvNet setting.
  • MixUp: MixUp with k = 4 or k = 6 is insufficient to defend the strongest attack, while MixUp plus p = 0.99 pruning leaves digits highly recognizable.The combined defense improves protection but does not make the reconstruction unrecognizable.
  • Intra-InstaHide: Intra-InstaHide alone performs somewhat better than MixUp, and combining it with p = 0.99 pruning makes reconstruction almost unrecognizable.The evaluation considers k = 4 and k = 6 for Intra-InstaHide.

A.4 More results for encoding-based defenses

The additional experiments visualize reconstructed batches and datasets under MixUp and Intra-InstaHide across batch sizes. Gradient pruning is interpreted as gradient compression that removes information available to the attacker.

  • Visualization results: Figures 9–12 visualize original and reconstructed batches or datasets under MixUp and Intra-InstaHide for batch sizes 1, 16, and 32.Figure 9 includes absolute images for Intra-InstaHide, while Figures 10–12 show reconstructed datasets.
  • Visualization results: Intra-InstaHide makes pixel-wise matching harder when comparing original and reconstructed batches.The visualization includes both original and absolute images for the Intra-InstaHide defense.
  • Defense mechanism: Gradient pruning compresses the gradient vector by setting selected gradient information unavailable to the attacker, making inversion harder.It is described as a non-oblivious application of sketching techniques.

B.2 MixUp and InstaHide

MixUp and InstaHide can be analyzed as batched vector-sum problems because first-layer gradients are linear combinations of input images. The analysis extends this connection from small batches to deeper networks and incorporates InstaHide’s sign flipping.

  • MixUp and InstaHide: Mixing k images is essentially similar to using batch size k for networks whose first layer is fully connected.This provides the stated intuition for why MixUp and InstaHide reduce attack effectiveness.
  • MixUp and InstaHide: InstaHide’s random sign flipping can weaken the effectiveness of a total-variation image prior applied to absolute values.Small adjacent absolute-value differences do not guarantee small differences between signed neighboring pixels.
  • Small-batch gradients: For ReLU networks, each row of the first-layer weight gradient is a linear combination of the images in a small batch.The batch contains b images x_1, ..., x_b in R^d.
  • Small-batch gradients: Recovering images from observed gradients is a batched vector-sum problem, making gradient attacks a variation of MixUp/InstaHide attacks.The abstraction can be generalized to InstaHide by observing absolute values and to random coefficients.
  • Deeper networks: The analysis defines multilayer networks and computes gradients for later layers, while treating the first-layer gradient separately.The first-layer gradient cannot be written in the same general form as gradients for layers ℓ ≥ 2.
Loading 2112.00059v1…