PaperlayerView exact revision on arXiv

Source-linked AI summary

Inverting Gradients -- How easy is it to break privacy in federated learning?

Jonas Geiping, Hartmut Bauermeister, Hannah Dröge, Michael Moeller

arXiv:2003.14053v2cs.CVcs.CRcs.LG

TL;DR

Federated learning shares gradients to train models while keeping user data on-device, but the privacy protection offered by those gradients is uncertain. The paper develops analytical and optimization-based inversion attacks and finds that realistic deep networks remain vulnerable, including under gradient averaging. It concludes that stronger privacy protections are needed, while noting accuracy and implementation costs for available defenses.

  • Problem

    Prior gradient-recovery attacks succeeded mainly in contrived or limited settings, leaving the privacy of realistic federated computer-vision systems insufficiently established.

  • Method

    The paper combines analytical reconstruction for fully connected layers with optimization-based attacks using improved loss and optimization strategies, and evaluates architectures and distributed settings.

  • Results

    The attacks reconstruct inputs from gradients in realistic trained deep networks, and reconstruction remains possible from averaged gradients over multiple images or iterations.

  • Takeaways & Limitations

    Gradient sharing and multi-image federated averaging do not by themselves guarantee privacy for computer-vision data.

  • Takeaways & Limitations

    The experiments target modern computer-vision image-classification architectures, and the threat model assumes an honest-but-curious server that cannot alter the training algorithm or model parameters.

Abstract

from arXiv · show

The idea of federated learning is to collaboratively train a neural network on a server. Each user receives the current weights of the network and in turns sends parameter updates (gradients) based on local data. This protocol has been designed not only to train neural networks data-efficiently, but also to provide privacy benefits for users, as their input data remains on device and only parameter gradients are shared. But how secure is sharing parameter gradients? Previous attacks have provided a false sense of security, by succeeding only in contrived settings - even for a single image. However, by exploiting a magnitude-invariant loss along with optimization strategies based on adversarial attacks, we show that is is actually possible to faithfully reconstruct images at high resolution from the knowledge of their parameter gradients, and demonstrate that such a break of privacy is possible even for trained deep networks. We analyze the effects of architecture as well as parameters on the difficulty of reconstructing an input image and prove that any input to a fully connected layer can be reconstructed analytically independent of the remaining architecture. Finally we discuss settings encountered in practice and show that even averaging gradients over several iterations or several images does not protect the user's privacy in federated learning applications in computer vision.

1 Introduction

Federated learning shares parameter gradients rather than user data, but those gradients can still reveal private inputs. The paper shows reconstruction remains possible across realistic networks and even when gradients are averaged.

  • Privacy motivation: Federated learning shares gradients instead of original user data while collaboratively training a neural network.Users retain training examples locally and send parameter updates to the server.
  • Privacy risk: Gradient updates still carry significant information about supposedly private inputs, breaking the intended privacy of federated learning.The paper illustrates this failure with reconstructions from trained ImageNet models.
  • Main findings: The attack reconstructs input data from gradients in realistic deep and non-smooth architectures with both trained and untrained parameters.The authors present this as a central empirical contribution.
  • Main findings: Deep networks provide little defense-in-depth: with the right attack, they are as vulnerable as shallow networks.The finding concerns the difficulty of reconstructing inputs from gradient information.
  • Main findings: Any input to a fully connected layer can be reconstructed analytically, independent of the remaining network architecture.The paper also examines the practical implications of this result for distributed learning.
  • Practical settings: Reconstruction of multiple images from averaged gradients remains possible across multiple epochs, local mini-batches, and averages of up to 100 images.Thus, local gradient averaging does not necessarily prevent recovery of private visual data.

2 Related Work

Earlier gradient-inversion research established recovery in limited settings, while related work also studied attribute leakage, model inversion, and inversion from intermediate representations. The paper positions its approach against these narrower or technically challenging precedents.

  • Prior gradient recovery: Earlier recovery attacks focused mainly on shallow networks with limited practical relevance.Prior results covered single neurons, linear layers, and a four-layer CNN with a large fully connected layer.
  • Prior gradient recovery: Prior convolutional attacks constructed an input representation and improved it with a GAN, while later work extended the setting.These approaches addressed image recovery but remained distinct from the paper’s broader realistic-architecture focus.
  • Optimization-based recovery: Earlier methods optimized Euclidean gradient matching with L-BFGS, requiring higher-order derivatives that are challenging for ReLU networks.Differentiating the gradient with respect to the input requires second-order derivatives, while L-BFGS constructs a third-order approximation.
  • Attribute leakage: Related attacks recover input attributes from local updates, including attributes unrelated to the network’s task.This is a related but easier problem than reconstructing complete input images.
  • Other inversion settings: Model inversion recovers training images from learned parameters, whereas representation inversion reconstructs plausible inputs from intermediate neural-network outputs.Model inversion is generally challenging for deeper networks without additional information.

3 Theoretical Analysis: Recovering Images from their Gradients

The analysis shows that gradients can reveal inputs analytically in networks containing fully connected layers, under a nonzero-gradient condition. This reconstruction is independent of surrounding layer types and can expose activations in common prediction modules.

  • Input recovery depends on the relationship between input dimension n and gradient dimension p, as well as the gradient operator’s nonlinearity and conditioning.
  • A fully connected layer’s input can be computed analytically from parameter gradients, independent of the layer’s position or surrounding architecture.The result assumes a technical condition preventing zero gradients.
  • For a network of biased fully connected layers preceded by fully connected layers, the input is uniquely reconstructable when each layer’s loss derivative has a nonzero entry.
  • The proof uses the chain rule to iteratively recover preceding layer inputs from derivatives of the loss with respect to layer outputs.
  • Because many classifiers end with fully connected prediction layers, their input activations—and potentially ground-truth labels—can be exposed through gradients.

4 A Numerical Reconstruction Method

The proposed reconstruction method replaces Euclidean gradient matching with a magnitude-invariant directional loss and adversarial-optimization-inspired updates. It constrains candidate images to valid pixel ranges and adds a total-variation prior.

  • The method matches the direction of observed and candidate gradients rather than their magnitudes, reducing sensitivity to gradient scale.
  • Candidate images are constrained to [0,1]^n and regularized with total variation.
  • The directional objective is optimized using only the sign of its gradient, with Adam and step-size decay inspired by adversarial-attack optimization.
  • Although signed gradients affect Adam’s momentum, the accumulated momentum still produces unsigned update steps that can accurately recover an image.
  • The authors treat the label as known because prior work analytically reconstructs label information for classification tasks.

5 Single Image Reconstruction from a Single Gradient

Single-image experiments show that gradient inversion remains effective on trained, deep, and realistic networks, including ResNet-152. Reconstruction quality varies with image content and architecture, while training and augmentation alter recovered details and localization.

  • The proposed method recognizes images from trained ResNets, whereas Euclidean loss with L-BFGS completely fails on the trained ResNet baseline.
  • Information leakage depends strongly on image content: some ImageNet examples are highly compromised, while others leak almost no usable information.
  • Trained networks produce smaller gradient magnitudes, but magnitude-oblivious inversion still recovers important visual information from gradient direction.
  • Reconstructions from trained networks are biased toward typical class features, obscuring fine details and backgrounds despite broad privacy compromise.
  • Training with data augmentation makes object localization more difficult and can duplicate objects, while reconstruction still succeeds with some location information lost.
  • Wider networks measurably improve reconstruction quality but increase variance and attacker computation rather than providing greater security.With multiple restarts, PSNR rises from 19 to almost 23 as channels increase from 16 to 128.
  • Faithful ImageNet reconstructions remain possible through a trained ResNet-152, with little degradation as network depth increases.

6 Distributed Learning with Federated Averaging and Multiple Images

The paper tests gradient inversion under federated averaging with multiple local updates and multiple images. Reconstruction remains possible across practical multi-image and multi-step settings, although quality varies and often decreases.

  • Federated Averaging: Federated averaging sends locally updated parameters after multiple mini-batch gradient steps and averages them across users.The local procedure uses E n/B update steps for n images, local epochs E, and mini-batch size B.
  • Multiple Local Updates: Multiple local epochs do not necessarily prevent reconstruction: a single image remains recoverable even after 100 local gradient descent steps.The only demonstrated failure used a learning rate of 1e-1, corresponding to a divergent and therefore unusable training update.
  • Multiple Images: 100-image gradient averaging still leaks recognizable information, although most reconstructed images are unrecognizable.Figure 6 shows the five most recognizable images from a CIFAR-100 batch of 100 images on ResNet32-10.
  • General Case: Privacy leakage persists across multi-image reconstruction tasks, including random mini-batches and multiple federated averaging epochs.Single-image reconstruction has the highest PSNR, but lower-quality reconstructions still show leakage in all evaluated multi-image settings.

7 Conclusions

The paper analyzes gradient inversion in modern computer-vision architectures and concludes that federated learning updates can reveal input data. It argues that provable differential privacy is needed to guarantee security, potentially even for larger batches.

  • 7 Conclusions: The study combines an analytical reconstruction result for fully connected layers with an optimization-based attack across architectures and network parameters.Experiments use modern computer-vision architectures for image classification.
  • 7 Conclusions: The experiments indicate that federated learning does not guarantee privacy, including when updates contain larger batches of data points.The conclusion identifies provable differential privacy as the only stated way to guarantee security.

Broader Impact - Federated Learning does not guarantee privacy

The paper shows that federated learning updates can enable image reconstruction under an honest-but-curious threat model, while malicious architectures or parameters can make reconstruction easier. It also identifies image classification as especially vulnerable and frames the attack as an initial step toward stronger privacy attacks.

  • Broader implications: Image classification may be especially vulnerable, and the demonstrated attack is described as a first step toward stronger attacks on privacy during collaborative training.The paper attributes this vulnerability to image structure, large classification networks, and users owning relatively few images.
  • Threat model: The study considers an honest-but-curious server that does not maliciously modify the architecture or global parameters.Allowing either type of modification changes the threat model examined in the main analysis.
  • Malicious model changes: Malicious architecture changes can make reconstruction nearly trivial by inserting a fully connected layer early or directly connecting the input to the network output.These mechanisms follow the analytical reconstruction result for fully connected layers.
  • Malicious model changes: Even with a fixed architecture, malicious global parameters can substantially improve reconstruction by propagating the input unchanged to the classification layer.For an architecture without strides that flattens convolutional features, setting convolution layers to identity enables analytical computation of the input.
  • Detection and cost: Optimizing the parameters sent to users could maximize reconstruction quality while being difficult to detect, but the attack would be computationally intensive.The paper presents this as a subtler theoretical approach than choosing conspicuously malicious parameters.
  • Label flipping: Permuting two classification-layer rows and biases flips label semantics, boosts gradient magnitude, and can improve reconstruction while remaining difficult for users to detect.The figure compares reconstructed images using PSNR and shows the gradient magnitude below each input.

B Experimental Details

The experiments use convolutional and residual architectures, reconstruct inputs by optimizing signed-gradient similarity, and examine federated averaging and attack-design choices. The ablation identifies Adam and similarity loss as central, while total variation and signed gradients provide smaller benefits.

  • Architecture: The baseline ConvNet has eight convolution layers, batch normalization and ReLU after each layer, and default channel scale D = 64.It is used because it is relatively fast to optimize and exceeds 90% accuracy on CIFAR-10.
  • Federated averaging: Federated averaging updates can be rewritten as averages of local updated gradients when the server knows the old parameters, local updates, learning rate, and update count.The server subtracts the old parameters and applies the reconstruction approach to the resulting average update.
  • Assumptions: The number of local updates is assumed known to the server, although it could be found by brute force when the update count is small.This assumption applies to the federated-averaging reconstruction procedure.
  • Ablation: Adam and the similarity loss are central ablation choices, whereas total variation and signed gradients provide smaller improvements.The ablation is reported for a trained ResNet-18 on CIFAR-10.
  • Attack setup: The attack reconstructs network inputs with Adam, signed gradients, cosine similarity, Gaussian initialization, and total-variation regularization.The default total-variation parameter is 0.01, while attack hyperparameters depend on the scenario.

C.1 Settings for the experiments in Sec. 5

The experiments compare the proposed attack with prior optimization approaches on LeNet (Zhu) and ResNet20-4, using trained and untrained models. Additional experiments vary spatial information and batch composition on CIFAR-10.

  • Comparison to previous approaches: The comparison reimplements the network from [38] as LeNet (Zhu) and also evaluates ResNet20-4.Both architectures are tested with trained and untrained parameters.
  • Settings: The experiments use total-variation regularization values specified separately for the baseline and network-input reconstruction settings.These values are documented in Tables 4 and 5.
  • Comparison to previous approaches: The baseline L-BFGS-L2 method uses learning rate 1e-4 and 300 iterations, with 16 restarts for LeNet (Zhu) and 8 for ResNet experiments.The proposed approach uses one restart and 4800 iterations in the described settings.
  • Spatial Information: The spatial-information experiments use a ConvNet with D = 64 channels and 100 experiments on different CIFAR-10 validation images.Each experiment uses learning rate 1, 4800 iterations, one restart, and specified total-variation weights.

D Proofs for section 3.1

The proofs establish analytical input reconstruction at fully connected layers under nonzero downstream derivatives or bias gradients. Supporting experiments report extreme reconstruction examples for trained and untrained ConvNet and ResNet20-4 models.

  • Propositions: For a biased fully connected layer followed by ReLU, the input can be uniquely determined when a bias-gradient component is nonzero.The proof uses the relationship between the bias gradient and the derivative with respect to the layer preactivation.
  • Experimental illustrations: Reported reconstruction values include 18.04dB, 14.85dB, 14.60dB, and 30.26dB alongside corresponding scientific-notation quantities.The passage lists these values without identifying their individual row or metric mappings.
  • Experimental illustrations: Figure 9 compares worst- and best-case CIFAR-10 reconstructions for trained ConvNet and ResNet20-4 models, including untrained-model worst cases.The figure reports gradient magnitude below inputs and PSNR below reconstructions.
  • Propositions: For a fully connected layer followed by ReLU, a nonzero derivative with respect to an output coordinate enables deriving the input from the corresponding weight-gradient row.The condition is stated for an input xl and output derivative with at least one nonzero component.

E.2 Visualization of experiments in Sec. 5

The appendix visualizes reconstruction behavior across network width, depth, ImageNet examples, and multi-image experiments. It includes neutrally selected examples, failure cases, and evidence that multi-image success depends on initialization.

  • Network Width: Figure 10 shows reconstructions for the first six CIFAR images across ResNet-18 architectures with different widths.The experiment isolates network width as the varying architectural factor.
  • Network Depth: Figure 11 presents reconstruction results for different deep ResNet architectures.This visualization isolates network depth as the architectural factor.
  • ImageNet examples: ImageNet examples for trained ResNet-18 include very good, translated, and failure reconstructions; fine writing and exact petal counts remain hidden in cited failures.The examples come from the ILSVRC2012 validation set.
  • Multi-image recovery: The full 100-image CIFAR-100 multi-image recovery results are shown in Figure 20, while separate-image success is semi-random and depends on initialization.Figures 15–19 show subsets of the experiments under different E, n, and B settings.
  • ImageNet examples: Figures 13 and 14 use additional single-image reconstructions from trained ResNet-152 and select examples neutrally by ImageNet validation-set IDs.Rows place ground truth above reconstruction, and the examples are not handpicked.
Loading 2003.14053v2…