PaperlayerView exact revision on arXiv

Source-linked AI summary

Gradient Inversion with Generative Image Prior

Jinwoo Jeon, Jaechang Kim, Kangwook Lee, Sewoong Oh, Jungseul Ok

arXiv:2110.14962v1cs.LG

TL;DR

Gradient inversion threatens federated learning’s privacy premise because observed gradients can reveal private training data, especially when an attacker has a data prior. The paper introduces GIAS to exploit a pretrained generative model and GIML to learn such a prior from gradients alone, finding significant leakage and effective recovery from the learned prior. These results motivate stronger and more conservative privacy defenses.

  • Problem

    Gradient inversion can recover private user data from FL gradients, while input reconstruction is often under-determined without prior knowledge.

  • Method

    The paper proposes GIAS for inversion with a pretrained generative prior and GIML for meta-learning a generative model from multiple observed gradients.

  • Results

    GIAS demonstrates significant privacy leakage, and GIML learns a generative prior from gradients that provides the same data-recovery level as a given pretrained model.

  • Takeaways & Limitations

    The findings warn that existing FL defense standards may be insufficient and support using a higher, more conservative standard for privacy protection.

  • Takeaways & Limitations

    The applicability of an existing batch-normalization-statistics attack is limited when clients do not report exact statistics and approximations require recomputation over the entire data distribution as the model changes.

Abstract

from arXiv · show

Federated Learning (FL) is a distributed learning framework, in which the local data never leaves clients devices to preserve privacy, and the server trains models on the data via accessing only the gradients of those local data. Without further privacy mechanisms such as differential privacy, this leaves the system vulnerable against an attacker who inverts those gradients to reveal clients sensitive data. However, a gradient is often insufficient to reconstruct the user data without any prior knowledge. By exploiting a generative model pretrained on the data distribution, we demonstrate that data privacy can be easily breached. Further, when such prior knowledge is unavailable, we investigate the possibility of learning the prior from a sequence of gradients seen in the process of FL training. We experimentally show that the prior in a form of generative model is learnable from iterative interactions in FL. Our findings strongly suggest that additional mechanisms are necessary to prevent privacy leakage in FL.

1 Introduction

Federated learning shares model updates rather than user data, but gradients can still leak private training data. This paper develops attacks that exploit pretrained or gradient-learned generative priors and reports strong privacy leakage across FL settings.

  • Federated learning aggregates model updates instead of directly sharing end-user data, motivating its privacy premise.
  • Gradient inversion can recover private user data from observed gradients, threatening the premise that FL protects data privacy.
  • GIAS searches a pretrained generative model’s lower-dimensional latent space, then adapts the model to each input using the gradient.The two stages each provide substantial reconstruction improvement.
  • GIML meta-learns a generative model from multiple gradients observed across FL epochs or participating nodes when the user distribution is initially unknown.The learned model is then used to reconstruct data.
  • GIAS demonstrates significant privacy leakage in challenging FL scenarios, while GIML learns an explicit prior from gradients and achieves the same recovery level as a given pretrained model.
  • The results warn that defenses judged safe under earlier gradient-inversion standards may require more conservative evaluation and additional protection.

2 Related work

Prior work studies privacy attacks through gradient inversion and generative-model training, but existing methods often require auxiliary information or a pretrained model. This paper instead learns and exploits a generative prior using gradients alone.

  • Earlier FL attacks recover information through membership inference, shallow-network reconstruction, or analytical inversion across broader model classes.
  • Optimization-based inversion incorporates priors such as total variation, batch-normalization statistics, or pretrained generative models.
  • Unlike prior generative-model attacks requiring a pretrained model, GIML trains a generative model from gradients only.
  • Earlier generative-model training attacks require an auxiliary dataset, whereas this paper trains and uses a generative model using transmitted gradients only.

3 Problem formulation

The gradient-inversion problem asks whether sampled inputs can be reconstructed from gradients computed during supervised FL training. Because gradients may be under-determined, the formulation focuses on input recovery given true labels.

  • In FL, a node reports gradients computed from sampled input-label pairs instead of transferring the data directly.
  • Gradient inversion reconstructs the sampled data that produced a reported gradient.
  • The inversion objective minimizes discrepancy between reported and reconstructed gradients, using measures such as ℓ2 distance or negative cosine similarity.
  • Labels can be recovered accurately from the final-layer gradient, but reconstructing inputs remains challenging because the problem is often under-determined.

4 Methods

The methods use generative models to constrain gradient inversion, first adapting a pretrained model and then learning such a prior from multiple inversion tasks. GIAS searches latent and parameter spaces sequentially, while GIML meta-learns model parameters from gradients.

  • Gradient inversion with trained generative model: GIAS mitigates under-determination by searching the lower-dimensional latent space of a pretrained generative model rather than the ambient input space.The latent dimension k is typically much smaller than the input dimension m, narrowing the reconstruction domain.
  • Gradient inversion with trained generative model: If the generator approximates the input data accurately, latent-space inversion reconstructs the target under the stated canonical-model assumptions.The reconstruction error bound δ(ε) approaches zero as the generator approximation error ε approaches zero.
  • Gradient inversion with trained generative model: GIAS first searches latent codes and then adapts generative-model parameters to each input using the observed gradient.The parameter search addresses reconstruction error caused by imperfections in the pretrained generator.
  • Gradient inversion with trained generative model: GIAS optimizes w after z because joint optimization or optimizing w first reduces the benefit of the smaller latent search space.The paper empirically justifies this search order and reports that recursive implementations can suffer error accumulation in deep networks.
  • Gradient inversion to meta-learn generative model: GIML learns a generative model from a set of gradient inversion tasks when no pretrained prior is available.It updates shared model parameters using tasks subsampled from S, rather than solving each task independently.
  • Gradient inversion to meta-learn generative model: GIML regularizes latent-space search and integrates parameter optimization because latent search with an untrained generator can diverge.After meta-learning, the trained parameters are used with GIAS for gradient inversion.

5 Experiments

The experiments evaluate gradient inversion across search-space choices, baselines, varying inversion difficulty, and learning a generative prior from gradients. GIAS improves reconstruction with pretrained priors, while GIML learns a useful prior through iterative gradient-based training.

  • Algorithms: GI-z/w is the proposed GIAS method, whereas GI-x searches the input space and the algorithms differ by their optimized spaces.The evaluated spaces are the input x, latent code z, and model parameters w.
  • Comparison with state-of-the-art models: Adding generative modeling improves PSNR, SSIM, and LPIPS over GI-x and GI-x + BN baselines.GI-z/w without BN has lower reconstruction error than GI-x + BN.
  • Comparison with state-of-the-art models: GIAS is superior in best-in-batch performance, which better reflects worst-case privacy risk than average performance.The comparison covers average PSNR and best PSNR throughout the experiments.
  • The gain from fully exploiting pretrained generative model: Larger batch size, higher gradient sparsity, and more gradient noise make inversion harder, but GI-z/w surpasses GI-x across difficulty levels.With batch size 4, GI-x has averaged PSNR comparable to GI-z/w at batch size 32 or 99% sparsity.
  • Learning generative model from gradients: GIML improves generative-model quality and reconstruction as its training progresses, while an improper CIFAR10 prior causes GI-z to fail on the target data.The experiments use gradient sparsity 0.95 in this difficult setting.
  • Learning generative model from gradients: A generative model trained with GIML yields higher reconstruction quality and faster convergence than a wrong or untrained model.The experiment generates gradients from four rounds of reports from 200 nodes for computationally tractable DCGAN training.

6 Conclusion

The conclusion presents GIAS and GIML as complementary attacks: one exploits a pretrained generative prior, while the other learns such a prior from gradients. The accompanying algorithms specify latent-space and parameter-space optimization procedures for inversion and meta-learning.

  • Conclusion: GIAS exploits a pretrained generative model to improve gradient inversion and demonstrates significant privacy leakage in challenging FL scenarios.The method adds gains to existing approaches and uses the generative prior during reconstruction.
  • Conclusion: GIML meta-learns a generative model from gradients observed across multiple inversion tasks, such as FL epochs or participating nodes.Its purpose is to learn an explicit prior from gradients rather than assume one is pretrained.
  • GIAS: GIAS first searches latent codes and then searches generator parameters before returning reconstructed samples.The algorithm initializes latent codes randomly, optimizes z, replicates generator parameters across samples, optimizes w, and returns Gw′(z).
  • GIML: GIML samples inversion tasks, performs regularized latent-space searches, updates task-specific parameters, and moves the meta-parameters toward those adapted parameters.The procedure uses task batches, local iterations, a latent regularizer, and step sizes α and β.

B Proof of Property 1

The appendix proves Property 1 by combining continuity of the inversion cost with a generative model’s approximation of the target input. These assumptions yield a controlled reconstruction error, while the figures illustrate layer-wise error behavior.

  • B Proof of Property 1: Property 1 follows directly from Lemmas 1 and 2 under the stated standard-network assumptions.The proof connects the approximation result to the continuity conditions established for the model class.
  • B Proof of Property 1: Lemma 1 extends Property 1 to a generative model that approximates the target input within error ε.The cost function is continuous and has a unique global minimizer at the target.
  • B Proof of Property 1: As the generative model approximation error ε approaches zero, the reconstructed solution’s distance from the target is bounded by δ(ε), which also approaches zero.This follows from continuity of the cost and uniqueness of the minimizer.
  • B Proof of Property 1: Lemma 2 establishes continuity of the inversion cost for standard R-layer neural networks with continuously differentiable activations, losses, and ℓ2 discrepancy.The stated model class includes multilayer perceptrons and convolutional neural networks.
  • B Proof of Property 1: The appendix figures show rank deficiency in a convolution layer and layer-wise error growth in gradient inversion.Figure A1 compares R-GAP with its generative-model variant, while Figure A2 depicts convolution-layer errors.

C Another method for gradient inversion: R-GAP [32]

R-GAP recursively reconstructs intermediate layer outputs through linear programs, while a generative model can replace parts of this search. The hybrid approach gains from reduced search spaces but inherits error accumulation across layers.

  • C Another method for gradient inversion: R-GAP [32]: R-GAP decomposes gradient inversion into recursive linear programs that reconstruct layer outputs from the penultimate layer toward the input.The method excludes the final layer’s output from this recursive reconstruction.
  • C Another method for gradient inversion: R-GAP [32]: Each R-GAP linear program uses a matrix A_r and vector b_r determined by previously reconstructed activations, layer parameters, and gradients.Its closed-form solution can reduce computational cost in some cases.
  • C Another method for gradient inversion: R-GAP [32]: A generative model can represent a layer output and replace recursive reconstruction with optimization over an alternated search space.The generative representation is f_θ,r(G_w(z)), and the search space may be changed arbitrarily.
  • C Another method for gradient inversion: R-GAP [32]: Reconstruction error from upper layers propagates downward, so deeper models reduce reconstruction quality while increasing parameter count.This is identified as a limitation inherited from R-GAP and illustrated by cumulative layer-wise error.
  • C Another method for gradient inversion: R-GAP [32]: The generative-model variant gains substantially for a few layers, especially when the linear program is under-determined, but it can incur larger errors in other settings.The comparison describes a trade-off between linear programming and generative-model optimization.
  • D Another potential gain from inverting a set of gradients: With multiple gradients, a learned generative model can overcome the limit of inverting a single gradient, and accumulated observations can eventually yield perfect reconstruction.The potential gain is demonstrated on a sequence of gradients from an FL procedure.

E Strong generative prior

A stronger prior increases the benefit of generative-model-based gradient inversion. Experiments using FFHQ indicate a larger advantage than on the more diverse ImageNet data.

  • E Strong generative prior: On FFHQ, GI-z significantly outperforms GI-x, whereas their performance gap is small on ImageNet.The paper attributes this difference to FFHQ’s lower diversity relative to ImageNet’s one-thousand-class distribution.

F Possible defense methods against gradient inversion attacks

The paper discusses label obfuscation, large mini-batches, and Gaussian noise as defenses against gradient inversion. These defenses reduce reconstruction quality but can trade privacy against FL training stability.

  • F Possible defense methods against gradient inversion attacks: Making label reconstruction challenging may reduce generative-prior attack gains because labels narrow the candidate input set.The paper links this defense to conditional generative models.
  • F Possible defense methods against gradient inversion attacks: Larger mini-batches significantly reduce reconstructed-data quality for GI-z/w, GI-x, and GI-x+BN as measured by PSNR.The cited defense works by reducing the information contributed by each individual data item.
  • F Possible defense methods against gradient inversion attacks: Sufficiently large Gaussian noise can prevent gradient inversion algorithms, including the paper’s methods, from optimizing their objectives.Combining large mini-batches with noise further increases reconstruction degradation.
  • F Possible defense methods against gradient inversion attacks: Defense mechanisms require balancing FL stability against privacy leakage because these approaches can make model training unstable.The paper emphasizes that each defense has distinct advantages and disadvantages.

G Convergence speed comparison with GIML

GIAS converges faster when using a meta-learned generative model trained with GIML. The comparison includes wrong and untrained generative models, with GIML also improving convergence speed.

  • G Convergence speed comparison with GIML: GI-w does not perform latent-space search for 0 to 1000 iterations in the reported comparison.
  • G Convergence speed comparison with GIML: GIAS with a GIML meta-learned generative model converges faster than the other compared models.Figure A4 compares it with a wrong GIML model and an untrained generative model.
  • G Convergence speed comparison with GIML: Each GIAS inversion corresponds to a task, so meta-learning improves not only performance but also convergence speed.

H Experiment settings

Experiments use resized ImageNet images and a randomly initialized ResNet18, selected as the most difficult setting within the ResNet architecture. The paper also notes implementation and licensing details, including an imperfect baseline implementation.

  • H Experiment settings: The default task is 64 × 64 image classification on ImageNet using a randomly initialized ResNet18 learning model.Resizing is used for computational tractability.
  • H Experiment settings: ResNet18 is chosen as the most difficult setting within the ResNet architecture because deeper and wider models make gradient inversion easier.
  • H Experiment settings: The baseline implementation for [29] uses BNexact and group lazy regularizers rather than the group registration regularizer.
  • H Experiment settings: The authors state that this baseline implementation might be imperfect, while still demonstrating performance improvement from adding their method.
  • H Experiment settings: ImageNet and FFHQ are used under licenses permitting specified non-commercial research or redistribution and adaptation, respectively.The paper states that the data themselves were not included, redistributed, or changed.
  • H Experiment settings: Some source-code components come from open-source implementations of previous research, with further details in the source-code README.
Loading 2110.14962v1…