PaperlayerView exact revision on arXiv

Source-linked AI summary

Auditing Privacy Defenses in Federated Learning via Generative Gradient Leakage

Zhuohang Li, Jiaxin Zhang, Luyang Liu, Jian Liu

arXiv:2203.15696v1cs.CV

TL;DR

Federated learning can leak private training information through shared gradients, while existing defenses and reconstruction methods leave important privacy questions unresolved. The paper introduces Generative Gradient Leakage, combining GAN image priors with adaptive objectives and gradient-free optimization. Experiments show that high-fidelity images remain recoverable under certain defenses, motivating GGL as an empirical privacy-auditing tool.

  • Problem

    Shared gradients can leak private data, while existing reconstruction studies and defenses provide limited evidence about privacy under degraded-gradient settings.

  • Method

    GGL optimizes gradient matching in a GAN-learned image manifold using adaptive losses for defense transformations and gradient-free optimization.

  • Results

    GGL reconstructs high-fidelity images from shared gradients under certain defenses on two image datasets, while existing attacks fail.

  • Takeaways & Limitations

    GGL can serve as an empirical privacy-auditing tool for evaluating and improving federated-learning defenses.

  • Takeaways & Limitations

    Prior reconstruction research often assumes an undefended federated-learning setting, and the paper’s demonstrations concern certain defenses and two image datasets.

Abstract

from arXiv · show

Federated Learning (FL) framework brings privacy benefits to distributed learning systems by allowing multiple clients to participate in a learning task under the coordination of a central server without exchanging their private data. However, recent studies have revealed that private information can still be leaked through shared gradient information. To further protect user's privacy, several defense mechanisms have been proposed to prevent privacy leakage via gradient information degradation methods, such as using additive noise or gradient compression before sharing it with the server. In this work, we validate that the private training data can still be leaked under certain defense settings with a new type of leakage, i.e., Generative Gradient Leakage (GGL). Unlike existing methods that only rely on gradient information to reconstruct data, our method leverages the latent space of generative adversarial networks (GAN) learned from public image datasets as a prior to compensate for the informational loss during gradient degradation. To address the nonlinearity caused by the gradient operator and the GAN model, we explore various gradient-free optimization methods (e.g., evolution strategies and Bayesian optimization) and empirically show their superiority in reconstructing high-quality images from gradients compared to gradient-based optimizers. We hope the proposed method can serve as a tool for empirically measuring the amount of privacy leakage to facilitate the design of more robust defense mechanisms.

1. Introduction

Federated learning protects privacy by keeping client data local, but shared gradients can still leak sensitive information despite gradient-degradation defenses. This work introduces Generative Gradient Leakage, which uses generative priors and gradient-free optimization to reconstruct high-quality private images under such defenses.

  • Federated learning lets clients collaboratively train a global model by sharing updates or gradients rather than private data.
  • Existing defenses degrade gradients with mechanisms such as differential privacy, compression, or representation perturbation, yet their privacy guarantees against leakage require further evaluation.
  • Generative Gradient Leakage solves gradient reconstruction in a GAN-learned image manifold, using public-data priors to compensate for degraded gradient information.
  • The method uses adaptive loss functions that account for gradient transformations and compares gradient-free optimizers with conventional gradient-based optimizers.
  • On CelebA and ImageNet, GGL recovers high-resolution images under the considered defenses, whereas existing gradient leakage attacks fail.

2. Related Work

Prior work established that model updates can reveal private information and reconstruct training data, but much of that literature evaluates undefended federated learning. Related privacy-preservation research therefore studies cryptographic protection and deliberate gradient degradation.

  • Privacy leakage research progressed from membership inference to inferring attributes and reconstructing client data from exchanged model updates.
  • Existing reconstruction studies often assume a bare-bone federated learning system without additional privacy defenses, limiting their alignment with industrial practice.
  • Privacy-preservation approaches are broadly categorized as cryptography-based or gradient-degradation-based methods.
  • Secure multiparty computation uses protocols such as homomorphic encryption or secret sharing, but MPC alone is not sufficient to resist inference attacks over outputs.
  • Gradient-degradation defenses include differential privacy, compression or sparsification, and representation perturbation, trading reduced information disclosure against possible model utility loss.

3. Methodology

The method models defended gradient leakage as a nonlinear inverse problem and reconstructs images using a generative prior, adaptive defense modeling, and optimization strategies suited to the resulting nonconvex objective.

  • Threat Model: The threat model assumes an honest-but-curious server observes gradients after clients apply local privacy transformations and may estimate unknown defense parameters.The adversary can use public datasets and may know or infer the client’s defense strategy.
  • Problem Formulation: Gradient reconstruction is formulated as matching observed gradients produced by a forward operator that computes loss gradients for a candidate image and label.The defended formulation applies a lossy transformation and additive noise to the forward gradients.
  • Generative Prior: Existing hand-crafted image priors can return invalid natural images when gradients are low-fidelity and noisy, motivating a learned generative prior.The limitation is attributed to the restricted identification ability of standard priors under degraded gradients.
  • Generative Gradient Leakage: GGL optimizes a latent vector through a generator while matching transformed generated gradients to observed gradients and regularizing the latent distribution.The reconstructed image is obtained as G(z*) after solving the latent-space optimization problem.
  • Label Inference: The attack can infer classification labels from the sign pattern of final-layer gradients and use the inferred label in the training-loss evaluation.For one-hot cross-entropy with softmax, the negative final-layer gradient identifies the ground-truth class index.
  • Defense Transformations: The attack accounts for client defenses including clipping, sparsification, and representation perturbation by modeling or reverse-engineering their transformations.Representation perturbation masks selected defended-layer gradients, while clipping and sparsification alter gradient magnitude or support.
  • Optimization Strategy: Gradient-free Bayesian optimization is used for global search in the GAN latent space because the inverse problem is highly nonlinear and non-convex.The method uses trust region Bayesian optimization to handle high-dimensional latent optimization.

4. Experiments

Experiments evaluate GGL on CelebA and ImageNet using multiple attack baselines, optimizers, and gradient defenses. GGL reconstructs informative or high-quality images under the considered defenses, while gradient-free optimization is especially effective on ImageNet.

  • Experimental Setup: Experiments use CelebA gender classification and ImageNet 1000-class classification with ResNet-18 models and compare GGL against established gradient leakage attacks.CelebA images are 32 × 32 and ImageNet images are 224 × 224.
  • Loss Configuration: Squared ℓ2 gradient matching with KLD regularization produces the best reconstruction among the tested loss configurations on CelebA.The comparison uses MSE-I and PSNR over 10 randomly selected CelebA images with Adam optimization.
  • Choice of Optimization Strategy: On ImageNet, gradient-free BO and CMA-ES recover samples resembling the private images, whereas Adam recovers little beyond the class label and depends strongly on initialization.On CelebA, Adam performs slightly better visually and statistically than the gradient-free optimizers.
  • Experimental Setup: The evaluation considers additive noise, gradient clipping, 90% gradient sparsification, and 80% Soteria representation pruning.The noise setting uses σ = 0.1, while clipping uses S = 4.
  • Comparison Under Defenses: GGL achieves PSNR >10.1 on CelebA and >5.7 on ImageNet across the evaluated defense scenarios, while existing attacks generally fail to reconstruct realistic images.Gradient clipping is an exception for IG because cosine-distance matching is invariant to gradient magnitude.
  • Comparison Under Defenses: Under defenses, CelebA reconstructions reveal attributes such as gender, hair, skin color, posture, and background, while ImageNet reconstructions preserve the original composition.Combined clipping and noise still permits high-resolution reconstruction, although it produces the lowest PSNR among the compared combined-defense settings.

5. Discussion

GGL can recover private images from degraded gradients, but its GAN prior constrains faithful reconstruction of out-of-distribution samples. The gradient-matching landscape is non-convex and imperfectly aligned with visual similarity, yet these issues do not by themselves guarantee privacy.

  • Limitations: GGL’s GAN latent-space constraint makes faithful reconstruction difficult for out-of-distribution images.Examples include changed object orientation and missing semantics in reconstructed ImageNet samples.
  • Loss Landscape: Gradient-matching loss is non-convex, contains local minima, and can disagree with the visually most similar reconstruction.The observed gradient-based loss surface differs from the ground-truth LPIPS landscape.
  • Privacy Implications: The observed loss-surface inconsistency is insufficient to provide privacy guarantees against generative gradient leakage attacks.The paper reports that even suboptimal reconstructions can still reveal private information.

6. Conclusion

The paper presents GGL, which uses public-data generative priors to reconstruct images from gradients degraded by privacy defenses. Experiments show resilient recovery of high-fidelity images when existing attacks fail, supporting GGL as an empirical privacy-auditing tool.

  • Conclusion: GGL uses generative-model priors learned from public datasets to improve reconstruction from degraded gradients.The method is evaluated on two image-classification datasets.
  • Conclusion: GGL reconstructs high-fidelity images under gradient perturbations and lossy transformations where existing attacks fail.The reported result concerns experiments on two image classification datasets.
  • Conclusion: The authors propose GGL as an empirical analysis tool to support future privacy-defense design.Its intended use is empirical privacy auditing.

A. Additional Reconstruction Samples

Additional ImageNet reconstructions evaluate GGL under four gradient defenses, while Figure 10 presents the corresponding original images and reconstructions.

  • Additional Reconstruction Samples: The additional ImageNet results cover additive noise, gradient clipping, gradient sparsification, and Soteria defenses.The listed settings use σ = 0.1, S = 4, 90% pruning, and 80% pruning, respectively.
  • Additional Reconstruction Samples: Figure 10 arranges original ImageNet images in the first row and GGL reconstructions under various defenses in the remaining rows.The figure provides visual comparisons across defense settings.

B. Implementation Details

The implementation uses dataset-specific GAN configurations: a trained DCGAN for CelebA and a pretrained BigGAN for ImageNet. Additional figures document the CelebA generator structure.

  • Optimization Configuration: Adam uses dataset-specific learning-rate schedules, while BO is configured with the TurBO-1 algorithm.The passage specifies warm-up and cosine decay for ImageNet Adam optimization.
  • GAN Configuration: CelebA uses a DCGAN with latent dimension 128, trained for 100 epochs with Wasserstein loss and gradient penalty weight 10.Training uses Adam with learning rate 0.0001 and batch size 64.
  • GAN Configuration: ImageNet uses a pretrained BigGAN with latent dimension 128 and 256 × 256 output images.The configuration is dataset-specific and differs from the trained CelebA DCGAN.
  • GAN Configuration: Figure 11 presents the GAN structure used for the CelebA dataset.The figure accompanies the CelebA generator configuration.

C. Loss Landscape Analysis

The analysis compares gradient-based reconstruction with GAN inversion and examines how defenses reshape the loss landscape. Adaptive transformation at the adversary’s side mitigates defense-induced deformation, enabling high-quality reconstruction.

  • Comparison with GAN Inversion: GAN inversion provides an upper-bound reference for image quality because it inverts a known image into the GAN latent space.The attack instead reconstructs an unknown private image from shared gradients, so its quality is constrained by the generator’s fitting ability.
  • Comparison with GAN Inversion: Figure 12 compares images reconstructed by the proposed method with images produced by GAN inversion.The comparison directly contrasts gradient-based recovery with the GAN’s image-fitting reference.
  • Different Defenses: Additive noise and gradient sparsification have limited impact on the observed loss landscape, whereas adaptive transformation greatly mitigates defense-induced deformation.The transformed landscape enables reconstruction of high-quality images despite these defenses.

D. Larger Batch Sizes or Multiple Local Steps

The paper evaluates GGL beyond the basic single-image, single-step setting. GGL restores visual information for batched high-resolution images and multiple local steps, including images protected by Soteria.

  • Scope and Prior Limitations: Existing studies generally struggle with high-resolution batch recovery and multiple local steps, while GGL targets privacy leakage under these settings.Prior work is described as limited to small images, a single ImageNet image, or local step = 1 in the cited comparisons.
  • GGL Under Extended Settings: GGL restores a decent amount of visual information for batched ImageNet images at 224×224px and for experiments with multiple local steps.These evaluations are presented in Figures 14 and 15.
  • GGL Under Extended Settings: The proposed GGL can be strengthened with additional prior information, such as batch-normalization statistics.This is stated as a possible enhancement after evaluating batched images and multiple local steps.
  • GGL Under Extended Settings: With batch size = 4, Figure 14 compares original images with reconstructions from GGL without defense and with Soteria defense.The figure organizes originals in the first row, undefended reconstructions in the second, and Soteria-protected reconstructions in the third.
  • GGL Under Extended Settings: Figure 15 presents reconstructions by GGL when multiple local steps are used.This extends the evaluation beyond the single-local-step setting.

E. Recovering In-the-wild Data

The evaluation targets a practical attacker who can use publicly accessible data as prior information. Testing images are kept disjoint from GAN training data, including tests outside the GAN training distribution.

  • Evaluation Setting: CelebA and ImageNet are used because they are Internet-based datasets that an attacker can access as public prior information.The evaluation is designed around the practical scenario in which attackers exploit publicly accessible data.
  • Evaluation Setting: The GAN training and evaluation images are disjoint, so the testing images were not used during GAN training.The setup isolates reconstruction of evaluation images from direct exposure during prior learning.
  • Evaluation Setting: The study also examines GGL when the testing image is not from the GAN training distribution.This probes performance beyond the distribution used to train the GAN prior.
Loading 2203.15696v1…