Source-linked AI summary
Deep Leakage from Gradients
Ligeng Zhu, Zhijian Liu, Song Han
TL;DR
The paper asks whether gradient sharing truly protects private training data in distributed and collaborative learning. It introduces DLG, which optimizes dummy inputs and labels to match shared gradients, and shows pixel-wise image recovery and token-wise text matching across vision and language tasks. The paper concludes that gradient sharing poses critical privacy risks and evaluates defenses, while noting boundaries in the attack’s current operating settings.
Problem
The paper investigates whether gradients assumed safe to share can completely reveal participants’ private training data.
Method
DLG optimizes randomly initialized dummy inputs and labels to minimize the distance between their gradients and the shared real gradients.
Results
DLG recovers pixel-wise accurate images and token-wise matching texts from gradients without generative models or extra data priors.
Takeaways & Limitations
Gradient sharing is not always reliable for protecting training privacy, creating a security challenge for modern multi-node learning systems.
Abstract
from arXiv · showhide
Exchanging gradients is a widely used method in modern multi-node machine learning system (e.g., distributed training, collaborative learning). For a long time, people believed that gradients are safe to share: i.e., the training data will not be leaked by gradient exchange. However, we show that it is possible to obtain the private training data from the publicly shared gradients. We name this leakage as Deep Leakage from Gradient and empirically validate the effectiveness on both computer vision and natural language processing tasks. Experimental results show that our attack is much stronger than previous approaches: the recovery is pixel-wise accurate for images and token-wise matching for texts. We want to raise people's awareness to rethink the gradient's safety. Finally, we discuss several possible strategies to prevent such deep leakage. The most effective defense method is gradient pruning.
1 Introduction
The paper challenges the assumption that shared gradients protect training privacy by introducing an optimization attack that reconstructs private inputs and labels. DLG recovers exact images and texts across vision and language tasks and examines defenses against the leakage.
- Motivation: Distributed and collaborative learning keep training data local while synchronizing workers through exchanged gradients.This enables joint training across multiple data sources without centralizing their datasets.
- Method: DLG optimizes dummy inputs and labels to minimize their gradient distance from the shared gradients.It generates dummy data, computes dummy gradients, and updates the dummy inputs and labels rather than model weights.
- Problem: DLG demonstrates that private training data can be obtained from publicly shared gradients.The attack targets the privacy assumption underlying gradient sharing.
- Results: DLG can reveal pixel-wise accurate images and token-wise matching texts using only gradients.Unlike conventional approaches, it does not require extra prior information or generative models and recovers original samples rather than synthetic alternatives.
- Defenses: The paper analyzes gradient perturbation, low precision, and gradient compression as defenses against deep leakage.Its introduction reports that half precision fails, while pruning more than 20% of gradients successfully defends the attack.
2 Related Work
Prior work showed that gradients can reveal limited properties or components of training data, but distributed training research had not established full sample recovery. These approaches generally require additional information or produce unordered words and synthetic look-alike images.
- Distributed Training: Distributed training uses synchronous local computation followed by gradient exchange through centralized or decentralized architectures.Parameter-server systems aggregate gradients centrally, whereas decentralized systems exchange gradients between neighboring nodes.
- Collaborative Learning: Collaborative learning keeps privacy-sensitive datasets on participants’ local servers while sharing only gradients.The paper cites medical and predictive-keyboard applications as examples of this setup.
- Existing Leakage: Embedding gradients can reveal which words appeared in another participant’s training set, but the words are unordered and sentences are difficult to infer.This is characterized as shallow leakage.
- Existing Leakage: Membership and property inference classify batch contents, while GAN-based methods synthesize images resembling training data under additional attack limitations.These methods infer records or properties rather than directly recovering complete original samples.
3 Method
DLG steals training inputs and labels by matching gradients from dummy data to shared gradients, extending the attack to batched data and relating gradient distance to reconstruction error.
- Attack setup: DLG targets a participant’s training data from received gradients while model function and weights remain shared.Each participant computes gradients from a private minibatch, and the attacker seeks the corresponding inputs and labels.
- Gradient matching: The algorithm initializes dummy inputs and labels, computes their gradients, and optimizes the dummy data rather than model weights.The dummy gradients are matched to the real gradients using a differentiable distance and standard gradient-based optimization.
- Optimization requirement: Twice differentiability of the model is assumed because optimizing gradient distance with respect to dummy data requires second-order derivatives.The paper states that this assumption holds for most modern neural networks and tasks.
- Batched data: For batched data, naive optimization converges slowly because the batch has N! possible permutations, so DLG updates one training sample at a time.The method is modified to force optimization toward a solution for batched recovery.
- Reconstruction signal: As dummy and real gradient distance decreases, the leaked image becomes closer to the original image.This relationship is measured through layer-wise gradient MSE and image reconstruction MSE.
4 Experiments
Experiments evaluate DLG on image classification and masked language modeling, showing near-exact image recovery and gradient-matching-based recovery of original words.
- Image classification: Image experiments use ResNet-56 with MNIST, CIFAR-100, SVHN, and LFW, modifying activations and strides to support twice-differentiable optimization.Labels are optimized through a softmax-relaxed vector rather than directly as discrete categorical values.
- Image recovery: Recovered images are almost identical to ground truth, although complex faces require more iterations and may retain negligible artifact pixels.Monochrome images with clean backgrounds are easiest to recover, while complex images take longer.
- Comparison: Ours < 0.03 v.s. previous > 0.2 on MSE across all four image datasets.Images are normalized to the range [0, 1], and the comparison uses the MSE of leaked images against ground truth.
- Masked language modeling: The language experiment applies DLG to BERT-based masked language modeling by optimizing embeddings and reversely mapping them to the closest vocabulary entries.This adapts gradient matching to discrete words represented through continuous embeddings.
- Language recovery: During optimization, meaningful words and sentence fragments progressively appear as dummy embedding gradients approach the original gradients.One example shows “annual conference” appearing at iteration 20 and the leaked sentence becoming close to the original by iteration 30.
5 Defense Strategies
The paper evaluates noise, reduced precision, gradient pruning, training-setting changes, and cryptographic defenses against gradient leakage, revealing trade-offs between privacy protection and model accuracy. Gradient pruning is effective beyond roughly 20% sparsity, while cryptographic methods are secure but limited in compatibility or deployment scope.
- Noisy Gradients: Noise variance, rather than noise type, mainly determines defense strength: variance 10^-4 fails, 10^-3 leaves recoverable leakage, and variance above 10^-2 significantly degrades accuracy.The experiments compare Gaussian and Laplacian noise over variances from 10^-1 to 10^-4.
- Reduced Precision: Both IEEE float16 and bfloat16 fail to protect the training data.The paper also evaluates Int-8 representation as another low-bit perturbation.
- Gradient Compression and Sparsification: At 1%–10% sparsity, pruning has almost no effect; around 20% produces artifacts, and larger ratios make recovered images unrecognizable.Pruning makes gradient matching harder because small-magnitude gradients are set to zero.
- Gradient Compression and Sparsification: Gradient compression can exceed 300× without accuracy loss through error compensation, surpassing DLG’s approximately 20% sparsity tolerance.This suggests compression is a practical defense against deep leakage.
- Large Batch and High Resolution: Increasing batch size or input resolution makes leakage more difficult, but the evaluated attack currently works only up to batch size 8 and resolution 64×64.Input upscaling may require changes to CNN architectures.
- Cryptology: Cryptographic defenses are described as the most secure, but secure aggregation requires integer gradients and homomorphic encryption targets parameter-server settings only.These constraints limit general compatibility and applicability.
6 Conclusions
The paper introduces DLG to obtain local training data from publicly shared gradients without generative models or extra data priors. Experiments on vision and language tasks show critical leakage risks and indicate that defenses succeed only when they begin degrading accuracy.
- DLG obtains local training data from publicly shared gradients without relying on generative models or extra priors.The paper presents this as a challenge to distributed training and federated learning.
- Experiments on vision and language tasks demonstrate critical risks from deep gradient leakage.
- Defenses prevent deep leakage only when they start to degrade model accuracy.The conclusion frames this trade-off as a challenge for modern multi-node learning systems.