Source-linked AI summary
iDLG: Improved Deep Leakage from Gradients
Bo Zhao, Konda Reddy Mopuri, Hakan Bilen
TL;DR
The paper examines whether sharing gradients protects private training data in distributed learning, given DLG’s unreliable label recovery. It proposes iDLG, which analytically extracts labels from gradient signs before reconstructing data. iDLG extracts ground-truth labels with 100% accuracy and improves data-extraction fidelity, within a setting where gradients for every datum are shared.
Problem
Gradient sharing can leak private training data, while DLG does not consistently recover ground-truth labels or high-quality dummy data.
Method
iDLG analytically identifies ground-truth labels from gradient signs under cross-entropy loss with one-hot labels, then uses those labels for data extraction.
Results
iDLG extracts ground-truth labels with 100% accuracy and empirically demonstrates higher data-extraction fidelity than DLG on three datasets.
Takeaways & Limitations
Sharing gradients of a neural network trained with cross-entropy loss definitely leaks ground-truth labels, enabling more effective extraction with iDLG.
Takeaways & Limitations
iDLG identifies ground-truth labels only when gradients with respect to every sample in a training batch are provided.
Abstract
from arXiv · showhide
It is widely believed that sharing gradients will not leak private training data in distributed learning systems such as Collaborative Learning and Federated Learning, etc. Recently, Zhu et al. presented an approach which shows the possibility to obtain private training data from the publicly shared gradients. In their Deep Leakage from Gradient (DLG) method, they synthesize the dummy data and corresponding labels with the supervision of shared gradients. However, DLG has difficulty in convergence and discovering the ground-truth labels consistently. In this paper, we find that sharing gradients definitely leaks the ground-truth labels. We propose a simple but reliable approach to extract accurate data from the gradients. Particularly, our approach can certainly extract the ground-truth labels as opposed to DLG, hence we name it Improved DLG (iDLG). Our approach is valid for any differentiable model trained with cross-entropy loss over one-hot labels. We mathematically illustrate how our method can extract ground-truth labels from the gradients and empirically demonstrate the advantages over DLG.
1 Introduction
The paper addresses gradient sharing’s risk of private-data leakage by improving DLG, which struggles to recover labels reliably. iDLG analytically extracts ground-truth labels from gradient signs and uses them to improve data extraction.
- Motivation: Gradient sharing, commonly used in Collaborative Learning and Federated Learning, can expose private training data despite beliefs that sharing gradients protects it.DLG previously demonstrated the possibility of recovering private data from shared gradients.
- Prior approach: DLG synthesizes dummy data and labels by iteratively matching their gradients to shared gradients, but it often recovers incorrect labels and low-quality data.The method jointly updates randomly initialized dummy data and labels while minimizing gradient differences.
- Proposed approach: iDLG derives a gradient-sign relationship in cross-entropy classification that identifies the correct label because correct- and wrong-label gradients have opposite signs.The approach also applies the rule to last-layer weight gradients when output gradients are unavailable.
- Proposed approach: The extracted labels simplify DLG’s optimization objective, enabling more reliable recovery of good-quality data.The paper names this improved approach iDLG.
- Evaluation: 100% accuracy is reported for iDLG’s extracted labels, with improved extracted-data fidelity demonstrated against DLG on three datasets.The comparison evaluates label accuracy and data fidelity.
- Paper organization: The paper’s remaining sections present the label-extraction procedure, experimental comparison, and concluding discussion.
2 Methodology
iDLG first extracts the ground-truth label analytically from shared gradients, then reconstructs the private datum by matching dummy and shared gradients. The label rule relies on cross-entropy training with one-hot labels and applies across model architectures and training stages.
- 2.1 Extracting Ground-truth Labels: For cross-entropy loss over one-hot labels, the correct output has a negative loss gradient while incorrect outputs have positive gradients.The correct-label gradient lies in (−1, 0), whereas gradients for other labels lie in (0, 1).
- 2.1 Extracting Ground-truth Labels: When output gradients are unavailable, the method uses signs of last-layer weight gradients connected to each logit to identify the label.The last-layer gradient factors into the output-gradient term and the preceding activation vector, whose non-negative entries preserve the relevant sign relationship.
- 2.1 Extracting Ground-truth Labels: The predicted label is the index whose corresponding last-layer weight gradient is negative, while the other class gradients are non-negative.This rule is independent of model architectures and parameters, including the network’s training stage and initialization.
- 2.2 Improved DLG (iDLG): iDLG extracts the ground-truth label analytically from shared gradients before reconstructing the private datum.The method extracts a label c′, initializes dummy data, and optimizes it using gradient matching.
- 2.2 Improved DLG (iDLG): After label extraction, iDLG initializes dummy data randomly and updates it by minimizing the squared Frobenius distance between dummy and shared gradients.The procedure runs for N iterations with learning rate η using gradient descent on the dummy datum.
3 Experiments
Experiments compare DLG and iDLG on MNIST, CIFAR-100, and LFW using extracted-label accuracy, data fidelity, and convergence. iDLG consistently recovers labels and data more effectively, with a marked advantage on the harder LFW task.
- Experimental setup: Experiments evaluate DLG and iDLG on MNIST, CIFAR-100, and LFW using extracted-label accuracy and data fidelity.Both methods are optimized for 300 iterations, and experiments are repeated 1000 times with randomly initialized networks.
- Accuracy of Extracted Labels: iDLG always extracts the correct label, whereas DLG reaches 89.9% on MNIST, 83.3% on CIFAR-100, and 79.1% on LFW.DLG’s label accuracy decreases on the harder tasks.
- Fidelity of Extracted Data: iDLG consistently outperforms DLG in the percentage of extracted data meeting each MSE fidelity threshold across all three datasets.Lower thresholds represent stricter fidelity requirements, and iDLG’s advantage is especially notable on LFW.
- Fidelity of Extracted Data: On an LFW example, iDLG reaches similar performance in 90 training iterations, compared with 200 iterations for DLG.The example indicates that iDLG is easier to converge.
4 Discussion and Conclusion
The paper concludes that gradient signs analytically reveal ground-truth labels and support higher-fidelity data extraction. Its current method assumes gradients for every datum in a training batch are available.
- Conclusion: iDLG analytically relates labels to gradient signs and extracts ground-truth labels with 100% accuracy, facilitating higher-fidelity data extraction.The conclusion presents this as an approach to stealing data and corresponding labels from shared gradients.
- Limitation: The current method works only when gradients with respect to every sample in a training batch are provided.This is the stated scope boundary for identifying ground-truth labels.