Source-linked AI summary
Amnesiac Machine Learning
Laura Graves, Vineel Nagisetty, Vijay Ganesh
TL;DR
The paper examines how to delete learned personal data from neural networks under GDPR when ordinary record deletion may leave information exposed to leakage attacks. It proposes Unlearning and Amnesiac Unlearning and empirically evaluates their efficiency, privacy protection, and model performance. The reported results indicate that these methods remove sensitive learned information while preserving performance on unrelated data, with Amnesiac Unlearning requiring substantial update storage.
Problem
Deleting training records may not remove their learned information from neural networks, which can leak class information or record membership through model inversion and membership inference attacks.
Method
The paper introduces Unlearning and Amnesiac Unlearning and evaluates them against naive retraining using attack, accuracy, efficiency, and removal-process experiments.
Results
The methods are reported to efficiently remove traces of sensitive data exposed by state-of-the-art attacks without harming performance on unrelated data.
Takeaways & Limitations
Amnesiac Unlearning is most effective for laser-focused removal of small data segments, while both proposed methods are presented as practical deletion approaches for trained models.
Takeaways & Limitations
Amnesiac Unlearning requires large storage for parameter updates, and its model efficacy decreases when it is used to remove more batches frequently.
Abstract
from arXiv · showhide
The Right to be Forgotten is part of the recently enacted General Data Protection Regulation (GDPR) law that affects any data holder that has data on European Union residents. It gives EU residents the ability to request deletion of their personal data, including training records used to train machine learning models. Unfortunately, Deep Neural Network models are vulnerable to information leaking attacks such as model inversion attacks which extract class information from a trained model and membership inference attacks which determine the presence of an example in a model's training data. If a malicious party can mount an attack and learn private information that was meant to be removed, then it implies that the model owner has not properly protected their user's rights and their models may not be compliant with the GDPR law. In this paper, we present two efficient methods that address this question of how a model owner or data holder may delete personal data from models in such a way that they may not be vulnerable to model inversion and membership inference attacks while maintaining model efficacy. We start by presenting a real-world threat model that shows that simply removing training data is insufficient to protect users. We follow that up with two data removal methods, namely Unlearning and Amnesiac Unlearning, that enable model owners to protect themselves against such attacks while being compliant with regulations. We provide extensive empirical analysis that show that these methods are indeed efficient, safe to apply, effectively remove learned information about sensitive data from trained models while maintaining model efficacy.
Introduction
The paper addresses GDPR-driven deletion of learned personal data from neural networks, where simply deleting training records may leave information vulnerable to leakage attacks. It introduces Unlearning and Amnesiac Unlearning as efficient alternatives intended to remove learned data while preserving model performance.
- GDPR’s right to be forgotten requires businesses to erase individuals’ personal data without undue delay.
- Neural networks can leak training information through model inversion and membership inference attacks, so deleting records alone may not protect removed data.
- The paper asks how to remove learned data safely and efficiently without harming network performance or leaving models vulnerable to state-of-the-art leakage attacks.
- It introduces Unlearning and Amnesiac Unlearning as efficient methods for removing learned data from neural networks.
- The evaluation measures attack protection, efficiency, and model performance, including model inversion and membership inference defenses.
Related Work
Related work establishes that machine learning models can leak record membership and class information, while existing removal and privacy approaches address different model types or training settings. The paper positions its methods for already-trained deep neural networks without requiring aggregate weak-learner architectures.
- Membership inference attacks determine whether specific records appeared in training data, while model inversion attacks recover class information.
- Differential privacy bounds information leakage from individual records but represents a different approach to GDPR-compliant machine learning training.
- Prior removal methods addressed specific data points in k-means models or linear classifiers through removal efficiency and certified removal.
- SISA training reduces retraining by partitioning data across disjoint sub-models and retraining only affected components after removal requests.
- By contrast, this paper targets already-trained deep learning models independently of neural-network architecture and without requiring aggregate weak learners.
Proposed Methods
The proposed methods remove learned sensitive information through either relabeling and retraining or selectively reversing recorded batch updates. Their trade-offs concern retained sensitive-data copies, update-storage requirements, and the amount of data removed at once.
- Naive Retraining: Naive retraining removes sensitive records and retrains the model, but evaluation finds it too slow and insufficient to prevent data leaks.
- Unlearning: Unlearning randomly relabels sensitive examples with incorrect labels and retrains for a small number of iterations to muddy the model’s understanding.
- Unlearning: Unlearning requires maintaining a copy of sensitive data during processing, creating a potential legal concern despite the GDPR’s removal window.
- Amnesiac Unlearning: Amnesiac Unlearning records batch membership and parameter updates during training, then removes updates from batches containing sensitive data.
- Amnesiac Unlearning: Because updates depend on current parameters, reversing selected batches does not reproduce training directly on the remaining data.
- Amnesiac Unlearning: When few sensitive batches are affected, Amnesiac Unlearning enables focused single-record removal with minimal impact on the rest of the model.
- Amnesiac Unlearning: Amnesiac Unlearning requires substantial storage for per-batch parameter updates, so storage-constrained owners may prefer Unlearning.
Empirical Evaluation
The evaluation compares naive retraining, unlearning, and amnesiac unlearning across model accuracy and privacy leakage attacks on MNIST and CIFAR100. Unlearning methods remove target-data information faster and more effectively than naive retraining, while amnesiac unlearning is especially suited to small, focused removals.
- Experimental Design: Experiments compare the methods using target and non-target accuracy, model inversion, membership inference, and varying amounts of amnesiac unlearning.The study uses MNIST, CIFAR100, ResNet18, and two state-of-the-art attack methods.
- Model Accuracy and Inversion: Naive retraining retains sensitive information for a long period and performs poorly at removing target-data knowledge.On CIFAR100, approximately 40% target-data prediction accuracy remains after 10 retraining epochs.
- Model Accuracy and Inversion: Unlearning rapidly reduces target-data accuracy and almost immediately removes useful class information from model inversion attacks.On CIFAR100, target-data accuracy becomes very small within 2 epochs, and virtually all recognition ability is removed after 5 epochs.
- Model Accuracy and Inversion: Amnesiac unlearning produces dark, jumbled inversion outputs, with attacks after 5 and 10 epochs becoming almost unrecognizable.The results suggest this method is best suited to removing a small number of data points.
- Membership Inference Attacks: Both unlearning methods protect against membership inference attacks with less than one epoch of retraining, while amnesiac unlearning requires no retraining.Naive retraining does not prevent these attacks for more than 2 full epochs.
- Effect of Amnesiac Unlearning on Model Efficacy: Amnesiac unlearning preserves model accuracy when 1% or less of batches are removed, but efficacy drops quickly as more learned batches are removed.Small amounts of fine-tuning can remedy the accuracy drops; unlearning is recommended for larger removals.
Discussion
The paper cautions that attack-based privacy evaluations do not measure all retained sensitive information, while Figure 7 examines accuracy as amnesiac unlearning increases.
- Test accuracy, model inversion attacks, and membership inference attacks indicate propensity to leak sensitive data but do not measure retained information comprehensively.The paper notes that no known method comprehensively evaluates how much private data a model can leak through any means.
- Black-box access alone is insufficient because attackers can extract model functionality and subsequently attack a stolen model in a white-box setting.
- Figure 7 reports mean test accuracy across all classes over 10 runs as amnesiac unlearning increases.
Conclusion
The paper evaluates unlearning and amnesiac unlearning as efficient ways to remove learned data while protecting privacy and preserving non-target performance. It finds both methods effective against model inversion and membership inference attacks, with unlearning suited to larger removals and amnesiac unlearning to focused removals.
- The paper introduces unlearning and amnesiac unlearning to remove learned data while protecting target-data privacy without significant cost or degradation on non-target data.
- Both removal methods effectively protect data against model inversion and membership inference attacks.
- Unlearning is better for removing large amounts of learned data, whereas amnesiac unlearning is better for laser-focused removal of small data segments.