Source-linked AI summary
A Survey of Machine Unlearning
Thanh Tam Nguyen, Thanh Trung Huynh, Zhao Ren, Phi Le Nguyen, Alan Wee-Chung Liew, Hongzhi Yin, Quoc Viet Hung Nguyen
TL;DR
Machine unlearning is needed because deleting data from databases does not ensure that machine learning models forget it, while existing work lacks common frameworks and resources. This survey synthesizes the field's concepts, scenarios, methods, applications, and research directions, organizing approaches and outlining unresolved challenges. It presents a comprehensive resource and unified taxonomy for understanding and advancing machine unlearning.
Problem
Machine learning models can retain private training information, while existing machine-unlearning research has not completely solved the problem because common frameworks and resources are lacking.
Method
The paper conducts a comprehensive survey and organizes machine-unlearning strategies into model-agnostic, model-intrinsic, and data-driven approaches.
Results
The survey provides a systematic examination of machine-unlearning concepts, scenarios, mechanisms, applications, datasets, implementations, findings, trends, and future research directions.
Takeaways & Limitations
The unified taxonomy and compiled resources are intended to help researchers categorize studies, compare methodologies, and identify unlearning strategies for different applications.
Takeaways & Limitations
No existing unlearning approach satisfies all design requirements, and current algorithms mostly address approximate unlearning and data-item removal.
Abstract
from arXiv · showhide
Today, computer systems hold large amounts of personal data. Yet while such an abundance of data allows breakthroughs in artificial intelligence, and especially machine learning (ML), its existence can be a threat to user privacy, and it can weaken the bonds of trust between humans and AI. Recent regulations now require that, on request, private information about a user must be removed from both computer systems and from ML models, i.e. ``the right to be forgotten''). While removing data from back-end databases should be straightforward, it is not sufficient in the AI context as ML models often `remember' the old data. Contemporary adversarial attacks on trained models have proven that we can learn whether an instance or an attribute belonged to the training data. This phenomenon calls for a new paradigm, namely machine unlearning, to make ML models forget about particular data. It turns out that recent works on machine unlearning have not been able to completely solve the problem due to the lack of common frameworks and resources. Therefore, this paper aspires to present a comprehensive examination of machine unlearning's concepts, scenarios, methods, and applications. Specifically, as a category collection of cutting-edge studies, the intention behind this article is to serve as a comprehensive resource for researchers and practitioners seeking an introduction to machine unlearning and its formulations, design criteria, removal requests, algorithms, and applications. In addition, we aim to highlight the key findings, current trends, and new research areas that have not yet featured the use of machine unlearning but could benefit greatly from it. We hope this survey serves as a valuable resource for ML researchers and those seeking to innovate privacy technologies. Our resources are publicly available at https://github.com/tamlhp/awesome-machine-unlearning.
1 INTRODUCTION
Machine unlearning addresses the problem that deleting personal data from databases does not remove information memorized by machine learning models. This survey organizes the emerging field's frameworks, formulations, scenarios, methods, resources, and open challenges.
- Motivation: Machine learning models can memorize training data, enabling attacks that extract private information despite database deletion.The paper describes models as lossy data-compression mechanisms and notes that adversarial attacks can reveal whether information belonged to training data.
- Motivation: Machine unlearning seeks to remove requested data from a model without complete retraining, supporting the right to be forgotten while reducing retraining costs.The target is model-level deletion rather than only backend data removal.
- Survey scope: The survey responds to incomplete solutions and missing common frameworks and resources by comprehensively examining machine unlearning's definitions, scenarios, mechanisms, and applications.It also makes related resources publicly available.
- Reasons for machine unlearning: The paper covers security, privacy, usability, and fidelity as major reasons users may request data removal.Examples include data-leak risks, incorrect recommendations, and bias associated with learned features or affected data items.
- Challenges in machine unlearning: Machine unlearning faces stochastic training, incremental parameter effects, and potentially exponential performance degradation as more data is unlearned.The survey identifies the latter phenomenon as catastrophic unlearning and describes preventing it naturally as an open question.
- Contributions: The survey proposes a unified framework covering design requirements, removal requests, model verification, formal problem definitions, scenarios, taxonomies, datasets, implementations, and future research directions.Its related-work discussion distinguishes machine unlearning from data deletion and data redaction by emphasizing model-level privacy concerns.
2 UNLEARNING FRAMEWORK
The framework processes removal requests by unlearning specified information, verifying the resulting model, and retraining when privacy criteria are not met. It covers item, feature, class, and task removal while balancing completeness, timeliness, and accuracy.
- Unlearning Workflow: A removal request sends the current model through an unlearning algorithm, followed by verification against privacy criteria.If verification fails, the model is retrained using the remaining data.
- Unlearning Requests: Item removal is common, while feature, class, and task removal address group-level attributes, categories, and continual-learning tasks.Feature and class removal can affect many data items, whereas task removal may create cascading performance degradation across tasks.
- Unlearning Requests: Repeatedly unlearning many affected items is computationally expensive and can reduce model performance, motivating feature- or label-level removal.This concern is especially relevant when suspicious features or labels affect many samples.
- Unlearning Requests: Influence functions can replace iterative optimization with compact parameter updates when unlearning groups of training data.The update is computed from first- and second-order derivatives.
- Unlearning Requests: Feature unlearning can progressively detach requested features from latent and output representations while preserving remaining features.The described disentangled-representation method is mostly applicable to image-domain deep neural networks.
- Unlearning Verification: Verification methods certify unlearning by testing whether the unlearned model is difficult to distinguish from a retrained counterpart, whereas unlearning metrics can guide optimization or guarantees.Verification metrics are used for evaluation only.
3 UNLEARNING DEFINITION
Machine unlearning formalizes removal as making an unlearned model indistinguishable from one trained without the forget set. Exact approaches match model distributions, while approximate approaches reduce computational cost by relaxing this requirement.
- Problem Formulation: An unlearning mechanism receives the training data, forget set, and trained model, then returns a sanitized model comparable to retraining on the remaining data.Learning and unlearning algorithms are modeled as randomized, and the mechanism varies by setting.
- Exact Unlearning: Exact unlearning compares the distribution of models produced by retraining without the forget set with the distribution of unlearned models.This distributional formulation accounts for stochastic learning and unlearning algorithms.
- Exact Unlearning: Exact unlearning can require equality of distributions over model weights or outputs, depending on the chosen model representation and metric space.Outputs are represented through the model’s parameterized mapping, while weights represent model parameters.
- Indistinguishability Metrics: Distributional comparison is expensive because generating distribution samples can require training the whole model, motivating point-based distances in weight or output space.These alternatives seek to reduce the cost of comparing retrained and unlearned models.
- Approximate Unlearning: Approximate unlearning reduces retraining costs by modifying final weights, changing architectures, or filtering outputs while relaxing exact distributional equivalence.Differential privacy implies approximate unlearning, but strong privacy conditions can reduce accuracy.
- Approximate Unlearning: The stated approximate-unlearning bounds apply to a single removed sample, leaving constant guarantees for larger forget sets open.A relaxed formulation additionally permits a failure probability bounded by δ.
4 OTHER UNLEARNING SCENARIOS
Machine unlearning includes restricted-access settings such as zero-glance, zero-shot, and few-shot unlearning, each limiting what data the algorithm can use. These methods approximate forgotten data or rely on model-derived information, creating setting-specific trade-offs and scope limits.
- Restricted-access settings: Zero-glance unlearning uses only retained data after a forgetting request, and smaller retained subsets provide better privacy.The retained dataset is D_r = D \ D_f; the method cannot use forgotten samples even for model-weight manipulation.
- Zero-glance unlearning: Error-maximizing noise damages forgotten-class parameters, then retained-data training repairs performance on retained classes.The procedure trains for one epoch on retained data plus synthetic noise, followed by one repair epoch on retained data.
- Zero-shot unlearning: Zero-shot unlearning has no access to training data, so classification methods approximate forgotten and retained information through synthetic noise.The objective is to make outputs resemble those of a model retrained without the forgotten classes.
- Few-shot unlearning: Few-shot unlearning receives only a small forgotten-data subset, which can address mislabeled or malicious samples but may be undermined by privacy regulations.Typically, D_f_sub is much smaller than D_f; model inversion can retrieve a proxy training set before filtering and relearning.
- Few-shot unlearning: Model-inversion few-shot unlearning can operate without original training data, but it is limited to classification models using cross-entropy loss.The framework retrieves a proxy dataset, filters undesirable examples, interpolates target samples, and relearns.
5 UNLEARNING ALGORITHMS
Machine-unlearning algorithms range from model-agnostic techniques to model-intrinsic and data-driven approaches. They trade computational efficiency, privacy access, theoretical guarantees, and model scope against accuracy and retraining requirements.
- Algorithm categories: Machine-unlearning methods are categorized as model-agnostic, model-intrinsic, or data-driven, with the goal of saving time and computational resources versus full retraining.Model-agnostic methods apply across model types, although formal guarantees may remain restricted to particular classes.
- Model-agnostic approaches: Differentially private publishing and certified removal mechanisms bound model changes or residual information to approximate retraining behavior.Certified removal includes guarantees for linear models and convex losses, as well as computable bounds for SGD-based algorithms and deep networks.
- Model-agnostic approaches: Perturbed gradient-descent extensions support weakly convex losses while providing guarantees on indistinguishability, accuracy, and unlearning time.These extensions include regularized and distributed perturbed gradient descent.
- Model-agnostic approaches: Total-variation stability bounds the transport cost between model distributions trained on datasets sharing one data item, supporting analysis of streaming removal.The bound is expressed through a rho-TV-stability condition.
- Statistical-query methods: Statistical-query learning supports item removal and partial stream removal, but complex models can make query counts exponentially large and reduce efficiency.It supports exact unlearning for non-adaptive queries only partially and can support zero-shot unlearning through aggregate statistics.
- Decremental learning: Decremental learning uses quantization and data partitioning to avoid unnecessary unlearning, but applies when model parameters are few relative to dataset size.The approach was developed for k-means clustering and is designed to prevent catastrophic accuracy degradation from small data changes.
- Model-intrinsic approaches: Teacher-based methods train a student to mimic a competent teacher on retained data and an incompetent teacher on forgotten data.The student is initialized from the competent teacher and optimized using KL-divergence losses to both teachers.
- Data-driven approaches: Mnemonic codes assign class-specific synthetic signals during training, eliminating the need to retain training data for later unlearning.The code strength is controlled by the hyperparameter lambda.
6 PUBLISHED RESOURCES ON MACHINE UNLEARNING
The survey collects implementations, datasets, and evaluation metrics that support machine-unlearning research. These resources cover varied model and data settings, while metrics assess accuracy, forgetting, efficiency, uncertainty, and residual information.
- Published algorithms and models: Table 4 catalogs published implementations with programming language, platform, model, and repository information.The implementations have contributed to baseline experiments in machine unlearning.
- Published datasets: Table 5 organizes widely used datasets into image, tabular, text, sequence, and graph application groups.Dataset details are provided in the survey’s technical report because of space limits.
- Evaluation metrics: Evaluation commonly measures accuracy, completeness, unlearning time, distance, and forgetting scores across forgotten and retained data.Accuracy should be compared with a retrained model, especially on retained data.
- Evaluation metrics: Rapid relearning of forgotten-data performance suggests that information about those data remains in the model.Relearn time acts as a proxy for the amount of forgotten information left behind.
- Evaluation metrics: Lower activation distance and lower JS-divergence indicate closer agreement between unlearned and retrained model behavior.JS-divergence complements activation distance by comparing prediction distributions.
- Evaluation metrics: Membership inference evaluates whether forgotten samples remain detectable, while ZRF compares forgotten-data outputs with those of a randomly initialized model.A successful unlearning process should reduce inference attack probability on forgotten data.
- Evaluation metrics: AIN ranges from 0 to 1, with values closer to 1 indicating better unlearning and low values suggesting rapid relearning.AIN is defined using relearning time relative to an original model and a model trained from scratch.
- Evaluation metrics: Efficacy measures overall information exposure without requiring a retrained model, but it does not isolate forgotten-data information and should accompany accuracy.Lower efficacy is considered better, although lower exposure does not necessarily imply good predictive performance.
7 UNLEARNING APPLICATIONS
Machine unlearning applications span recommender systems, federated learning, graph representation learning, and mitigating catastrophic forgetting. These settings require specialized methods because data dependencies, distributed aggregation, and task interactions complicate removal.
- Unlearning in Recommender Systems: Recommender-system unlearning must preserve user-item similarity information because arbitrary training-set partitioning can break collaborative-filtering coupling.Smart retraining partitions data by user-item resemblance and combines submodels with attention-based aggregation.
- Unlearning in Recommender Systems: Specialized recommender methods include rollback for federated systems, second-order optimization for neural recommenders, multimodal unlearning, and attribute-wise privacy removal.The cited approaches target efficient erasure, heterogeneous data, legal risks, and sensitive user attributes.
- Unlearning in Federated Learning: Federated unlearning is difficult because global weights mix client contributions, clients may share data, and gradient manipulation can degrade accuracy or create privacy threats.Federated training aggregates client updates rather than exposing raw gradients, complicating attribution of individual training items.
- Unlearning in Federated Learning: Current federated-unlearning studies commonly assume that forgotten data belongs entirely to one client, while historical-update storage creates scalability–precision trade-offs.Calibration training is reported to work better for shallow architectures than deep neural networks, and server-side knowledge distillation may require synthetic unlabeled data.
- Unlearning in Graph Representation Learning: Graph unlearning is challenging because correlated nodes and edges can affect neighbors and multi-hop structures, while nonuniform components can produce imbalanced partitions.Proposed responses include preference-based graph partitioning, one-shot embedding unlearning, and GNNDELETE’s deleted-edge consistency and neighborhood-influence components.
- Unlearning for Model Repair: Machine unlearning can also counter catastrophic forgetting by removing obsolete or redundant data, and may improve accuracy through information-bottleneck-like compression.The cited discussion presents these as applications beyond privacy-driven deletion.
8 DISCUSSION AND FUTURE PROSPECTS
The survey identifies influence-based unlearning, reachability concerns, and independent verification as central discussion points, while outlining open needs for unified requirements, benchmarks, and causality-aware methods. It also notes applications to bias removal and model repair.
- Summary and Trends: Influence functions estimate how individual data items affect model parameters or performance, enabling faster unlearning by reversing associated updates.The survey notes that the resulting offset may be bounded in some methods.
- Summary and Trends: Parameter-space similarity alone may not establish forgetting because the same parameters can be reachable with or without a data item, allowing residual effects.The survey therefore calls for independent unlearning verification or data auditing.
- Summary and Trends: Machine unlearning can remove biased features while maintaining model quality and can repair overtrained networks by unlearning useless, obsolete, or redundant samples.The survey also relates forgetting to compression in information bottleneck theory.
- Open Research Questions: Future systems need unified design requirements covering scenarios, removal requests, completeness, timeliness, and accuracy, alongside common benchmarks and more public source code.Existing approaches often focus on approximate unlearning and data-item removal, while studies target differing algorithms or applications.
- Open Research Questions: Concept drift requires contradiction analysis between old and new data, adding a condition for unlearning methods that address changing data distributions.The cited passage identifies this analysis as necessary in the discussed setting.
- Open Research Questions: Causality analysis could help unlearn large-scale pollution when only a small subset of affected data is detected, supporting guarantees that polluted information is absent from the final model.The survey frames this as a future research direction for data pollution attacks.
9 CONCLUSION
The survey systematically examines machine unlearning and proposes a unified taxonomy covering model-agnostic, model-intrinsic, and data-driven strategies. It summarizes and compares methods to support future categorization and application-specific strategy selection.
- Conclusion: The survey presents a unified taxonomy dividing machine-unlearning strategies into model-agnostic, model-intrinsic, and data-driven approaches.It is intended to help categorize future studies and provide insight into methodologies.
- Conclusion: The survey offers summaries and comparisons of unlearning methodologies, providing a comprehensive view of current work and the machine-unlearning process.The stated goal is to assist researchers in identifying strategies for different applications.