PaperlayerView exact revision on arXiv

Source-linked AI summary

Machine Unlearning: A Survey

Heng Xu, Tianqing Zhu, Lefeng Zhang, Wanlei Zhou, Philip S. Yu

arXiv:2306.03558v1cs.CRcs.LG

TL;DR

Privacy, usability, and deletion requests motivate removing samples and their influence from trained models, including in federated settings with constraints such as unavailable training data. This survey classifies unlearning methods and synthesizes their strengths and limitations, including trade-offs between storage, applicability, guarantees, and accuracy.

  • Problem

    Requests to delete training samples and their influence arise from privacy and entitlement concerns, while federated settings add constraints such as unavailable training data and unstable connections.

    Source: §1 · p. 1, §6.2 · p. 29

  • Method

    The survey develops a taxonomy and reviews machine unlearning techniques, focusing on data reorganization and model manipulation, their principles, and verification approaches.

    Source: §5.2.2 · p. 23, §7 · p. 30

  • Results

    Data reorganization can support varied requests and models with approximate retraining-indistinguishability, but segmentation trades faster unlearning for storage and may reduce accuracy; model manipulation uses less intermediate storage.

    Source: §4.4 · p. 17, §5.4 · p. 25

  • Takeaways & Limitations

    Efficient unlearning mechanisms that preserve validity without adding too much storage remain a worthwhile research direction.

    Source: §4.4 · p. 17

    Data reorganization can reduce model consistency and accuracy as unlearning continues because each sub-model is trained on only part of the dataset.

    Source: §4.4 · p. 17

Abstract

from arXiv · show

Machine learning has attracted widespread attention and evolved into an enabling technology for a wide range of highly successful applications, such as intelligent computer vision, speech recognition, medical diagnosis, and more. Yet a special need has arisen where, due to privacy, usability, and/or the right to be forgotten, information about some specific samples needs to be removed from a model, called machine unlearning. This emerging technology has drawn significant interest from both academics and industry due to its innovation and practicality. At the same time, this ambitious problem has led to numerous research efforts aimed at confronting its challenges. To the best of our knowledge, no study has analyzed this complex topic or compared the feasibility of existing unlearning solutions in different kinds of scenarios. Accordingly, with this survey, we aim to capture the key concepts of unlearning techniques. The existing solutions are classified and summarized based on their characteristics within an up-to-date and comprehensive review of each category's advantages and limitations. The survey concludes by highlighting some of the outstanding issues with unlearning techniques, along with some feasible directions for new research opportunities.

1 INTRODUCTION

Machine unlearning addresses requests to remove specific samples and their influence from both training data and trained models. This survey categorizes existing approaches, compares their characteristics and verification strategies, and identifies open issues and research directions.

1.1 The Motivation of machine unlearning

  • Privacy, legal, and usability demands motivate removing specific samples and their influence from both training data and trained models.Retraining from scratch can remove model information, but may be costly or impossible when the training data are inaccessible.Source: §1 · pp. 1–2, §1.1 · pp. 2–3
  • Unlike related techniques with different objectives, machine unlearning removes user-specific training samples or reverses updates; retraining may be costly or infeasible when data are inaccessible.Source: §1.1 · pp. 2–3

1.2 Contributions of this survey

  • The survey proposes a taxonomy based on unlearning rationale and strategy, and summarizes methods, verification approaches, advantages, shortcomings, and open research directions.Source: §1.2 · p. 3

1.3 Comparison to existing surveys in machine unlearning

  • Compared with prior surveys, this work analyzes verification alongside unlearning schemes and compares methods across data needs, storage, supported requests, models, and category-level shortcomings.It also discusses trends and potential solutions not fully covered by the surveyed works [35] [37].Source: §1.3 · pp. 3–4

2 PRELIMINARIES

Machine unlearning seeks to remove selected samples’ influence from a trained model, with the result judged against retraining on the retained data. The preliminaries distinguish unlearning targets and set out consistency, accuracy, and verifiability as desiderata.

2.1 Definition of Machine Unlearning

  • Machine unlearning maps a trained model, its training data, and the requested removal set to a model that performs as though it had never seen those samples.The retained dataset is the original training set minus the unlearning set.Source: §2.1 · p. 5

2.2 Targets of Machine Unlearning

  • Native retraining on the retained data is the performance baseline, but it can be costly and impossible when training data are inaccessible.The survey identifies federated learning as an example of an inaccessible-data setting.Source: §2.2 · p. 6
  • Exact unlearning makes the unlearned and retrained model distributions indistinguishable, while approximate unlearning allows a tolerable difference.Approximate unlearning is described as suitable for complex models, whereas exact unlearning is limited to simple, well-structured models.Source: §2.2 · pp. 6–7
  • Strong approximate unlearning compares internal parameter distributions, while weak approximate unlearning compares final activation distributions.Approximate unlearning is usually guaranteed using differential privacy techniques.Source: §2.2 · p. 7

2.3 Desiderata of Machine Unlearning

  • Consistency compares unlearned-model behavior with retrained-model behavior, and accuracy measures correct predictions on retained samples.The survey treats accuracy as necessary for the usability of the model after unlearning.Source: §2.3 · pp. 7–8
  • Verifiability means a verification function can distinguish the original model from the unlearned model after the removal process.A backdoor check is given as an example: the injected backdoor should be detected in the original model but not the unlearned model.Source: §2.3 · p. 8

3 TAXONOMY OF UNLEARNING AND VERIFICATION MECHANISMS

The survey classifies unlearning by whether methods reorganize training data or manipulate model weights, and groups verification methods by their evaluation approach. It describes empirical and theoretical verification strategies alongside challenges in establishing that unlearning succeeded.

3 TAXONOMY OF UNLEARNING AND VERIFICATION MECHANISMS

  • The taxonomy separates data reorganization, which modifies training data, from model manipulation, which directly adjusts trained-model weights.Verification schemes are first divided into experimental or theoretical methods, then grouped by the metrics they use.Source: §3 · p. 8

3.1.1 Data Reorganization.

  • Data reorganization uses obfuscation, pruning, or replacement to prepare training data for unlearning.Obfuscation adds choreographed data and fine-tunes; pruning retrains affected submodels after partitioning; replacement uses transformed data that can be updated after deletion.Source: §3.1.1 · pp. 8–9

3.1.2 Model Manipulation.

  • Model manipulation shifts parameters, replaces selected parameters with precomputed ones, or prunes parameters, often under model-specific assumptions or structures.The survey notes that accurately estimating sample influence can be difficult for complex neural models, while pruning is generally followed by fine-tuning.Source: §3.1.2 · pp. 9–10

3.2 Verification Mechanisms

  • Unlearning is difficult to verify because providers may claim removal without achieving it, while small deletions or similar samples can make output-based checks inconclusive.These difficulties are especially pronounced for complex deep models and large training datasets.Source: §3.2 · pp. 10–11

3.2.1 Empirical evaluation. •

  • Empirical verification includes retraining, privacy attacks, relearning time, and output accuracy comparisons against a model trained without the deleted data.The survey also notes accuracy recovery after an attack as a possible verification signal.Source: §3.2.1 · pp. 11–12

3.2.2 Theoretical calculation. •

  • Theoretical verification uses certified definitions to require indistinguishability from retraining, or bounds residual information about samples marked for deletion.The survey presents less residual information as indicating a more effective unlearning operation.Source: §3.2.2 · p. 12

4 DATA REORGANIZATION

Data-obfuscation schemes alter data or model behavior to reduce the influence of selected samples, but the model may retain sample information and generating noise can be time-consuming. The survey describes attack-based and relearning-based approaches to checking these schemes.

4.1.1 Unlearning Schemes Based on Data Obfuscation.

  • Randomly incorrect relabeling or poisoned labels followed by fine-tuning are used to confuse the relationship between samples and model outputs, but parameters may still retain sample information.Graves et al. [45] and Felps et al. [46] use these approaches; Tarrun et al. [27] instead learn error-maximizing noise and repair performance using remaining data.Source: §4.1.1 · p. 12
  • Generative noisy data can adjust retrieval-model weights while preserving other data contributions, but generating noise for unlearning can take additional time.Zhang et al. describe static and dynamic learning branches for limiting the noisy data's effect to the samples being forgotten.Source: §4.1.1 · pp. 12–13

4.1.2 Verifiability of Schemes Based on Data Obfuscation.

  • Verification examples use model inversion and membership-inference attacks to assess retained information, or measure relearning time and distances among original, unlearned, and retrained models.Graves et al. [45] use the attacks; Tarrun et al. [27] evaluate relearning epochs and model distances.Source: §4.1.2 · p. 13

4.2 Reorganization Based on Data Pruning

Data-pruning approaches partition training data and retrain affected submodels after deletion, limiting retraining to relevant shards. The survey discusses performance and storage trade-offs, graph-specific methods, and verification strategies.

4.2.1 Unlearning Schemes Based on Data Pruning.

  • SISA partitions data into disjoint shards, trains a submodel per shard, aggregates their predictions, and retrains only submodels containing a deleted sample.The retraining cost depends on shard size; smaller shards lower that cost but leave less training data per submodel.Source: §4.2.1 · p. 13
  • As unlearning data increases, SISA performance degrades, restricting its suitability to small-scale settings; smaller shards also risk lowering ensemble accuracy, and cached intermediate results consume storage.The survey also notes that schemes may require substantial cached results.Source: §4.2.1 · pp. 13–14
  • Naively partitioning graph data into subgraphs can damage graph structure and model utility, so GraphEraser uses graph partitioning, shard-model training, and shard-model aggregation for graph unlearning.Its balanced label-propagation and embedding k-means strategies aim to avoid highly unbalanced shard sizes.Source: §4.2.1 · p. 14
  • Other pruning approaches address adaptive deletion requests, reuse intermediate models, or adapt partitioning and aggregation for recommendation and patient-similarity tasks.Neel et al. [50] use update-based unlearning and a secret state; He et al. [48] stitch models by reusing unaffected areas but provide no theoretical verification.Source: §4.2.1 · pp. 14–15

4.2.2 Verifiability of Schemes Based on Data Pruning.

  • Affected-submodel retraining provides a verification guarantee for schemes, while He et al. [48] use a backdoor trigger and studies [49] [50] give no verification method.The backdoor approach treats a lower attack success rate as evidence that unlearning removed the data.Source: §4.2.2 · pp. 15–16

4.3 Reorganization Based on Data Replacement

Data replacement transforms training data so that only transformations affected by a deletion need updating and the model can then be recomputed. The survey describes accuracy-based verification and notes that the summation-based approach applies to only a limited set of learning algorithms.

4.3.1 Unlearning Schemes Based on Data Replacement.

  • Cao et al. transform training data into efficiently computable summations, update only those affected by a deleted sample, and recompute the model.Because the approach relies on statistical-query learning, its applicability is limited to a few algorithms, including naïve Bayes, support vector machines, and k-means.Source: §4.3.1 · p. 16

4.3.2 Verifiability of Schemes Based on Data Replacement.

  • Cao et al. verify unlearning using attack outcomes or recovery of pre-pollution model performance, while Takashi et al. propose an accuracy-based measure, LSFM.For the LensKit model, the cited verification uses a system inference attack; other models are checked for restoration to their pre-pollution performance.Source: §4.3.2 · p. 17

4.4 Summary of Data Reorganization

Data-reorganization methods can support varied unlearning requests and models, with many approximating retraining, but segmentation trades faster unlearning for storage and can reduce accuracy. The survey also identifies limited theoretical validation and challenges in balancing unlearning effectiveness with model usability.

4.4 Summary of Data Reorganization

  • Most surveyed algorithms retain intermediate parameters and the original dataset, segment the data, and retrain affected sub-models, reducing unlearning time at the cost of storage.Table 6 summarizes differences among surveyed studies using data reorganization techniques.Source: §4.4 · p. 17, Table 6 · p. 18
  • Data reorganization supports requests from samples to classes or sequences and models from support vector machines to deep neural networks, often with unlearned-model distributions approximately indistinguishable from retraining.These methods modify the training-data distribution rather than operating directly on the model, supporting application to complex models.Source: §4.4 · p. 17
  • Data reorganization can reduce model consistency and accuracy as unlearning continues because each sub-model trains on only part of the dataset.The survey notes that no-isolation unlearning and data replication may address this accuracy issue.Source: §4.4 · p. 17
  • Most schemes are validated experimentally, while theoretical support for their validity remains lacking and further theoretical research is needed.The proposed theory could quantify attacker access to sensitive information or similarity between unlearned and retrained models.Source: §4.4 · p. 17
  • Obfuscation may fail to remove information from complex models, pruning can affect accuracy, and replacement cannot reproduce all original-dataset information; the survey calls for balancing effectiveness and usability.The survey identifies these as challenges for the respective data-reorganization approaches.Source: §4.4 · pp. 17, 19

5 MODEL MANIPULATION

The section reviews methods that unlearn by directly manipulating a model, including discussion of verification techniques within each category.

5 MODEL MANIPULATION

  • Model manipulation targets learned input-output relationships by directly changing a model to remove specific relationships; the section reviews these methods and their verification techniques.Source: §5 · p. 19

5.1 Manipulation Based on Model Shifting

Model-shifting methods update model parameters to remove the influence of unlearning data, using influence-based, Fisher-based, and other approaches. The survey also reviews how these schemes are verified, while noting limits in applicability and theoretical guarantees.

5.1.1 Unlearning Schemes Based on Model Shifting.

  • Influence unlearning includes certified removal for L2-regularized linear models with differentiable convex loss, while projection residual update reduces linear-regression runtime from quadratic to linear complexity.Certified removal bounds the difference between unlearned and retrained models and perturbs training loss; these approaches remain focused on simple models or linear decision layers.Source: §5.1.1 · p. 19
  • Shifting methods also address feature and label removal, federated updates, and unseen-sample loss, but their applicability can be restricted by data type, training assumptions, or model complexity.Warnecke et al. [53] limit their feature-unlearning solution to tabular data; Liu et al. report significant accuracy reduction on complex models; the mixed-privacy scheme [52] restricts requests to data that are easy to unlearn.Source: §5.1.1 · pp. 20–21
  • Fisher-based weight scrubbing targets a class or subset of samples, using Newton or noisy Newton updates to unlearn a cohort while maintaining good accuracy on remaining samples, subject to assumptions.The approach bounds retained information using KL-divergence and Shannon mutual information.Source: §5.1.1 · p. 20
  • Deep-network methods may bound information in final activations rather than weights, but a white-box attacker could still infer information from middle layers [43].The survey notes that equivalent deep-network solutions can produce the same test-sample activations.Source: §5.1.1 · p. 20
  • Other shifting schemes include decremental updates for collaborative filtering, ridge regression, and k-nearest neighbors, plus amnesiac unlearning that reverses updates from batches containing requested data.Amnesiac unlearning may require fine-tuning to recover performance and trades cached-data storage against unlearning efficiency.Source: §5.1.1 · p. 21

5.1.2 Verifiability of Schemes Based on Parameter Shifting.

  • Verification includes retraining similarity, attack-based and accuracy-based tests, confidence and information-bound metrics, activation distance, and theoretical distribution-similarity guarantees.Golatkar et al. [40] [43] [52] use confidence, information-bound, activation-distance, and attack-based checks; Guo et al. [41], Warnecke et al. [53], and Sekhari et al. [42] provide theoretical verification.Source: §5.1.2 · pp. 21–22

5.2 Manipulation Based on Model Pruning

Model-pruning approaches produce unlearned models by pruning trained models, often in federated learning. The survey describes cached-update and channel-pruning methods, alongside verification approaches and limitations.

5.2.1 Unlearning Schemes Based on Model Pruning.

  • FedEraser stores clients’ historical parameter updates and uses calibration training, update calibration, aggregation, and model updating to remove a target client’s influence.Its effectiveness decreases as requests accumulate because cached gradients are not updated for subsequent requests, and caching also increases storage needs.Source: §5.2.1 · p. 22
  • Federated channel pruning uses TF-IDF scores to identify channels discriminative for target classes, then prunes and fine-tunes the model.Pruning also makes parameters associated with other classes incomplete, affecting performance; the model is available only after fine-tuning finishes.Source: §5.2.1 · pp. 22–23, Figure 9 · p. 23
  • Linear-filtration unlearning filters outputs of a logistic-regression layer using naive unlearning, normalization, randomization, or zeroing, but middle-layer parameters may retain information accessible to an attacker.Source: §5.2.1 · p. 23

5.2.2 Verifiability of Schemes based on Model Pruning.

  • Pruning-scheme verification includes membership-inference attack precision and recall, prediction differences, model accuracy, Bayes-error divergence between unlearned and retrained outputs, and model-inversion attacks.The Bayes-error measure indicates similar output distributions when its value is close to zero.Source: §5.2.2 · p. 23

5.3 Manipulation Based on Model Replacement

Model-replacement approaches prepare substitute model components in advance, while other surveyed methods adjust model outputs or approximate retraining. Verification remains limited: only one described approach is assessed with an attack and false-negative rates, and three others lack guarantees.

5.3.1 Unlearning Schemes Based on Model Replacement.

  • Schelter et al.’s Hedgecut preserves robust tree splits, updates leaf statistics, and selects from maintained subtree variants when non-robust splits change.The variants are kept inactive during training and their split criteria are recomputed after a sample-removal request.Source: §5.3.1 · p. 24
  • Brophy et al.’s DaRE forests [58] precompute thresholds and node statistics for selective subtree retraining, while random top nodes reduce the need to retrain influential splits.Stored intermediate statistics allow split criteria to be recomputed without iterating through the data.Source: §5.3.1 · p. 24
  • Replacement methods store precomputed submodels for affected components, but their storage demands and reliance on advance enumeration limit use beyond models such as trees.The survey notes that constructing partial structures in advance is difficult for deep-learning models.Source: §5.3.1 · pp. 23–24
  • WGAN unlearning [59] alternates generator and discriminator training until the discriminator cannot distinguish outputs for unlearning data from third-party data, yielding only limited efficiency improvement over retraining from scratch.The generator is initialized as the trained model and becomes the final unlearned model.Source: §5.3.1 · pp. 24–25
  • DeltaGrad [60] uses cached intermediate parameters, exact gradients on remaining data, and L-BFGS-based Quasi-Hessians to approximate later retraining updates, with possible accuracy and efficiency costs.The survey reports that approximation can reduce model performance and increase the iterations needed for convergence.Source: §5.3.1 · p. 25

5.3.2 Verifiability of Schemes Based on Model Replacement.

  • Chen et al. [59] assess WGAN unlearning with membership inference and false-negative rates, whereas Hedgecut, DaRE, and DeltaGrad report runtime and accuracy without theoretical or experimental verifiability guarantees.A higher false-negative rate is described as indirect evidence that the attack treats training samples as non-training data.Source: §5.3.2 · p. 25

5.4 Summary of Model Manipulation

The survey contrasts model-manipulation approaches with data-reorganization schemes and identifies trade-offs in storage, applicability, accuracy, and verification. It highlights unresolved challenges for deep models and the limits of current verification approaches.

5.4 Summary of Model Manipulation

  • Model-manipulation methods use little intermediate data compared with data-reorganization schemes, which can reduce intermediate-storage resource consumption.The survey attributes this difference to manipulating the model rather than reorganizing the training dataset.Source: §5.4 · p. 25
  • Evaluating sample influence on deep models remains difficult because their non-convex losses complicate analysis, and unlearning standard convolutional networks remains open.The survey says existing work mainly targets simpler convex problems and some preliminary deep-learning results apply only to small-scale settings.Source: §5.4 · pp. 25, 27
  • Model manipulation can reduce model consistency or prediction accuracy when influence estimates are unreliable or parameter pruning removes predictive information.The survey calls for mechanisms that preserve performance while ensuring effective unlearning.Source: §5.4 · p. 27
  • Theory-based and information-bound verification can be more convincing than accuracy-, relearning-, or attack-based checks, but more theory-based evaluation techniques are needed.The survey notes that output confidence may hide unlearned-sample effects even when information remains detectable in model weights.Source: §5.4 · p. 27
  • Model-shifting methods make assumptions about training, pruning can affect performance, and replacement requires advance calculation and storage of possible parameters.The survey identifies balancing usability, storage costs, and applicability as an urgent research problem.Source: §5.4 · p. 27

6 OPEN QUESTIONS AND FUTURE DIRECTIONS

The survey identifies gaps in the universality, security, and user accessibility of machine-unlearning solutions, alongside emerging applications. It proposes future research directions spanning system operation, federated learning, disturbance, feature-based influence estimates, and privacy–utility trade-offs.

6.1.1 The universality of unlearning solutions.

  • Current unlearning schemes are often tailored to particular models or training processes, while universal methods should cover varied model structures, training methods, and data types.Data-pruning approaches may break correlations needed by some models.Source: §6.1.1 · pp. 27–28

6.1.2 The security of machine unlearning.

  • Unlearning can increase privacy risk because before-and-after model comparisons may expose details of removed samples, while cached intermediate information also needs security analysis.The survey describes membership-inference and poisoning attacks as relevant threats.Source: §6.1.2 · p. 28

6.1.3 The verification of machine unlearning.

  • Simple verification metrics may provide weak evidence, theoretical guarantees often rely on assumptions invalid for complex models, and existing schemes can be difficult for ordinary users.The survey calls for understandable checks that users can carry out through simple operations.Source: §6.1.3 · p. 28

6.1.4 The applications of machine unlearning.

  • Potential applications include accelerating leave-one-out cross-validation, removing adversarial or poisoning samples, identifying valuable data, and addressing fairness through feature unlearning.The survey also warns that unlearning requests could be used to affect model performance, fairness, or unlearning efficiency.Source: §6.1.4 · pp. 28–29

6.2 Future Directions

  • Future work should address prediction requests during delayed unlearning and efficient removal in federated settings with inaccessible data and unstable connections.Rejecting predictions during the update interval may harm utility and service standards, while serving stale models may reveal unlearned data.Source: §6.2 · p. 29
  • Data disturbance during training or parameter adjustment is proposed as a direction for reducing privacy leaks associated with differences between pre- and post-unlearning models.The survey cites loss perturbation as an example used to hide information about unlearned samples.Source: §6.2 · p. 29
  • Feature-based unlearning could estimate sample influence from selected features rather than directly calculating influence from original training samples.The survey suggests feature extraction, generation, and selection as potentially relevant technologies.Source: §6.2 · p. 29
  • A game-theoretic formulation could model the provider’s trade-off between model performance and user privacy, including over-unlearning and insufficient protection.The survey presents this as an open question for future research.Source: §6.2 · pp. 29–30

7 CONCLUSION

The survey presents a taxonomy and overview of machine-unlearning techniques, reviewing their strengths, limitations, and verification approaches. It concludes with open research issues and identifies federated learning with verifiability as a focus for future work.

7 CONCLUSION

  • The survey organizes machine-unlearning techniques around data reorganization and model manipulation, proposing a taxonomy of their principles and targets.It also reviews the strengths and limitations of existing studies within each category.Source: §7 · p. 30
  • The survey emphasizes verification of unlearning processes and reviews different approaches for assessing them.Source: §7 · p. 30
  • The authors identify unresolved issues and future research directions, with future work focused on federated learning with a verifiability property.Source: §7 · p. 30

Related papers

Loading 2306.03558v1…