A Survey of Machine Unlearning

1 INTRODUCTION

• Because models can retain training information and expose it through attacks, machine unlearning [4] aims to remove requested data without full retraining.Regulations establish a right to have data deleted, but model parameters can obscure a data item’s influence, making targeted removal difficult.Source: §1 · p. 1

1.1 Reasons for Machine Unlearning

• The survey groups unlearning requests into security, privacy, usability, and fidelity motivations, including adversarial data, privacy risks, unwanted recommendations, and biased outputs.Bias may originate in training data, motivating removal of affected data items or features.Source: §1.1 · p. 2

1.2 Challenges in Machine Unlearning

• Unlearning must address stochastic training, incremental effects across samples, and catastrophic performance degradation, which can worsen exponentially as more data is unlearned.Special loss functions have been explored to mitigate catastrophic unlearning [60], but preventing it naturally remains open.Source: §1.2 · pp. 2–3

1.3 Contributions of this survey

• The survey contributes a framework and formulations for unlearning, covers scenarios and a three-branch taxonomy, and compiles datasets, implementations, findings, trends, and research directions.Its framework covers design requirements, request types, and verification; its formulations distinguish exact from approximate unlearning and define indistinguishability metrics for comparing unlearned and retrained models.Source: §1.3 · p. 3

1.4 Differences between this and previous surveys

• Unlike data deletion, machine unlearning addresses privacy from the model perspective, while data redaction [45] requires forgotten data to be known during original training.The survey also compares its scope with prior machine-unlearning surveys and related work in Table 1.Source: §1.4 · p. 3

2.1 Unlearning Workflow

• After a removal request, an unlearning algorithm processes the current model; the result is audited for privacy leaks and evaluated on performance before deployment or retraining.If verification fails, the model is retrained on the remaining data; the unlearning component is invoked again for subsequent requests.Source: §2.1 · p. 3

2.2 Unlearning Requests

• Item removal is the most common request, while feature-level removal addresses correlated groups that may require repeated retraining and harm model performance.Influence functions provide a compact update for group removal [196]; disentangled representations can progressively detach features while preserving others, chiefly for image-domain deep networks [66].Source: §2.2 · pp. 3–4
• Class removal may require repeated passes and is difficult when class information is implicit; task removal risks degrading other tasks, while stream removal handles sequential requests.Data augmentation can remove a class without its samples, followed by repair training [172], and filtration operators offer a class-specific alternative [4]. Task removal may require declaring tasks permanent or temporary [101]; stream requests may be adaptive or non-adaptive [67].Source: §2.2 · pp. 4–5

2.3 Design Requirements

• Design requirements include prediction consistency with retraining, timely and accurate unlearning, low overhead, provable guarantees, model-agnostic applicability, and user-verifiable privacy.Completeness and timeliness can trade off: repeated unlearning of large forget sets may degrade accuracy, making retraining preferable in some cases.Source: §2.3 · p. 5

2.4 Unlearning Verification

• Verification aims to certify that an unlearned model is hard to distinguish from its retrained counterpart; verification metrics evaluate models, whereas unlearning metrics can guide optimization or guarantees.Feature injection tests track a feature-associated weight before and after unlearning, but are limited to linear and logistic models; privacy attacks can measure residual traces, including whether a sample is still inferable [80].Source: §2.4 · pp. 5–6
• Verification approaches include membership-inference and backdoor tests, output-space interclass confusion tests, federated client checks, and cryptographic proofs; poisoning can also target unlearning efficiency.Membership and backdoor attacks can reveal residual data leakage, though backdoor checks are not infallible [23] [166]. Cryptographic proofs can attest to model updates and prevent forgotten items from being re-added [44].Source: §2.4 · pp. 6–7

3.1 Problem Formulation

• An unlearning mechanism U takes the training data, a forget set, and the trained model, and returns a sanitized model expected to match or resemble retraining without the forgotten data.Learning and unlearning algorithms are randomized, so their outputs are modeled as distributions. Requests may be motivated by privacy, security, or usability; database deletion alone does not remove data’s influence on a model.Source: §3.1 · p. 7

3.2 Exact Unlearning (Perfect Unlearning)

• Exact unlearning compares distributions of unlearned models with models trained on the remaining data, with equality defined over either model weights or outputs.Retraining guarantees equality but is computationally costly and cannot handle batched removal requests; output-distribution equality is also called weak unlearning [4].Source: §3.2 · pp. 7–8

3.3 Approximate Unlearning (Bounded/Certified Unlearning)

• Approximate unlearning relaxes exactness with bounded or certified removal, but whether constant bounds extend from one removed sample to larger subsets remains open.Differential privacy implies approximate unlearning, but the paper describes it as a very strong condition that can significantly reduce accuracy even for large 𝜖 [1].Source: §3.3 · p. 8

3.4 Indistinguishability Metrics

• Metrics include ℓ2 verification error for weights and KL or Jensen-Shannon divergence between model distributions [60], but each requires costly or nontrivial comparison.ℓ2 verification requires naive retraining and faces threshold ambiguity from training stochasticity; divergence requires modeling or sampling distributions of retrained models.Source: §3.4 · p. 8
• Weight-leakage metrics use gradients to assess whether a linear classifier was trained without a removed sample [64], but their assumptions may fail for nonlinear models.They also require access to the remaining data and loss function, which may not be available.Source: §3.4 · p. 9

4.1 Zero-glance Unlearning

• In zero-glance unlearning, Tarun et al. [172] use retained data and loss-maximizing noise to damage forgotten-class parameters, then train on retained data to repair retained-class performance.The method uses a subset of retained data, with a smaller subset providing better privacy; more epochs or a larger subset can improve performance while trading accuracy against time.Source: §4.1 · p. 9

4.2 Zero-shot Unlearning

• In zero-shot unlearning, Chundawat et al. [29] cannot access forgotten data and use retained classes to make unlearned-model outputs resemble those of a retrained model.Their error-maximizing and error-minimizing noise approach yields poor outcomes, so they also propose gated knowledge transfer that blocks forgotten-class knowledge from passing from teacher to student.Source: §4.2 · p. 9

4.3 Few-shot Unlearning

• Few-shot unlearning receives only a small subset of forgotten data; Yoon et al. [207] use model inversion to retrieve, filter, and relearn proxy data.The setting can address mislabeled data or malicious effects, but access to forgotten data may be restricted and the subset is often much smaller than the full forgotten set; the framework also applies without original training data but only to classification models with cross-entropy loss.Source: §4.3 · pp. 9–10
• For few-shot unlearning, Peste et al. approximate costly Hessian-based influence calculations with an empirical Fisher Information Matrix and rank-one updates for fast inversion.Because only the removal data is needed, samples can be erased through closed-form updates.Source: §4.3 · p. 10

5 UNLEARNING ALGORITHMS

• Model-agnostic, model-intrinsic, and data-driven approaches form the survey’s three categories for unlearning without retraining from scratch.These approaches aim to save time and computational resources [21].Source: §5 · p. 10

5.1 Model-Agnostic Approaches

• Differential privacy and certified removal offer model-agnostic routes, but their guarantees and applicability depend on the method and model.Setting 𝜖=0 prevents learning from training data; certified removal perturbs updates, and may fall back to retraining when approximation error exceeds a threshold. Extensions address weakly convex losses and guarantees for indistinguishability, accuracy, and unlearning time.Source: §5.1 · p. 10
• Statistical-query unlearning recomputes statistics on retained data, while decremental learning prioritizes accuracy over complete forgetting.Statistical queries can support adaptive methods and cached sums, but may become exponentially numerous for complex models; exact unlearning is only partially supported for non-adaptive queries, and streaming updates may be unbounded. Decremental methods are limited by the number of model parameters or features relative to dataset size [57].Source: §5.1 · p. 11
• Knowledge adaptation distills retained and forgotten data through separate teachers, whereas MCMC sampling can unlearn without access to the forgetting set.Knowledge adaptation applies across unlearning requests but lacks an easy theoretical guarantee [28]. MCMC is motivated when the forgetting set is small relative to training data.Source: §5.1 · pp. 11–12

5.2 Model-Intrinsic Approaches

• Model-intrinsic methods specialize to model types, including class removal for softmax classifiers, small-group removal for linear models, and robust-split updates for trees.Tree methods store subtree variants for non-robust splits, which can increase storage costs when many items are forgotten; Bayesian approaches approximate the retrained posterior by minimizing KL divergence when direct sampling is unavailable, helping avoid catastrophic unlearning and preserve accuracy.Source: §5.2 · pp. 12–13
• Deep-network unlearning uses layer-specific methods and influence approximations, but large forget sets destabilize influence functions and Fisher-based methods lack complete-removal guarantees.Fisher-based approaches approximate Hessian computation and inject noise to mitigate leaks, potentially affecting performance [60].Source: §5.2 · p. 13

5.3 Data-Driven Approaches

• Data partitioning speeds retraining by restricting updates to affected shards, while augmentation-based noise can prevent learning before training or impair predictions on a target class.SISA partitions data into shards and slices and stores checkpoints; error-minimizing noise and Fawkes target particular data items before training, whereas error-maximizing noise targets a class rather than an individual item [172].Source: §5.3 · pp. 13–14
• Data-influence methods subtract stored training updates or approximate parameter effects, but may degrade accuracy and do not verify removal; mnemonic codes enable one-shot unlearning without retaining training data.Influence-based methods have no concrete guarantee in the described Fisher-based variant. The mnemonic-code approach adds noise to sensitive parameters estimated with the Fisher information matrix and uses class-specific synthetic signals, reducing storage needs [206].Source: §5.3 · p. 14

6.1 Published Unlearning Algorithms

• Published algorithm and model implementations are cataloged with their programming languages, platforms, and repository links.Table 4 summarizes implementations that have contributed to baseline experiments.Source: §6.1 · p. 14

6.2 Published Datasets

• Published machine-unlearning datasets are grouped by application area into image, tabular, text, sequence, and graph categories.Further dataset details are provided in the technical report [130].Source: §6.2 · p. 14

6.3 Evaluation Metrics

• Evaluation covers accuracy on forgotten, retained, and test data, completeness against retraining, and unlearning and retraining time.Relearning time indicates how much forgotten information may remain; layer-wise weight differences should resemble those of a retrained model [172].Source: §6.3 · pp. 14–15
• Shorter activation distance and lower JS-divergence between unlearned and retrained models indicate better unlearning [92].Layer-wise distance should be comparable to retraining: too little may indicate ineffective unlearning, while much more may signal information leaks or a Streisand effect.Source: §6.3 · p. 15
• Membership-inference attack probability should fall for forgotten samples [23], while ZRF approaches 1 when forgotten-sample predictions are random [28].AIN is closer to 1 for better unlearning; values well below 1 can indicate retained information or rapid relearning, while values much greater than 1 may indicate detectable parameter changes.Source: §6.3 · pp. 15–16
• The efficacy score is computationally efficient and needs no retrained model, but measures overall rather than forgotten-data information and should accompany accuracy [5].Model inversion is a qualitative, often image-focused measure; samples reconstructed from the unlearned model should not reveal the forgotten class [63].Source: §6.3 · p. 16

7.1 Unlearning in Recommender Systems

• Recommender-system unlearning must preserve user-item similarity structure, motivating partition-aware retraining and specialized methods for removing user contributions, unwanted data, or sensitive attributes.Chen et al. partition data by user-item resemblance and combine partition submodels with attention; other approaches include rollback-based FRU, second-order AltEraser [104], and attribute-wise unlearning. Removal can also address privacy requests, poisoning, or outdated data.Source: §7.1 · p. 17

7.2 Unlearning Federated Learning

• Federated unlearning is complicated by aggregated updates, overlapping client data, and the common assumption that removed data belongs wholly to one client.Calibration training works for shallow architectures but not well for deep networks, while knowledge distillation [197] is costly and approximate; smart retraining avoids communication protocols but is limited to models with ≤10K parameters and requires storing potentially privacy-threatening snapshots. FRAMU uses reinforcement learning and real-time feedback for unlearning decisions in decentralized settings.Source: §7.2 · p. 18

7.3 Unlearning in Graph Embedding

• Graph unlearning must handle correlated data, node or edge requests, and uneven partitions; proposed methods include specialized partitioning, exact one-shot deletion, and GNNDELETE.Chen et al. [22] use assignment preferences, neighbor counts, and message passing; Cong et al. give algorithmically exact unlearning using linear-GNN operations and access only to data being forgotten. GNNDELETE [26] targets deleted-edge consistency and neighborhood influence, while later work covers mutual evolution, heterogeneous federated knowledge graphs, inductive unlearning, and graph classifiers with limited data.Source: §7.3 · p. 18

7.4 Unlearning in Lifelong Learning

• In lifelong learning, unlearning can counter catastrophic forgetting and exploding losses, while task-level removal must preserve transfer and remaining-task knowledge under online, zero-glance conditions.Du et al. unlearn harmful samples before updating to preserve prior performance and regularize extreme losses. Liu et al. [101] use isolated temporary task models merged into the main model for task-specific requests.Source: §7.4 · pp. 18–19

7.5 Unlearning in Large Language Models

• LLM unlearning methods selectively remove memorized data or undesirable behaviors while aiming to retain model capabilities and language quality.Chen et al. [20] add lightweight unlearning layers and fuse them across requests; DeMem uses reinforcement feedback to encourage paraphrasing. Other approaches include E2URec [190], parameter-efficient extraction and subtraction, behavior-specific neuron pruning [139], and Quark.Source: §7.5 · p. 19

7.6 Unlearning in Generative Models

• Generative-model unlearning spans concept erasure, image-generator data removal, black-box pattern removal, and GAN methods for forgetting outdated or undesired information.Unlearncanvas [217] provides a stylized image dataset for benchmarking diffusion-model unlearning; FAST [133] uses feature similarity to remove patterns without significantly affecting accuracy.Source: §7.6 · p. 19

8.1 Summary and Trends

• Influence functions are dominant because they estimate data items’ effects on model parameters or performance, enabling faster unlearning by reversing associated updates [3].The resulting offset may be bounded, as promising results suggest [29] [113].Source: §8.1 · pp. 19–20
• Parameter-space equivalence may leave forgotten-data effects intact, so independent verification is needed, although its success criteria and application-dependent thresholds remain controversial [60] [61] [63].Only a few verification studies exist [53], and parameter-distribution bounds can still yield false positives [64].Source: §8.1 · p. 20
• Federated learning can precisely remove a client using historical updates, while data locality may avoid catastrophic unlearning; non-IID data and partial-client requests remain challenges [107] [197].Source: §8.1 · p. 20
• Unlearning could repair poisoned models [106], remove biased features while maintaining model quality [66], and discard obsolete or redundant samples from overtrained networks [60].Influence functions can update problematic weights without costly retraining; forgetting may also boost accuracy through information-bottleneck-like compression [162].Source: §8.1 · p. 20

8.2 Open Research Questions

• Future work needs unified design requirements and benchmarks: current methods lack an overall winner, cover limited scenarios and requests, and use sparse, non-comparable benchmarks.Requirements include completeness, timeliness, and accuracy; benchmarks need common settings, broader code availability, and coverage beyond decremental-method efficiency.Source: §8.2 · p. 20
• Adversarial unlearning must certify both accuracy and privacy under attacks [52], while interpretable unlearning could support confidence and verification but remains unsolved [53] [3].Explanations may be difficult because unlearning reverses learning; explaining the process, potentially with influence functions, remains an open task.Source: §8.2 · p. 20
• Streaming removal and OOD forget sets need tailored methods: approximate parameter changes can undermine drift detection, while OOD items may have unequal influence [70].Unlearning obsolete data after concept drift requires analyzing contradictions between old and new data.Source: §8.2 · pp. 20–21
• Causality analysis could automatically remove polluted information when attacks affect millions of items but only a few can be detected.Source: §8.2 · p. 21

• The survey systematically investigates machine unlearning techniques and addresses key difficulties and research advances in conceptualizing, planning, and solving unlearning problems.Source: §9 · p. 21
• Its unified taxonomy groups machine unlearning strategies into model-agnostic, model-intrinsic, and data-driven methods to help categorize future studies and understand methodologies.Source: §9 · p. 21
• The survey summarizes and compares unlearning methodologies, offering a comprehensive view of current work and helping researchers identify strategies for different applications.Source: §9 · p. 21