PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards Unbounded Machine Unlearning

Meghdad Kurmanji, Peter Triantafillou, Jamie Hayes, Eleni Triantafillou

arXiv:2302.09880v3cs.LGcs.CR

TL;DR

Deep machine unlearning asks how to remove selected training examples while preserving useful model behavior, a problem relevant to privacy, bias removal, and resolving mislabeled-data confusion. The paper evaluates application-specific forgetting criteria and proposes SCRUB, a scalable teacher-student method with rewinding for privacy-sensitive cases. Across the evaluated scenarios, SCRUB is consistently strong on forgetting quality, utility, and efficiency, although it lacks theoretical guarantees.

  • Problem

    Deep machine unlearning must remove selected training examples while preserving performance, but forgetting and its metrics differ across privacy, bias-removal, and confusion-resolution applications.

  • Method

    The paper evaluates three application-specific unlearning scenarios and proposes SCRUB, a teacher-student method complemented by rewinding for privacy-sensitive forgetting.

  • Results

    SCRUB is reported as the most consistent top performer across application-dependent forgetting metrics while remaining strong on retained-data accuracy, generalization, and efficiency.

  • Takeaways & Limitations

    Evaluating unlearning across applications exposes differing priorities and trade-offs, while SCRUB provides a practical scalable method that performs strongly across them.

  • Takeaways & Limitations

    SCRUB lacks theoretical guarantees, which may make it ill-suited for certain application scenarios.

Abstract

from arXiv · show

Deep machine unlearning is the problem of `removing' from a trained neural network a subset of its training set. This problem is very timely and has many applications, including the key tasks of removing biases (RB), resolving confusion (RC) (caused by mislabelled data in trained models), as well as allowing users to exercise their `right to be forgotten' to protect User Privacy (UP). This paper is the first, to our knowledge, to study unlearning for different applications (RB, RC, UP), with the view that each has its own desiderata, definitions for `forgetting' and associated metrics for forget quality. For UP, we propose a novel adaptation of a strong Membership Inference Attack for unlearning. We also propose SCRUB, a novel unlearning algorithm, which is the only method that is consistently a top performer for forget quality across the different application-dependent metrics for RB, RC, and UP. At the same time, SCRUB is also consistently a top performer on metrics that measure model utility (i.e. accuracy on retained data and generalization), and is more efficient than previous work. The above are substantiated through a comprehensive empirical evaluation against previous state-of-the-art.

1 Introduction

Deep machine unlearning seeks to remove selected training examples from trained neural networks, motivated by privacy and other data-quality applications. The paper argues that existing methods can be impractical or perform poorly, and introduces SCRUB as a scalable method evaluated across applications.

  • Motivation: Deep machine unlearning aims to remove a subset of training data from a trained neural network, supporting users’ right to be forgotten.The problem is practically relevant because deep learning systems are widely used across applications.
  • Motivation: Unlearning also targets outdated examples, outliers, poisoned samples, noisy labels, and harmful biases beyond privacy.These applications motivate application-dependent definitions and priorities for forgetting.
  • Challenges: Removing training-example influence is difficult because deep models memorize instances and have highly non-convex parameter relationships.Retraining from scratch guarantees removal of the forgotten cohort but is the naive alternative.
  • Challenges: Approximate unlearning methods often perform poorly across removing biases, resolving confusion, or user privacy, while previous methods scale poorly with training-set and model size.These limitations can make existing approaches impractical.
  • Contribution: SCRUB uses a teacher-student formulation in which the student selectively disobeys an all-knowing teacher about data to be deleted.The method is designed to avoid limiting assumptions and poor scalability limits.
  • Contribution: Across comprehensive evaluations, SCRUB is reported as the most consistent top performer for application-specific forgetting while also performing strongly on retained-data accuracy and generalization.The evaluation covers different baselines, datasets, architectures, and three application scenarios.

2 Problem Definition for Unlearning Unbound

The paper formalizes unlearning around a forget set and a retain set: produce an unlearned model that forgets selected examples while preserving utility. It treats forgetting as application-dependent and emphasizes trade-offs among forget quality, utility, and efficiency.

  • Notation and preliminaries: A training dataset contains input features x_i, labels y_i, and a neural network f(·; w) trained by empirical risk minimization.The loss uses cross-entropy over the training examples.
  • Deep Machine Unlearning: The original model f(·; w_o) is trained on D, while the forget set D_f contains examples to remove and the retain set D_r contains examples whose information may remain.The standard setup takes D_r as the complement of D_f.
  • Deep Machine Unlearning: The goal is to produce weights w_u that forget D_f without reducing performance on retained data or generalization beyond D.Utility includes performance on D_r and on data outside the training set.
  • Defining and measuring forgetting: Forgetting is application-dependent across removing biases, resolving confusion, and user privacy, with each scenario having its own metrics.Removing biases seeks high forget-set error, while resolving confusion addresses mislabeled examples.
  • Fundamental trade-offs: A top-performing method must jointly achieve high forget quality, retained and unseen-data accuracy, and efficiency or scalability.The paper identifies trade-offs both among these objectives and among application-specific notions of forget quality.

3 SCalable Remembering and Unlearning unBound (SCRUB)

SCRUB casts unlearning as selective teacher-student learning: preserve the teacher’s behavior on retained data while moving away from it on forgotten data. Rewinding adjusts forgetting when maximal error could undermine privacy, and the method is evaluated as efficient and broadly effective.

  • SCRUB: SCRUB is a teacher-student unlearning algorithm designed to forget efficiently without hurting model utility.It casts the original model as a teacher and trains a student to selectively retain or reject its information.
  • SCRUB: The student obeys the original teacher on retain examples but disobeys it on forget examples.This contrast targets remembering D_r while forgetting D_f.
  • SCRUB: SCRUB initializes the student from the teacher and then modifies it to unlearn the forget set.Initialization preserves good retain-set performance before unlearning begins.
  • SCRUB: The objective encourages divergence from the teacher on forget examples, closeness on retain examples, and task performance on the retain set.The retain task loss further strengthens the incentive to preserve utility.
  • Optimization: SCRUB alternates forget-set max-steps with retain-set min-steps, followed by additional retain updates when needed to restore performance.This optimization recipe addresses oscillations arising from competing objectives.
  • Empirical results: In removing-biases results, SCRUB is the only consistent top performer for forgetting and preserving utility, while Bad-T is faster but fails at forgetting and damages utility.The figure averages across ResNet and All-CNN variants and reports a retraining-runtime scale-up factor for large-scale results.
  • SCRUB and Rewind: SCRUB+R rewinds to a checkpoint where forget-set error is just high enough to approximate a reference point for privacy applications.The reference is obtained from an identically distributed validation set rather than retraining from scratch, and the procedure improves membership-inference defense empirically.

4 Related Work

Prior work offers exact, probabilistic, and theoretically guaranteed unlearning approaches, but the paper questions retrain-from-scratch indistinguishability as a universal success criterion. It instead evaluates application-specific forgetting, while distinguishing unlearning from related areas and comparing teacher-student baselines.

  • Unlearning definitions: Earlier work defined exact forgetting for structured problems, while later approaches introduced probabilistic or retrain-from-scratch indistinguishability formulations.These formulations provide progressively relaxed notions of unlearning.
  • Unlearning definitions: The paper argues that matching a retrained model is neither sufficient nor necessary for successful unlearning.Similar weights can arise from non-overlapping datasets, while retraining can produce different model distributions under changed settings.
  • Unlearning methods: Prior methods include exact partition-based unlearning and methods with theoretical guarantees under particular assumptions.The related work surveys approaches built on differential-privacy-inspired frameworks.
  • Related research areas: Differential privacy seeks to prevent storage of information about any training instance, whereas unlearning removes influence only from a designated forget set.Forgetting a task is also distinct from removing the influence of particular training examples.
  • Teacher-student and contrastive methods: SCRUB differs from teacher-student methods that move toward a bad teacher because it moves away from a single original teacher on forget examples.Bad-T forgets poorly compared with SCRUB and degrades model quality.
  • Applications and metrics: In the RC evaluation, SCRUB is identified as the strongest performer for resolving class confusion on CIFAR-5 with ResNet and All-CNN.The setup confuses 50 class-0 samples as class 1 and 50 class-1 samples as class 0.

5 Experiments

The experiments evaluate unlearning across removing biases, resolving confusion, and user privacy, using multiple datasets, architectures, settings, and forget-quality metrics. SCRUB is consistently strong across these applications while preserving model quality, whereas competing methods show narrower strengths or practical weaknesses.

  • Experimental setup: Experiments cover removing biases, resolving confusion, and user privacy, each with application-specific definitions and forget-quality metrics.The evaluation uses CIFAR-10 and Lacuna-10 with All-CNN and ResNet-18 architectures, in small-scale and large-scale settings.
  • Removing biases: SCRUB is by far the most consistent method for achieving high forget quality in removing-bias experiments without hurting retain and test errors.Figure 2 counts top performers using overlap between 95% confidence intervals and the best mean across small- and large-scale settings.
  • Resolving confusion: SCRUB is by far the most consistent method at eliminating class confusion while preserving model quality on retain and test sets.Resolving confusion places mislabeled examples in the forget set and evaluates confusion-related errors.
  • User privacy: Membership inference attacks assess whether unlearned examples can be distinguished from examples never seen by the model in the user-privacy setting.The evaluation includes a basic attack and the first reported adaptation of LiRA to the unlearning protocol.
  • User privacy: SCRUB+R is a strong performer against basic MIAs, while rewinding can substantially improve results when needed, especially for smaller selective forget sets.Figure 3 reports MIA results across forget-set sizes on CIFAR-10 with ResNet; rewinding was triggered only once in the reported basic-MIA setting.
  • Overview of results and take-aways: SCRUB+R outperforms other methods in defending the adapted LiRA attack, while competing approaches exhibit failures in forgetting, scalability, consistency, or runtime.Finetuning retains utility but fails to forget; NTK does not scale beyond small datasets, Fisher can be very slow, and SCRUB outperforms EU-k especially for resolving confusion.

6 Discussion and Conclusion

The paper presents SCRUB as a practical, scalable unlearning method evaluated across application-dependent forgetting criteria, while emphasizing that formal definitions and theoretical guarantees remain open challenges.

  • Discussion and Conclusion: Deep machine unlearning remains a young research area despite substantial recent attention.
  • Discussion and Conclusion: SCRUB is empirically a consistent top performer for forgetting quality across application-dependent metrics, with minimal utility degradation.The evaluation covers multiple application scenarios and metrics.
  • Discussion and Conclusion: Formal definitions of unlearning and well-established quality metrics remain elusive, so this work focuses on empirical application-dependent evaluation.
  • Discussion and Conclusion: SCRUB lacks theoretical guarantees, which may make it ill-suited for certain application scenarios.
  • Discussion and Conclusion: Future work should study scalability in larger models, additional architectures, training objectives, domains, and modalities.
  • Discussion and Conclusion: Because SCRUB cannot be proven to perfectly mitigate bias, unintended behavior, or privacy risks, practical auditing remains critical.

7 Appendix

The appendix organizes additional material on limitations, experimental details, metrics, membership inference attacks, ablations, scale, and application-specific results.

  • Appendix: The appendix includes sections on limitations and future work, experimental details and pseudocode, and formal metric descriptions.
  • Appendix: Additional appendix sections cover membership inference attacks, ablations, larger-scale settings, and results for removing biases, resolving confusion, and user privacy.

8 Limitations and Future Work

The paper identifies theoretical, optimization, rewinding, and scalability limitations of SCRUB and outlines future work across guarantees, larger settings, architectures, and objectives.

  • Limitations and Future Work: The paper describes SCRUB as consistently top-performing with a minimal performance drop, while noting that its limitations require future work.
  • Limitations and Future Work: SCRUB has no theoretical guarantees, and existing guaranteed methods either do not apply to deep neural networks or perform poorly and do not scale beyond small datasets.
  • Limitations and Future Work: Tuning SCRUB’s min-max objective is difficult and unstable, potentially causing oscillating behavior.
  • Limitations and Future Work: SCRUB’s rewinding procedure can encounter premature stopping when forget error is too low, preventing it from reaching a reference level.
  • Limitations and Future Work: The evaluated datasets and models are not too large because larger experiments would prevent comparisons with previous work.
  • Limitations and Future Work: Future work should investigate unlearning with larger models, architectures such as Transformers, and objectives such as self-supervised learning.

9 Experimental Details and Pseudocode

The experiments evaluate SCRUB and baselines on CIFAR and Lacuna datasets with All-CNN and ResNet models, using specified optimization, preprocessing, and scale settings.

  • Datasets: Experiments use CIFAR-10 and Lacuna-10, with smaller CIFAR-5 and Lacuna-5 variants created from five classes and sampled images.
  • Models: Models include modified All-CNN and ResNet-18 architectures, with reduced filters for small-scale experiments and exact architectures for large-scale experiments.
  • Pretraining: Models are pretrained for 30 epochs on CIFAR-100 or Lacuna-100 using SGD, cross-entropy loss, weight decay 0.0005, momentum 0.9, and batch size 128.
  • Baselines: Baselines include Original, trained on all data, and Retrain, trained on the retained set with the original training hyperparameters.
  • Baselines: For EU-k and CF-k, all layers except the final network block are frozen in both ResNet and All-CNN experiments.
  • Baselines: Bad-T uses one Adam unlearning epoch, temperature 4, and the whole retain set because using 30% produced high test errors.
  • SCRUB pseudocode and parameters: SCRUB uses the same optimizer for min and max steps, initializes learning rate at 0.0005, and decays it by 0.1 to control oscillations.

10 Formal Description of Metrics

The paper defines general error metrics and application-specific metrics for removing biases, resolving confusion, and protecting user privacy. Forgetting is evaluated separately from retained-data and test performance.

  • The paper introduces application-dependent metrics for resolving confusion and user privacy after first defining forget, retain, and test errors.
  • Retain, forget, and test errors are computed on Dr, Df, and Dt, respectively.
  • Metrics for Unlearning for Resolving Confusion (RC): Resolving confusion evaluates scenarios where mislabeled examples between classes A and B are placed in the forget set.
  • Error counts all misclassifications, with lower test and retain error preferred but higher forget error preferred.
  • Metrics for Unlearning for Resolving Confusion (RC): IC-ERR counts only mistakes involving the confused classes, while FGT-ERR focuses on confusion between those classes.

11 Membership Inference Attacks: Description and Additional Findings

The paper evaluates membership inference against unlearned models using Basic MIA and a new LiRA-for-unlearning adaptation. SCRUB(+R) provides the strongest reported defense, especially at small false-positive rates, while evaluation depends on comparable forget and test distributions.

  • Basic MIA: Basic MIA uses loss values from forget and held-out test examples to predict original training membership; 50% accuracy is the optimal defense.
  • Basic MIA: MIA evaluation can be unreliable when forget and test distributions differ, so the held-out test set is matched to the forget-set distribution.
  • LiRA-for-unlearning attack: The adapted LiRA attack models whether examples were forgotten by comparing confidence distributions from shadow unlearned and non-unlearned models.
  • LiRA-for-unlearning attack: LiRA-for-unlearning is presented as the first adaptation of LiRA to unlearning and gives the attacker knowledge of the unlearning algorithm.
  • Experimental setup: The privacy experiments use 256 shadow original models, 10K forget sets per method, and target forget sets of 200 class-5 CIFAR-10 examples.
  • LiRA-for-unlearning attack: SCRUB(+R) most successfully defends the LiRA-for-unlearning attack, particularly in the low-False-Positive-Rate region, and also has the best AUC.
  • Additional findings: CF-k usually fails to improve original-model privacy, while EU-k improves it only slightly and unreliably.

12 Ablations and Sensitivity Analysis

Ablations show that SCRUB needs balanced max- and min-steps, a final sequence of min-steps, and benefits from cross-entropy, while remaining relatively insensitive to γ and α.

  • Training dynamics: Alternating max-steps and min-steps is necessary to preserve retain and validation performance while increasing forget error.
  • Training dynamics: Too few max-steps lowers forget error, while omitting final min-steps prevents a good balance among forget, retain, and validation errors.
  • Loss ablation: Adding cross-entropy provides small but consistent protection against performance degradation during unlearning.
  • Hyperparameter sensitivity: SCRUB is not very sensitive to γ and α: retain error stays low across tested values, with several settings achieving desired test and forget errors.

13 Larger-scale Settings

The larger-scale and additional experiments examine SCRUB across larger models, datasets, applications, and forget-set configurations. Across reported settings, SCRUB remains a consistent top performer for forgetting while preserving utility and defending membership-inference attacks.

  • Larger-scale Settings: The scale-up experiments use VGG16+BN with almost 138M parameters on CIFAR-100 and vary forget-set sizes.
  • Efficiency: Table 6 defines scale-up as the fraction of retraining-from-scratch runtime over an unlearning algorithm's runtime.
  • Removing Biases (RB): SCRUB is a top performer for removing biases with minimal performance degradation in both small- and large-scale settings.
  • Resolving Confusion (RC): For resolving confusion, SCRUB is reported as the best performer by far while preserving performance on other classes across CIFAR-5 and Lacuna-5 experiments.
  • Additional results: SCRUB is consistently a top performer on the reported additional metric results across settings.
  • User Privacy (UP): SCRUB, especially with rewinding, consistently provides a strong defense against Basic MIAs across the evaluated settings.
Loading 2302.09880v3…