Source-linked AI summary
Evaluating Machine Unlearning via Epistemic Uncertainty
Alexander Becker, Thomas Liebig
TL;DR
Machine Unlearning needs practical evaluation because adversarial attacks and retraining-based comparisons are costly or insufficient. The paper introduces an epistemic-uncertainty metric, derives a more efficient upper bound, and tests it across forgetting approaches. Results show that the hypotheses hold for retraining and Fisher Forgetting but are rejected for Amnesiac Unlearning.
Problem
Existing Machine Unlearning evaluations rely on adversarial attacks or comparison with retraining from scratch, while a practical general metric is needed.
Method
The paper defines an epistemic-uncertainty-based metric, derives a more efficient theoretical upper bound, and evaluates it on three forgetting approaches.
Results
The hypotheses receive evidence for retraining and Fisher Forgetting but are rejected for Amnesiac Unlearning because forgetting changes model parameters in a different direction.
Takeaways & Limitations
The metric can be computed efficiently, avoids retraining, and supports an information-theoretical interpretation for practical forgetting evaluation.
Takeaways & Limitations
Computing the full Fisher Information matrix is unreasonable for larger models because it may not fit in memory and can undermine efficiency relative to retraining.
Abstract
from arXiv · showhide
There has been a growing interest in Machine Unlearning recently, primarily due to legal requirements such as the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act. Thus, multiple approaches were presented to remove the influence of specific target data points from a trained model. However, when evaluating the success of unlearning, current approaches either use adversarial attacks or compare their results to the optimal solution, which usually incorporates retraining from scratch. We argue that both ways are insufficient in practice. In this work, we present an evaluation metric for Machine Unlearning algorithms based on epistemic uncertainty. This is the first definition of a general evaluation metric for Machine Unlearning to our best knowledge.
1 Introduction and Related Work
Machine Unlearning spans many model types and use cases, but existing evaluation methods are either restrictive, costly, or insufficiently reliable. This work therefore proposes an efficient, theoretically grounded metric based on epistemic uncertainty.
- Scope and motivation: Existing unlearning methods cover decision trees, linear models, neural networks, Markov Chain Monte Carlo, and federated models, with applications extending beyond legal data protection.For example, forgetting has also been used to remove model backdoors [16].
- Existing evaluation approaches: Provably perfect-unlearning methods avoid evaluation but rely on strong assumptions, while broadly applicable SISA training incurs considerable performance losses.
- Existing evaluation approaches: Comparing an unlearned model with retraining from scratch is theoretically interesting but impractical when retraining is computationally expensive.
- Existing evaluation approaches: Adversarial-attack evaluation can be expensive and a failed attack does not guarantee that sensitive information was removed.Attack failure may also reflect hyperparameters, model-parameter estimates, or attack type rather than successful forgetting.
- Contributions: The paper defines an epistemic-uncertainty-based evaluation metric, provides a more efficiently computed theoretical upper bound, and studies it on three forgetting approaches.The metric also supports an information-theoretical interpretation.
2 Unlearning Algorithms
The paper compares retraining from scratch with Amnesiac and Fisher Forgetting as distinct ways to remove target-data influence. These methods differ in computational cost, parameter updates, and how closely they approximate retraining.
- Retraining from scratch: Retraining on the remaining data D_r is treated as optimal because it excludes target data D_f and preserves performance, but it is computationally expensive for large models and datasets.
- Amnesiac Unlearning: Amnesiac Unlearning removes target-data influence by reverting every recorded parameter update associated with a target point.It requires storing epoch- and batch-level updates together with their related data points.
- Amnesiac Unlearning: For Amnesiac Unlearning, iterative and batch forgetting are equivalent because the relevant updates are subtracted from the model parameters independently of forgetting order.
- Fisher Forgetting: Fisher Forgetting adds normally distributed parameter noise to hide the difference between the current model and a model obtained by retraining.Its covariance is based on an approximated Fisher Information matrix, while α combines λ and σ²_h; λ trades off remaining-data loss against deviation from retraining.
- Fisher Forgetting: Fisher Forgetting is imperfect and offers no guarantees, although it originates from the perfect-forgetting approach Optimal Quadratic Scrubbing.
3 Measuring the Success of Forgetting
The paper evaluates forgetting through separate efficacy, fidelity, and efficiency criteria, using Fisher-information-based epistemic uncertainty to quantify residual information about target data. It further derives computationally cheaper approximations and bounds for practical evaluation.
- Machine Unlearning should be evaluated separately for efficacy, fidelity, and efficiency rather than with one combined metric.Efficacy concerns information about target data, fidelity concerns performance on remaining data, and efficiency concerns outperforming retraining costs.
- 3.1 Evaluating Forgetting via Epistemic Uncertainty: The trace of the Fisher Information Matrix measures both information carried by parameters about a dataset and their epistemic uncertainty.The diagonal approximation suffices, with each entry representing information carried by one parameter; tracing it yields an overall information value.
- 3.1 Evaluating Forgetting via Epistemic Uncertainty: Practical evaluation uses an empirical Fisher approximation because computing the full matrix is expensive and may exceed memory for large models.The full computation can scale as O(|D| · F + |D| · G · |θ|^2), making it unsuitable when evaluation must improve on retraining efficiency.
- 3.1 Evaluating Forgetting via Epistemic Uncertainty: The efficacy score measures information exposed about target data, with lower values indicating greater uncertainty and therefore more effective forgetting.Exact unlearning or retraining may not reach zero because remaining data can generalize to target points, but the score still supports comparisons among algorithms.
- 3.1 Evaluating Forgetting via Epistemic Uncertainty: The squared gradient norm of cross-entropy loss lower-bounds uncertainty, yielding an efficacy upper bound that requires one parameter-gradient computation instead of one per data point.Its complexity is O(|D| · F + G · |θ|), compared with O(|D| · F + |D| · G · |θ|) for the efficacy score.
4 Experiments
Experiments on MNIST evaluate efficacy, its upper bound, and membership-inference behavior after retraining, Fisher Forgetting, and Amnesiac Unlearning. Retraining and Fisher Forgetting generally reduce efficacy as more target data are forgotten, whereas Amnesiac Unlearning can reverse this trend despite lowering attack probabilities.
- 4 Experiments: The experiments use MNIST and CIFAR10, compare retraining, Fisher Forgetting, and Amnesiac Unlearning, and repeat training with 20 random initializations per dataset.CIFAR10 results are omitted because the authors report identical observations to MNIST; the experiments are publicly available.
- 4.2 Results: Forgetting decreases efficacy as more target data are removed for retraining and Fisher Forgetting, but Amnesiac Unlearning instead increases efficacy with larger forgotten percentages.Amnesiac Unlearning reverts target-related updates, moving the model toward the initial model and causing efficacy to converge toward initial-model values.
- 4.2 Results: The efficacy upper bound preserves distribution shapes and relative comparisons while producing absolute values an order of magnitude larger than actual efficacies.This supports using the upper bound to compare multiple forgetting results efficiently.
- 4.2 Results: Forgetting lowers mean membership-inference attack probabilities as the forgotten percentage increases, including for Amnesiac Unlearning.The pre-trained model provides the upper baseline, while retraining provides a second reference because target points were excluded from training.
- 4.2 Results: Efficacy and membership-inference probabilities move together for retraining and Fisher Forgetting but show the opposite relationship for Amnesiac Unlearning.For single points, efficacy spans a large range while mean attack probabilities remain similar.
5 Conclusion and Future Work
The proposed efficacy metric captures aspects of forgetting that accuracy and adversarial attacks alone cannot establish, while revealing that evaluation depends on how algorithms change model parameters. Experiments support retraining and Fisher Forgetting more than Amnesiac Unlearning, motivating broader metric suites.
- 5 Conclusion and Future Work: The metric is efficient, information-theoretic, retraining-free, and preserves efficacy-distribution relationships through its theoretical upper bound.These properties are presented as important for practical evaluation of forgetting algorithms.
- 5 Conclusion and Future Work: Figure 4 relates efficacy to membership inference attack mean probability before and after forgetting on log-log axes.The figure frames adversarial-attack behavior as a separate comparison with efficacy.
- 5 Conclusion and Future Work: Accuracy reduction and resistance to adversarial attacks are necessary conditions, but neither guarantees that sensitive information was removed.Reinitializing only a classifier can damage performance while leaving training information in unchanged feature-extraction layers.
- 5 Conclusion and Future Work: Retraining and Fisher Forgetting support the tested hypotheses, whereas Amnesiac Unlearning rejects them because its parameter updates move toward the initial rather than retrained model.The approaches therefore remove information in fundamentally different directions.
- 5 Conclusion and Future Work: The efficacy metric is a first step toward evaluation metrics tailored to the direction in which an algorithm changes model parameters.The authors propose surveying and categorizing algorithms by their parameter-update directions in future work.
Appendix
The appendix reports CIFAR10 evaluation visuals comparing efficacy, upper bounds, membership-inference probabilities, and accuracies across forgetting conditions. These displays cover pre-training, post-training, and several forgetting approaches.
- Appendix: Figure 5 shows log-scaled distributions of efficacy scores and upper bounds across pre-trained CIFAR10 models before and after forgetting.Solid lines denote efficacy scores, dashed lines denote upper bounds, and distributions correspond to target-class percentages excluding 0.01 for readability.
- Appendix: Table 2 reports mean accuracy and standard deviation on remaining, target, and test data for CIFAR10 models.p denotes the target-data percentage, and an asterisk marks accuracies constant across percentages.
- Appendix: Figure 6 compares whole-target-class efficacy for Initial, Pre-trained, Retraining, Amnesiac Unlearning, and Fisher Forgetting conditions.Both axes are log scaled.
- Appendix: Figure 7 presents log-scaled distributions of membership inference attack mean probabilities before and after forgetting across target-class percentages.The distributions are computed over pre-trained CIFAR10 models.
- Appendix: Figure 8 shows the log-log relationship between efficacy and membership inference attack mean probability before and after forgetting.The visualization complements the separate efficacy and attack-probability distributions.