PaperlayerView exact revision on arXiv

Source-linked AI summary

DeepObliviate: A Powerful Charm for Erasing Data Residual Memory in Deep Neural Networks

Yingzhe He, Guozhu Meng, Kai Chen, Jinwen He, Xingbo Hu

arXiv:2105.06209v1cs.LGcs.AIcs.CR

TL;DR

Deep neural networks make retraining from scratch for data deletion costly, despite privacy and legal reasons to support machine unlearning. DeepObliviate uses stored intermediate models and temporal residual memory to retrain only affected models and reuse the rest. Across five datasets, it reports improved accuracy and prediction speed over SISA under identical settings, alongside reduced retraining costs versus scratch retraining.

  • Problem

    Retraining from scratch to remove data is costly at modern deep-learning scale, while data deletion matters for privacy and legal rights.

    Source: §I · p. 1

  • Method

    DeepObliviate stores intermediate models, measures deleted data’s temporal residual memory, retrains affected models, and stitches them with reusable models.

    Source: §I · p. 2

  • Results

    5.8% accuracy improvement, 1.01× retrain speedup, and 32.5× prediction speedup over SISA on average under identical settings.

    Source: §I · p. 2

  • Takeaways & Limitations

    Across five datasets and deep models, DeepObliviate reduces unlearning costs while preserving model performance and passes backdoor-based verification.

    Source: §VIII · p. 13

    Providers must protect stored intermediate models as strictly as the final model.

    Source: §VI · p. 13

Abstract

from arXiv · show

Machine unlearning has great significance in guaranteeing model security and protecting user privacy. Additionally, many legal provisions clearly stipulate that users have the right to demand model providers to delete their own data from training set, that is, the right to be forgotten. The naive way of unlearning data is to retrain the model without it from scratch, which becomes extremely time and resource consuming at the modern scale of deep neural networks. Other unlearning approaches by refactoring model or training data struggle to gain a balance between overhead and model usability. In this paper, we propose an approach, dubbed as DeepObliviate, to implement machine unlearning efficiently, without modifying the normal training mode. Our approach improves the original training process by storing intermediate models on the hard disk. Given a data point to unlearn, we first quantify its temporal residual memory left in stored models. The influenced models will be retrained and we decide when to terminate the retraining based on the trend of residual memory on-the-fly. Last, we stitch an unlearned model by combining the retrained models and uninfluenced models. We extensively evaluate our approach on five datasets and deep learning models. Compared to the method of retraining from scratch, our approach can achieve 99.0%, 95.0%, 91.9%, 96.7%, 74.1% accuracy rates and 66.7$\times$, 75.0$\times$, 33.3$\times$, 29.4$\times$, 13.7$\times$ speedups on the MNIST, SVHN, CIFAR-10, Purchase, and ImageNet datasets, respectively. Compared to the state-of-the-art unlearning approach, we improve 5.8% accuracy, 32.5$\times$ prediction speedup, and reach a comparable retrain speedup under identical settings on average on these datasets. Additionally, DeepObliviate can also pass the backdoor-based unlearning verification.

I. INTRODUCTION

DeepObliviate addresses the cost of removing training data while preserving privacy and model usability. It estimates residual memory in stored models, retrains affected models, and reuses unaffected ones; the reported evaluation shows lower retraining costs and strong accuracy across five datasets.

I. INTRODUCTION

  • Deep learning data deletion is motivated by privacy and legal rights, as well as provider needs to remove poisoned or outdated data.The passages cite the right to be forgotten and describe poisoning and concept drift as threats to model usability.Source: §I · p. 1
  • Retraining from scratch after removing data is costly at modern deep-learning scale, while existing approaches can trade off overhead and model usability.The introduction notes that SISA loses accuracy as shard count increases and another cited method is limited to federated learning.Source: §I · pp. 1–2
  • DeepObliviate measures temporal residual memory with parameter-change vectors, uses its trend to stop retraining, and stitches retrained with reusable stored models.The method divides stored models into unseen, deleted, affected, and unaffected areas; detrended fluctuation analysis helps determine when memory can be ignored.Source: §I · p. 2
  • 5.8% accuracy improvement, 1.01× retrain speedup, and 32.5× prediction speedup over SISA on average under identical settings.The reported comparison uses the same storage overhead.Source: §I · p. 2
  • The approach supports single, bulk, and frequent deletion requests, and the evaluation reports backdoor-based verification of unlearning.The paper evaluates the method on five datasets and five deep-learning models.Source: §I · p. 2

A. Machine Unlearning

Machine unlearning intentionally removes the effects of selected data, but measuring those effects in deep networks is difficult. The section formalizes unlearning and defines criteria for comparing its fidelity, usability, cost, and verifiability.

A. Machine Unlearning

  • Machine unlearning is an intentional process of eliminating the effects of selected data, rather than ordinary learning’s addition of information.The text likens the operation to a reverse process and notes the difficulty of measuring data effects in deep networks.Source: A. Machine Unlearning · pp. 2–3

B. Formalization of Machine Unlearning

  • The formalization represents training as F(D)=M and deletion of x_d as U(x_d; M), yielding an unlearned model M′.The symbols define a learning algorithm, training dataset, trained model, and unlearning operation.Source: B. Formalization of Machine Unlearning · p. 3
  • Naive unlearning retrains on the remaining data, providing the ideal deletion baseline but requiring substantial computation and time.Other approaches approximate this baseline and can introduce errors relative to it.Source: B. Formalization of Machine Unlearning · p. 3
  • The evaluation criteria are consistency with naive unlearning, predictive accuracy, retraining and prediction time plus storage cost, and verifiability.Consistency measures agreement on the same test samples; verifiability asks whether a requested deletion succeeded.Source: B. Formalization of Machine Unlearning · p. 3

III. OVERVIEW

DEEPOBLIVIATE partitions training data into blocks and stores models after each block. For unlearning, it retrains from the block containing the deleted data, then stitches the result with stored models once the data’s later influence is small and stable.

A. Workflow and Symbolization

  • Training processes each block as a complete dataset, then stores the model parameters after that block.This yields B stored models after training the full dataset.Source: A. Workflow and Symbolization · p. 3
  • If deleted data lies in block d, retraining starts from M_(d−1) and trains block d without that point.Earlier stored models are unaffected because they precede the block containing the data.Source: A. Workflow and Symbolization · p. 3
  • Retraining stops at block d+t when the deleted point’s subsequent influence is very small and nearly unchanged, then model stitching constructs the unlearned model.The approach subtracts the model influence after retraining from the retrained model.Source: A. Workflow and Symbolization · p. 4

B. Temporal Residual Memory in Unlearning

The paper defines temporal residual memory as the lingering effect of a deleted data point on later model updates. Its measurements show a sharp early decline followed by a nearly stable level with small fluctuations.

B. Temporal Residual Memory in Unlearning

  • Temporal influence measures a block’s effect as the difference between consecutive models, conditional on the preceding model.Models are represented by fixed-order parameter vectors, so model subtraction becomes vector subtraction.Source: B. Temporal Residual Memory in… · p. 4
  • Temporal residual memory measures the difference between update vectors with and without the deleted data point after t later blocks.The paper quantifies that difference using L1 distance.Source: B. Temporal Residual Memory in… · pp. 4–5
  • Across three MNIST settings, Δ drops sharply, then moderately, and eventually stabilizes with small fluctuations.The curves are zero before the block containing the deleted data; the paper attributes some fluctuations to DNN randomness.Source: B. Temporal Residual Memory in… · p. 5

IV. THE DEEPOBLIVIATE APPROACH

DEEPOBLIVIATE stores intermediate models during block-based training, then uses retraining and residual-memory analysis to support unlearning. Unlike model sharding, it still trains on the complete dataset and produces an intact model.

IV. THE DEEPOBLIVIATE APPROACH

  • The approach stores intermediate models, quantifies deleted-data residual memory, analyzes its trend on-the-fly, and stitches the unlearned model.The stored models allow unaffected parts to be reused during unlearning.Source: §IV · p. 5

A. Stored Training

  • Blocks should be label-balanced, remain fixed during training, and each be trained as a complete dataset before its model parameters are saved.The shuffle option should therefore be off during training.Source: A. Stored Training · p. 5
  • Unlike model sharding, stored training still trains on the complete dataset and yields an intact model, keeping training closer to the normal process.The authors state that this approach achieves higher model accuracy than replacing the intact model with smaller collaborative models.Source: A. Stored Training · p. 5

B. Quantification of Residual Memory

  • Residual memory compares original and retrained parameter-update vectors; their L1 distance reflects the deleted data’s influence on later training.When that distance becomes very small, the update vectors nearly overlap and the data’s influence can be ignored for model parameters.Source: B. Quantification of Residual Memory · p. 6

C. On-the-fly Trend Analysis

DEEPOBLIVIATE fits residual-memory measurements with a power-law decay function and uses the trend’s stationarity to decide when to stop retraining. DFA estimates the decay exponent, while a derivative threshold determines the stopping condition.

C. On-the-fly Trend Analysis

  • Δ is high near the deleted-data block, drops rapidly, then fluctuates around a value, so the paper fits its sequence to a power-law decay function.The fit tests whether the sequence has stabilized and Δ will not change considerably.Source: C. On-the-fly Trend Analysis · p. 6
  • DFA reduces noise in the Δ curve and estimates exponent h for the power-law decay of the sequence.The method fits parameters a and b using least squares after obtaining h.Source: C. On-the-fly Trend Analysis · p. 6
  • Retraining stops when the absolute derivative of the fitted power-law function is below ε; otherwise, another block is retrained.The derivative measures the trend at the sequence’s right end, d+t.Source: C. On-the-fly Trend Analysis · p. 6

D. Model Stitching

DeepObliviate constructs the unlearned model by retraining from the model before the affected block, then stitching in the influence of later stored blocks.

D. Model Stitching

  • After updating the affected block, retraining starts from the preceding stored model and proceeds over the updated block and a retrained interval.The updated dataset replaces Dd with D′d.Source: D. Model Stitching · p. 6
  • The remaining blocks’ influence is stitched onto the retrained model to form the unlearned model.The section presents this as a way to represent trained model MB jointly using intermediate model Md and model stitching.Source: D. Model Stitching · pp. 6–7
  • Model addition and subtraction treat models as matrices and produce a new matrix that is also a model.The equation describes stitching a model component with a model difference.Source: D. Model Stitching · p. 7

E. Algorithm Summary and Analysis

The algorithm measures residual memory during retraining and stops when its fitted decay trend becomes sufficiently flat, then stitches the remaining stored models.

E. Algorithm Summary and Analysis

  • The algorithm initializes the unlearned model, updates the affected block, then retrains while checking stationarity on-the-fly.It measures residual memory using the L1 distance between two model updates and fits a power-law decay curve.Source: E. Algorithm Summary and Analysis · p. 7
  • Retraining stops when the fitted residual-memory curve’s derivative at the current endpoint has absolute value below ε.The method fits parameters a and b by least squares after estimating exponent h with DFA.Source: E. Algorithm Summary and Analysis · p. 7
  • For one-point unlearning, retraining uses t = 0.01∼0.08×B, about 1%∼8% of naive-method training cost.Other costs are parameter-scale arithmetic and DFA on a sequence shorter than B, described as negligible relative to retraining time.Source: E. Algorithm Summary and Analysis · p. 7

V. EVALUATION

The evaluation tests DeepObliviate across five datasets and canonical models, assessing unlearning effectiveness, verification, and comparison with a state-of-the-art approach. The setup uses naive unlearning as its baseline and evaluates accuracy, consistency, and speed-up.

V. EVALUATION

  • Experiments use five datasets and five canonical deep-learning models to examine effectiveness, verification, and comparison with a state-of-the-art unlearning approach.The evaluation questions include consistency, accuracy, and speed-up.Source: §V · p. 7

A. Experimental Setup

  • To train qualified models for unlearning, the study paired LeNet-5 with MNIST, ResNet-18 with CIFAR-10, VGG-11 with SVHN, a one-hidden-layer DNN with Purchase, and ResNet-50 with ImageNet.Source: A. Experimental Setup · p. 7
  • Naive unlearning is the baseline, and Table I evaluates MNIST/LeNet-5 across deletion counts, positions, and ε using accuracy, consistency, and retraining speed-up.The original model’s test accuracy is 99.10%; for naive unlearning, consistency is 100% and speed-up is 1×.Source: Table I · p. 8

B. Effectiveness

On MNIST, DeepObliviate remains close to naive-unlearning accuracy while achieving retraining speed-ups, with results varying by deletion amount and the ε stopping condition. Deletions across multiple blocks can be reduced to one-block deletions, and deletion position has little effect in the reported MNIST tests.

B. Effectiveness

  • Deleting data from multiple blocks can be reduced to repeated one-block deletions, and the reported MNIST results show no obvious performance gaps across tested deletion positions.The evaluation assumes multi-point deletions within a block; multi-block deletions can be handled as multiple one-block deletions.Source: B. Effectiveness · p. 8, 1) Unlearning LeNet-5 on MNIST · p. 8

1) Unlearning LeNet-5 on MNIST:

  • 99.03% accuracy, very close to naive unlearning’s 99.07%, and 45.45× speed-up are reported for unlearning one MNIST data point.When deleting 60 points, the method reaches 97.69% accuracy and 99.26% consistency at 3.03% cost (33.0× speed-up) for ε = 0.1; at ε = 0.04 it reaches 98.01% accuracy and 99.55% consistency at 5.2% cost (19.2× speed-up).Source: 1) Unlearning LeNet-5 on MNIST · p. 8
  • Accuracy, consistency, and speed-up curves all decline as the number of unlearned points increases; the plotted range is 1 to 600 points.At 600 points, speed-up is 8∼19.2× for ε = 0.02∼0.1; at 60 points it is 14.3∼31.2×, and at 6 points it is 30.3∼50×.Source: 1) Unlearning LeNet-5 on MNIST · p. 8, Figure 6 · p. 9
  • At 600 deleted points, the maximum accuracy gap is 1.06% for ε = 0.10 and 0.30% for ε = 0.02, while consistency remains above 99%.The caption specifies consecutive deletions beginning at the 10% (100th) position.Source: 1) Unlearning LeNet-5 on MNIST · p. 8, Figure 6 · p. 9
  • A smaller ε improves accuracy and consistency but requires more retraining, while larger ε increases speed-up.The paper also reports that more unlearned data brings greater accuracy degradation and leaves more residual memory uneliminated.Source: 1) Unlearning LeNet-5 on MNIST · pp. 8–9

2) Unlearning VGG-11 on SVHN:

Across SVHN, CIFAR-10, Purchase, and ImageNet, DEEPOBLIVIATE retains accuracy close to naive unlearning while reducing retraining cost. The results also show that accuracy, consistency, and speed-up can shift with deletion size and the stopping threshold.

2) Unlearning VGG-11 on SVHN:

  • 94.91% accuracy, 98.99% consistency, and 8.2× speed-up were achieved on SVHN with 0.01% unlearned data at ε = 0.08.At ε = 0.1, deleting 0.1% of data yielded 94.50% accuracy, 98.18% consistency, and 6.5× speed-up; smaller ε raised accuracy and consistency while lowering speed-up.Source: 2) Unlearning VGG-11 on SVHN · p. 9, Table II · p. 10

3) Unlearning ResNet-18 on CIFAR-10:

4) Unlearning DNN on Purchase:

  • 97.15% accuracy, 99.70% consistency, and 33.3× speed-up were achieved on Purchase when unlearning 0.01% of data.For deletion of ≤10% of data, accuracy exceeded 93% and consistency exceeded 98.7% at less than 8% time cost.Source: 4) Unlearning DNN on Purchase · p. 9, Table IV · p. 10

5) Unlearning ResNet-50 on ImageNet:

  • 74.42% top-1 and 91.89% top-5 accuracy approached naive unlearning’s 74.60% and 92.05%, with 9.5× speed-up after deleting one first-position ImageNet point.Across model and dataset complexity, one-point deletion speed-up fell from 75× to 14× while accuracy remained within <1% of naive unlearning.Source: 5) Unlearning ResNet-50 on ImageNet · p. 10

C. Backdoor-based Unlearning Verification

Backdoor-trigger tests distinguish DEEPOBLIVIATE from the original model while producing results similar to naive unlearning. In comparison with SISA, it has higher accuracy at similar retraining speed under matched storage, and requires no additional prediction cost.

C. Backdoor-based Unlearning Verification

  • 13.4% maximum attack success on MNIST for DEEPOBLIVIATE approached 11.6% after naive unlearning, versus 86.7% on the original model.The verification uses crafted triggers in unlearned data because models may still predict deleted examples correctly when similar data remains.Source: C. Backdoor-based Unlearning… · pp. 10–11
  • DEEPOBLIVIATE passed backdoor-based unlearning verification on four datasets, with the backdoor data almost deleted and little reduction in model performance.On CIFAR-10, average attack success fell from 80.4% on the original model to 14.33% after DEEPOBLIVIATE, close to 11.63% after naive unlearning.Source: C. Backdoor-based Unlearning… · p. 11

D. Comparison with Other Methods

  • Under matched storage, DEEPOBLIVIATE achieves higher accuracy and similar retraining speed-up than SISA, without additional prediction cost.The comparison sets B = SR to equalize storage overhead.Source: D. Comparison with Other Methods · pp. 11–12
  • Compared with naive unlearning, DEEPOBLIVIATE accelerates unlearning 67×, 75×, 35×, and 14× on MNIST, SVHN, Purchase, and ImageNet, respectively, with accuracy reductions ≤0.5%.Unlike SISA’s independent shard models, DEEPOBLIVIATE trains on all blocks and considers when to terminate retraining.Source: D. Comparison with Other Methods · p. 12
  • DEEPOBLIVIATE needs no prior probabilities for which data will be deleted and supports deleting any number of points from any position.SISA’s shard isolation can weaken the aggregation model, and its performance depends on how the dataset is divided.Source: D. Comparison with Other Methods · p. 12

VI. DISCUSSION

DeepObliviate reuses stored models to reduce unlearning costs while preserving model performance, and its unlearning effect is verified via a backdoor attack. The discussion details storage and prediction costs, while the related work contrasts its approach with prior methods.

VI. DISCUSSION

  • Training adds no time overhead beyond normal training, but storing B intermediate models requires B× storage occupancy.The paper notes that 1,000 VGG-11 models require about 1 TB of hard-disk storage.Source: §VI · p. 12
  • At prediction time, DeepObliviate matches the original model’s 1× cost, while SISA incurs S× computation for S shards.For larger datasets, the paper suggests choosing a larger B; increasing B can improve retraining speedup with some loss of accuracy. On MNIST, deleting data of varying labels produces only slight differences in Δ curves, while deletion position and amount do not affect their curvature.Source: §VI · p. 12
  • Stored intermediate models must be protected as strictly as the final model, and differential privacy cannot by itself make a specific training point’s contribution zero.The paper also says DeepObliviate can remove the influence of poisonous data to some extent.Source: §VI · p. 13

VII. RELATED WORK

  • Parameter-manipulation methods directly change parameters to offset deleted data’s influence, whereas dataset-reorganizing methods retrain models on reorganized data.The related-work discussion says SISA’s shard separation changes the original training pattern and can reduce accuracy because collective weaker models may not match a complete model.Source: §VII · p. 13
  • Unlike prior influence measurement based on prediction or loss changes, DeepObliviate uses updated models to assess whether a point is completely unlearned.The paper also distinguishes its approach from prior memorization work by saying it removes both unintended memorization and intended memorized information of unlearned data.Source: §VII · p. 13

VIII. CONCLUSION

  • Across five datasets and deep models, DeepObliviate reduces unlearning costs while preserving performance, passes backdoor-based verification, and applies without interfering with the model.The conclusion presents it as a simple, fast, cost-effective method for unlearning target data.Source: §VIII · p. 13

Related papers

Loading 2105.06209v1…