PaperlayerView exact revision on arXiv

Source-linked AI summary

Mixed-Privacy Forgetting in Deep Networks

Aditya Golatkar, Alessandro Achille, Avinash Ravichandran, Marzia Polito, Stefano Soatto

arXiv:2012.13431v2cs.LGcs.AIcs.CV

TL;DR

The paper addresses how to remove selected training-data influence from large-scale vision networks while retaining accuracy and bounding residual information. It introduces mixed-privacy forgetting with linearized, separable weights, enabling efficient deletion and strong guarantees. The method achieves accuracy comparable to standard deep-network training and supports sequential forgetting requests, while remaining dependent on strongly convex regularization and the chosen forget subset.

  • Problem

    Forgetting selected data from deep networks is difficult because weights retain training information, while existing non-convex methods provide limited approximation quality and scalability.

  • Method

    ML-Forgetting linearizes the network and splits parameters into nonlinear core weights trained on protected data and linear user weights supporting efficient forgetting.

  • Results

    ML-Forgetting provides strong forgetting guarantees and DNN-comparable vision accuracy, handles sequential requests without performance degradation, and can delete all user data by discarding user weights.

  • Takeaways & Limitations

    Mixed-privacy training improves forgetting accuracy and information bounds, while complete user-data deletion can be implemented by erasing a designated weight subset.

  • Takeaways & Limitations

    The procedure relies on strongly convex regularization and is sensitive to the subset forgotten; efficient direct removal from highly non-convex deep networks remains unsolved.

Abstract

from arXiv · show

We show that the influence of a subset of the training samples can be removed -- or "forgotten" -- from the weights of a network trained on large-scale image classification tasks, and we provide strong computable bounds on the amount of remaining information after forgetting. Inspired by real-world applications of forgetting techniques, we introduce a novel notion of forgetting in mixed-privacy setting, where we know that a "core" subset of the training samples does not need to be forgotten. While this variation of the problem is conceptually simple, we show that working in this setting significantly improves the accuracy and guarantees of forgetting methods applied to vision classification tasks. Moreover, our method allows efficient removal of all information contained in non-core data by simply setting to zero a subset of the weights with minimal loss in performance. We achieve these results by replacing a standard deep network with a suitable linear approximation. With opportune changes to the network architecture and training procedure, we show that such linear approximation achieves comparable performance to the original network and that the forgetting problem becomes quadratic and can be solved efficiently even for large models. Unlike previous forgetting methods on deep networks, ours can achieve close to the state-of-the-art accuracy on large scale vision tasks. In particular, we show that our method allows forgetting without having to trade off the model accuracy.

1. Introduction

The paper frames forgetting as removing selected training-data influence without retraining or sacrificing accuracy, then introduces ML-Forgetting for mixed-privacy settings with protected core data. Its linearized, split-weight design provides strong guarantees, efficient deletion, and support for sequential requests.

  • Motivation: Forgetting seeks to remove selected cohorts’ influence and residual information without retraining, while preserving model accuracy.Restarting training is costly; zeroing all weights trivially forgets data but destroys accuracy.
  • Mixed-privacy setting: Mixed-privacy forgetting protects a core subset while targeting other training data for deletion.Core data may include generic pretraining data or stable task-specific data.
  • Method: ML-Forgetting trains nonlinear core weights and linear user weights, combining accuracy from the former with privacy guarantees from the latter.The method uses a first-order Taylor-inspired decomposition of the network.
  • Deletion: Erasing the user weights can completely forget all user data with the lowest possible performance drop.This provides a particularly simple deletion operation when every user datum must be forgotten.
  • Results: ML-Forgetting achieves strong forgetting guarantees and DNN-comparable test accuracy, while handling sequential requests without performance degradation.The paper presents this as the first such forgetting algorithm for large-scale vision datasets.

2. Related Work

Prior forgetting research covers statistical and specialized convex or combinatorial settings, while deep networks remain difficult because of non-convexity. Linearization offers a route toward tractable analysis and training, including vision performance comparable to nonlinear models.

  • Existing forgetting methods: Earlier methods achieve perfect forgetting by training multiple model combinations, but require heavy storage for models or gradients.Other approaches target statistical query learning or particular algorithms such as k-means.
  • Deep-network forgetting: Deep networks complicate forgetting because their highly non-convex loss functions make training-data effects difficult to estimate.Prior work uses SGD stability, information-theoretic scrubbing, and first-order approximations to address this challenge.
  • Linearization: Network linearization has developed from an analytical tool in NTK theory into a directly trainable model using efficient Jacobian-vector products.This makes finite linearized networks computationally practical.
  • Linearization: With architectural and training changes, linearized models can match nonlinear-model performance on many vision tasks while retaining a convex loss.This property motivates their use for efficient forgetting.

3. Preliminaries and Notations

The paper represents training data as input-target pairs, models predictions with parameterized functions, and defines empirical risk as average loss over a dataset. It also specifies training updates, vector norms, eigenvalues, and gradient notation.

  • Data and models: A dataset D consists of input-target pairs {x_i, y_i}, where images are example inputs and one-hot vectors are example classification targets.Inputs belong to X and outputs belong to Y.
  • Data and models: The model f_w maps inputs to outputs and is parameterized by a weight vector w in R^d.A deep neural network is one example of such a parameterized function.
  • Loss: The empirical risk L_D(w) is the average per-example loss over the training dataset.The paper also writes L_D(f_w) interchangeably with L_D(w).
  • Training notation: A training algorithm A applied for τ steps to L_D, initialized at w_0, produces the learned weights A_τ(L_D(w_0)).SGD is the paper’s example training algorithm.
  • Notation: The notation also includes the L2 norm, the largest eigenvalue λ_Max(Q), and shorthand for evaluating gradients at a specified weight.These quantities support later optimization and forgetting analyses.

4. The Forgetting Problem

The forgetting problem asks whether a trained network can become indistinguishable from one never trained on selected data, despite deep networks’ non-convexity. The paper evaluates this through readout functions and mutual information, while contrasting noise-based, convex, and quadratic approaches.

  • Problem formulation: A forgetting procedure transforms weights trained on D into weights indistinguishable from training without the target subset D_f.The procedure is modeled as S(w, D, D_F).
  • Evaluation: Readout functions test whether an observer can distinguish weights trained with, without, or scrubbed of D_f.Network confidence on forgotten samples is one example of a readout function.
  • Evaluation: Remaining mutual information I(S(w); D_f) upper-bounds the information extractable by any readout function.This bound is harder to estimate than individual readout success.
  • Trade-offs: Noise-based forgetting trades security against accuracy because increasing Gaussian-noise variance destroys more information while making weights more random.The attainable trade-offs can be represented by a Forgetting Lagrangian.
  • Convex forgetting: For strongly convex losses, perturbed projected-GD can remove initial-data influence, but gradient descent is impractical for large-scale deep networks.The guarantee follows from convergence to the unique minimum.
  • Non-convex forgetting: Earlier non-convex forgetting approximations were insufficient for complete removal and scaled quadratically with training samples and classes.These limitations restricted applicability to smaller datasets.

5. Mixed-Linear Forgetting

Mixed-Linear Forgetting combines nonlinear core weights with linear user weights, using a first-order network approximation to retain accuracy while isolating user-data information for forgetting.

  • Mixed-Linear model: The linearized model approximates fine-tuning the original deep network when the user-weight perturbation is small.The approximation is intended to preserve performance while making forgetting easier.
  • Mixed-Linear model: The model uses nonlinear core weights trained on core data and linear user weights trained on all data.This separates core information from user-data information while allowing the user weights to adapt the model to the user task.
  • Training: The user weights are optimized with a quadratic mean squared error loss, while core weights use a separate standard training loss.The quadratic user-weight problem enables efficient forgetting updates, whereas core weights need not be scrubbed.
  • Accuracy: The Mixed-Linear model achieves comparable accuracy to a fully fine-tuned DNN and outperforms a classifier trained only on last-layer features.This result is reported across the datasets evaluated in Figure 1.
  • Architecture: Replacing ReLUs with Leaky ReLUs improves the accuracy of the linearized model in large-scale vision fine-tuning.A straightforward linearization of a standard pretrained ResNet-50 otherwise performs only slightly better than the last-layer linear baseline.

6. Forgetting procedure

The forgetting procedure removes a selected subset of user data by solving a quadratic update approximately with SGD, then adding noise to suppress residual information while controlling accuracy loss.

  • Forgetting update: Forgetting a subset Df of user data uses the optimal quadratic-loss update based on the remaining data Dr.The core data is excluded from forgetting because Dc ∩ Df = ∅.
  • Efficient computation: The Hessian-based update cannot be stored or inverted for large models, so the procedure recasts it as an optimization problem.The auxiliary loss has the desired forgetting update as its unique minimizer and can be optimized without constructing the Hessian.
  • Optimization: 39?
  • Forgetting procedure: The forgetting update is approximated by training with SGD for τ iterations and applying the resulting change to the user weights.This defines the Mixed-Linear forgetting procedure for the model.
  • Noise trade-off: Adding Gaussian noise destroys information remaining from the approximate forgetting step.Larger σ improves forgetting but can reduce model performance.
  • Sequential forgetting: Sequential requests are handled by repeatedly applying the forgetting procedure to the weights produced by the preceding step.The method explicitly considers preventing excessive performance deterioration across multiple requests.

7. Bounds on Remaining Information

The paper derives computable bounds on information remaining after sequential forgetting requests and studies how noise, iterations, dataset fraction, and model size affect forgetting.

  • Information bounds: Theorem 1 bounds the information remaining in weights after K equally sized forgetting requests computed with τ-step SGD updates.The bound is formulated for convex, L2-regularized loss and tracks the union of forgotten datasets.
  • Noise and accuracy: Increasing noise variance σ^2 reduces leakage from an imperfect forgetting approximation but can reduce model performance.The resulting privacy–accuracy trade-off is illustrated in Figure 2.
  • Iterations and convergence: More forgetting iterations improve accuracy and reduce remaining information, but gains diminish because stochastic optimization variance can dominate.Larger batch size B reduces estimation variance and improves convergence.
  • Data fraction and model size: Forgetting a smaller data fraction r is easier, whereas increasing the parameter dimension d may make forgetting more difficult.This identifies data fraction and model size as scope-dependent factors in the bound and procedure.
  • Computational efficiency: Two to three forgetting epochs can replace 50 training epochs from scratch, providing a 16-25× speed-up per forgetting request.The trade-off is evaluated for forgetting 10% of Caltech-256 and Aircrafts through 10 requests.

8. Experiments

Experiments evaluate ML-Forgetting on large-scale vision tasks using sequential deletion requests and readout-based comparisons with retraining. They show that exact residual-gradient computation, task-relevant core data, and the proposed method improve forgetting outcomes.

  • Experimental setup: Experiments use ResNet-50 across six image-classification datasets, typically forgetting 10% of randomly selected training data through 10 sequential 1% requests.The evaluated datasets are Caltech-256, MIT-67, Stanford Dogs, CUB-200, FGVC Aircrafts, and CIFAR-10.
  • Readout functions: ML-Forgetting produces readout-function values indistinguishable from retraining from scratch, while Original and Fisher baselines fail to match it.The difference is especially visible for the Retrain Time readout, which uses full weight knowledge.
  • Residual-gradient estimation: Complete residual-gradient estimation gives lower test error and information leakage than stochastic estimation when using the same number of steps.The complete estimate benefits from smaller variance and faster convergence; stochastic estimation is equivalent to fine-tuning on remaining data.
  • Privacy evaluation: Membership attacks use model-output entropy, with ideal forgetting requiring attack success comparable to a retrained model.This attack is one of the readouts used to assess residual information about forgotten data.
  • Core-data choice: Core data containing task-relevant information improves both remaining-information reduction and test accuracy on fine-grained datasets.ImageNet plus 30% of Aircrafts performs better than generic ImageNet core data, and increasing the user-distribution fraction in the core improves test accuracy.

9. Conclusion

The conclusion presents ML-Forgetting as an efficient procedure for removing user-data influence through linearization, while identifying regularization and non-convexity as constraints.

  • ML-Forgetting linearizes deep networks so user-data influence can be removed efficiently without compromising accuracy.The quadratic training loss supports efficient subset forgetting, while discarding user weights removes all user-data information.
  • The procedure depends on strong convexity induced by L2 regularization, and efficient direct information removal from highly non-convex networks remains unsolved.The paper also notes that forgetting quality depends on the subset of data being forgotten.
  • The method also handles forgetting an entire class, where the target forget error is 100%.Experiments on Caltech-256 and FGVC-Aircrafts report robustness to different readout functions and outperform other methods in this setting.
  • Increasing L2 regularization improves forgetting by making optimization more strongly convex, but excessive regularization hurts model accuracy.Figure 8 examines the resulting trade-off between remaining information and test error across regularization coefficients.

A.3. More experiments using SGD for forgetting

Additional SGD experiments reproduce the main readout-function experiments across five vision datasets and report consistent results, including whole-class forgetting variants.

  • Consistent results are observed across Stanford Dogs, MIT-67, CIFAR-10, CUB-200, and FGVC Aircrafts.The experiments repeat the Figure 3 protocol across these datasets.
  • Figure 9 repeats the Figure 3 experiments for StanfordDogs.
  • Figures 10–13 repeat the Figure 3 experiments for MIT-67, CIFAR-10, CUB-200, and FGVC-Aircrafts.
  • Figures 14–15 repeat the Figure 2 experiment for StanfordDogs, CUB-200, and MIT67.

B. Experimental Details

The experiments use a linearized final-layer model built from an ImageNet-pretrained ResNet-50, with dataset-specific regularization and short SGD forgetting runs.

  • ML-Forgetting uses an ImageNet-pretrained ResNet-50 and linearizes only its final layers.Training uses MSE loss, scaled one-hot vectors, SGD for 50 epochs, and explicitly added L2 regularization.
  • Weight decay is set to 0.0005 for Caltech-256, StanfordDogs, and CIFAR-10, and 0.001 for MIT-67, CUB-200, and FGVC-Aircrafts.Higher weight decay increases the strong-convexity constant and facilitates forgetting.
  • Forgetting runs use momentum 0.999, halve the learning rate each epoch, and run for 3 or 4 SGD epochs depending on the dataset.Caltech-256, StanfordDogs, and CIFAR-10 use 3 epochs from learning rate 0.01; the other datasets use 4 epochs from 0.025.

C. Theoretical Results

The theoretical section formalizes forgetting for quadratic convex objectives, derives bounds for SGD-based scrubbing, and analyzes repeated forgetting requests under strong-convexity assumptions.

  • The proofs use smoothness, strong convexity, Cauchy–Schwarz, AM-GM, induction, and stochastic-gradient sampling assumptions.The convergence analysis specifically uses smoothness because the quadratic training-loss gradient is linear and unbounded.
  • The analysis assumes convex or quadratic losses with L2 regularization, bounded loss values, smoothness, and strong convexity.These assumptions support distance, convergence, and information-remaining bounds for empirical risk minimizers and SGD.
  • For quadratic losses, the smoothness constant is tied to the maximum Hessian eigenvalue, yielding gradient-bound expressions used in the analysis.The section states G = 2β in one quadratic-loss result and G = β in another smoothness result.
  • The forgetting definitions apply SGD-based scrubbing updates to single and multiple requests, optionally adding Gaussian noise and using remaining-data gradients and Hessians.The procedure compares scrubbed weights with weights retrained on the remaining dataset.
Loading 2012.13431v2…