PaperlayerView exact revision on arXiv

Source-linked AI summary

Selective Amnesia: A Continual Learning Approach to Forgetting in Deep Generative Models

Alvin Heng, Harold Soh

arXiv:2305.10120v2cs.LGcs.AI

TL;DR

Large text-to-image models can generate harmful or inappropriate content, while existing forgetting approaches may require costly retraining or model-specific assumptions. Selective Amnesia uses continual-learning objectives and a user-chosen surrogate distribution to forget concepts across conditional generative models. Experiments report forgetting of classes, celebrities, and nudity while supporting controllable remapping, with limitations in computation, global-concept removal, surrogate selection, and evaluation.

  • Problem

    Existing approaches make selective concept forgetting difficult because dataset filtering is challenging, retraining is costly, and prior methods have narrow model or data requirements.

  • Method

    Selective Amnesia unifies continual-learning objectives with a surrogate distribution to forget specified concepts while retaining others in conditional variational likelihood models.

  • Results

    Selective Amnesia induces customizable forgetting across MNIST, CIFAR10, STL10, and Stable Diffusion concepts including classes, celebrities, and nudity.

  • Takeaways & Limitations

    Users can remap forgotten concepts to semantically relevant images while erasing the target concept across several conditional generative models.

  • Takeaways & Limitations

    The method has expensive Fisher-information computation, is less proficient for global concepts such as nudity, and requires manual surrogate selection.

Abstract

from arXiv · show

The recent proliferation of large-scale text-to-image models has led to growing concerns that such models may be misused to generate harmful, misleading, and inappropriate content. Motivated by this issue, we derive a technique inspired by continual learning to selectively forget concepts in pretrained deep generative models. Our method, dubbed Selective Amnesia, enables controllable forgetting where a user can specify how a concept should be forgotten. Selective Amnesia can be applied to conditional variational likelihood models, which encompass a variety of popular deep generative frameworks, including variational autoencoders and large-scale text-to-image diffusion models. Experiments across different models demonstrate that our approach induces forgetting on a variety of concepts, from entire classes in standard datasets to celebrity and nudity prompts in text-to-image models. Our code is publicly available at https://github.com/clear-nus/selective-amnesia.

1 Introduction

Selective Amnesia addresses harmful misuse of deep generative models by selectively forgetting specified concepts without retraining from scratch. It generalizes continual-learning ideas to conditional generative models and supports user-controlled remapping of forgotten concepts.

  • Motivation: Selective Amnesia targets selective forgetting because filtering massive training datasets and retraining models for each forgotten concept are costly.The motivation includes harmful or inappropriate generations, including deepfakes and content involving real individuals.
  • Research gap: Prior forgetting methods are limited by discriminative-model assumptions, special data or model partitioning, or diffusion-specific design characteristics.The paper seeks a framework applicable to varied pretrained generative models without original training data.
  • Core idea: Selective forgetting is framed as continual learning in reverse: retain generation of B while forgetting generation of A from a model trained on both.This reframes the objective from preventing forgetting to inducing targeted forgetting.
  • Contribution: The method unifies continual-learning techniques into an objective for controllable forgetting and can remap a forgotten concept to a user-defined alternative.The scope covers conditional variational likelihood models, including VAEs and DDPMs.
  • Evaluation: Experiments span MNIST VAEs, CIFAR10 and STL10 DDPMs, and Stable Diffusion, covering forgotten classes, celebrities, and nudity.The reported applications demonstrate the method across models of varying complexity.

2 Background and Related Work

The paper situates Selective Amnesia among variational generative modeling, continual learning, data forgetting, and diffusion-model concept-erasure methods. Its background emphasizes ELBO-based training and EWC and generative replay as the relevant continual-learning tools.

  • Conditional variational models: Conditional VAEs model data and latent variables jointly, using an approximate posterior because the exact posterior is intractable and optimizing an ELBO.The ELBO lower-bounds the conditional log-likelihood.
  • Conditional variational models: Conditional diffusion models generate samples through iterative Markov denoising, with training based on a reweighted variational bound over noise-prediction errors.Sampling begins from Gaussian noise and proceeds through T denoising steps.
  • Continual learning: Continual learning seeks sequential task learning without catastrophic forgetting, using regularization, architectural changes, and data replay.The paper focuses on Elastic Weight Consolidation and Generative Replay.
  • Continual-learning tools: EWC protects parameters important to an earlier task through a Fisher-information-weighted quadratic penalty, while generative replay uses generated prior-task data without storing old datasets.For variational models, the Fisher information is adapted to measure sensitivity on the ELBO.
  • Related work: Existing data-forgetting and generative concept-erasure methods often rely on discriminative settings, special partitions, exact likelihoods, disentangled latent spaces, or diffusion-specific guidance.These restrictions motivate a more general framework for conditional variational models.

3 Proposed Method: Selective Amnesia

Selective Amnesia trains a pretrained conditional generative model to forget designated data while preserving remembered data, without access to the original training set. It combines Bayesian continual-learning ideas, generative replay, and a surrogate distribution whose choice controls the forgotten output.

  • Problem formulation: The problem partitions data into forgotten Df and remembered Dr, seeking a model that forgets Df while retaining Dr without access to the original dataset.The model starts from a conditional maximum-likelihood estimate trained on the full dataset.
  • Bayesian continual learning: The Bayesian formulation maximizes the posterior conditioned on Dr, lowering forgotten-concept likelihood while keeping parameters near the original model through EWC.This provides the continual-learning basis for forgetting.
  • Objective construction: Direct optimization fails because remembered-data performance degrades without replay and variational likelihoods are intractable, so the method adds generative replay and a surrogate objective.The revised objective addresses both shortcomings.
  • Objective construction: Conditional samples from the pretrained model approximate forgotten and remembered distributions, Fisher information, and the replay term, eliminating the need for the original training data.The replay term empirically improves generation of remembered data after forgetting.
  • Naive objective: Naively minimizing forgotten-data likelihood with ELBO substitutions can fail to forget the target and diminish the quality of other classes.Figure 2 contrasts original, naive Eq. 3, and revised-objective training for MNIST digit 0.
  • Surrogate objective: The surrogate objective maximizes likelihood under q(x|cf) rather than the forgotten distribution, guaranteeing reduced forgotten-data likelihood under the stated assumptions.A larger difference between q(x|cf) and p(x|cf) yields a lower achievable forgotten-class likelihood.
  • Surrogate objective: Users can choose q(x|cf), from a uniform distribution to realistic acceptable images, thereby controlling how the forgotten concept is remapped.The final variational objective replaces likelihood terms with their ELBOs; on MNIST, a uniform surrogate removes digit 0 while retaining other digits.

4 Experiments

Experiments show that Selective Amnesia forgets discrete classes and text-conditioned concepts across VAEs and diffusion models, while allowing forgotten concepts to be remapped to user-chosen substitutes. On benchmark datasets, forgetting substantially reduces classifier evidence for the target class, while Stable Diffusion experiments demonstrate controllable celebrity remapping and reduced nudity content, with trade-offs against baselines and image diversity.

  • Discrete-class forgetting: SA evaluates forgetting across MNIST, CIFAR10, and STL10 using a conditional VAE for MNIST and conditional DDPMs for CIFAR10 and STL10.The forgotten concepts are MNIST digit ‘0’ and the ‘airplane’ class in CIFAR10 and STL10.
  • Discrete-class forgetting: SA erases forgotten classes to noise while preserving visually indistinguishable samples for remembered classes, with minor degradation in remembered-class diversity.These qualitative findings are reported for MNIST, CIFAR10, and STL10.
  • Quantitative forgetting: 5.8%: MNIST samples from the forgotten ‘0’ class are classified as ‘0’ after training with λ = 100, versus 96.7% for the original model.Classifier entropy approaches the theoretical maximum of 2.30, indicating that digit information was erased.
  • Quantitative forgetting: Classifier entropy increases significantly for the forgotten CIFAR10 and STL10 classes, although it approaches the 2.30 maximum less closely than for MNIST.The experiments use classifier entropy and forgotten-class probability to assess residual information.
  • Ablations: Generative replay is crucial, while λ values from 10 to 100 provide a fidelity–diversity trade-off and larger values can increase information leakage.The FIM term primarily preserves fidelity; omitting generative replay significantly harms all metrics.
  • Stable Diffusion case studies: SA generates the most images with detected faces in the celebrity comparison, while related-celebrity tests reveal slight concept leakage to individuals with similar attributes.Celebrity resemblance is measured using no-face proportion and GCD Score.
  • Stable Diffusion case studies: SA substantially reduces nudity content on the I2P dataset, but SLD and ESD achieve better scores while SA preserves more semantically related prompt content.The I2P evaluation contains 4703 inappropriate prompts, and nudity is detected with NudeNet at threshold 0.6.

5 Conclusion, Limitations, and Future Work

Selective Amnesia provides controlled concept forgetting in conditional generative models, remapping forgotten concepts to semantically relevant alternatives. The authors identify computational, scope, surrogate-selection, evaluation, and misuse limitations.

  • Conclusion: Selective Amnesia combines continual-learning methods with a surrogate distribution to erase a target concept while preserving semantically relevant generation.The method supports user-specified remapping of the forgotten concept in Stable Diffusion experiments.
  • Limitations and Future Work: The FIM calculation can be expensive for diffusion models because the ELBO requires summing over T timesteps per sample.The paper proposes more efficient and accurate FIM computation as future work.
  • Limitations and Future Work: SA appears more proficient at removing local specific concepts than global concepts such as nudity.The authors call for methods that work well across both types of information.
  • Limitations and Future Work: SA requires manual selection of an appropriate surrogate distribution, motivating future automation.The paper also suggests human assessment for a more holistic evaluation of forgetting capabilities.
  • Broader Impacts: The authors warn that tools such as SA could be used to alter concepts inappropriately or maliciously, including erasing historical events.They emphasize careful community use to avoid propagating further harms.
  • Assumptions: The paper deliberately censors all bare chests, including socially acceptable depictions of topless males in many cultures.This reflects a conservative nudity-censoring choice in the experiments.

A Proofs

The proof develops the training objective by introducing a replay likelihood for remembered data and simplifying posterior terms with Bayes’ rule. The resulting expression combines forgetting, replay, and prior-related terms.

  • Proof setup: The derivation begins from the posterior over remembered data and introduces an extra likelihood term for the remembered distribution on both sides.This yields an equivalent expression containing log p(Dr|θ).
  • Bayesian simplification: Bayes’ rule rewrites the left-hand posterior-plus-likelihood terms into duplicated posterior terms and prior or marginal corrections.The substituted form is then simplified into the next equation.
  • Final objective: The simplified objective contains a forgetting likelihood, a joint remembered-and-forgotten posterior, a remembered-data likelihood, and the model prior.The displayed derivation presents this structure with doubled terms and an additive constant.

A.2 Proof of Theorem 1

Theorem 1 is proved by expressing conditional maximum likelihood as KL-divergence minimization and using an expressive-model assumption for the full dataset. Nonnegative KL terms then constrain the forgetting distribution at the optimum.

  • Lemma 1: Conditional maximum likelihood is equivalent to minimizing the expected KL divergence between the data conditional distribution and the model conditional distribution.The equivalence follows because the data entropy term is independent of θ.
  • Lemma 2: Under the asymptotic expressive-model assumption, maximum-likelihood training on the full dataset learns the underlying conditional distribution exactly.The proof then establishes the corresponding zero expected KL divergence on the forgetting distribution.
  • Lemma 2: Because the KL divergences are nonnegative and their positive weights sum to zero, each individual KL divergence must equal zero.This isolates the forgetting-distribution term used in the theorem.
  • Theorem 1: Theorem 1 assumes a surrogate q(x|c) differing from the original forgetting distribution and defines θq as its conditional maximum-likelihood optimum.The theorem compares expectations under the original and surrogate-optimal models over the data to forget.

A.3 Proof of Corollary 1

The corollary relates surrogate-model performance to the KL divergence between the original forgetting distribution and the surrogate. Its proof substitutes the surrogate optimum into the theorem’s expected log-likelihood gap.

  • Corollary 1: Corollary 1 assumes the surrogate optimum models q(x|c) exactly and states that the theorem’s gap equals the negative KL divergence from the original forgetting distribution to q.The result makes the surrogate mismatch determine the expected log-likelihood gap.
  • Proof: The proof rewrites the expected log-likelihood difference between θq and θ∗ as the negative KL divergence between the original forgetting distribution and the surrogate-optimal model.The second equality uses the assumption that p(x|θq,c)=q(x|c).
  • Experimental context: The experimental classifier evaluation averages forgotten-class metrics over 500 generated images for MNIST, CIFAR10, and STL10.MNIST uses a two-layer CNN, while CIFAR10 and STL10 use finetuned ResNet34 classifiers.

B.2 Stable Diffusion

The Stable Diffusion experiments evaluate Selective Amnesia on celebrity and nudity forgetting, using established baselines and classifier-based measurements. The reported setup emphasizes reproducibility and notes that computational efficiency was not optimized.

  • Forget Celebrities: Stable Diffusion v1.4 is used for celebrity experiments to enable fair comparison with SLD and ESD baselines.The experiments generate 5000 images for both FIM calculation and generative replay.
  • Implementation: The reported experiments did not optimize computational efficiency, although preliminary hyperparameter tuning reduced training to 2 A6000 GPUs and 6 hours.The authors leave further performance gains from tuning to future work.
  • Forget Celebrities: Celebrity forgetting is evaluated on 1000 generated images with the GIPHY Celebrity Detector, which recognizes 2306 celebrities.The detector uses face detection followed by celebrity classification and selects the highest-probability target face when multiple faces appear.
  • Forget Nudity: Nudity forgetting tunes only unconditional layers, targeting the prompts “nudity”, “naked”, “erotic”, and “sexual” for 500 epochs.The experiment uses λ = 50 and the same samples for FIM calculation and generative replay.
  • Forget Nudity: Nudity evaluation uses 4703 I2P images scored by NudeNet with a 0.6 threshold, counting only exposed nudity detections.Manual inspection identified classifier false positives in the evaluation.

C More Results

Additional results quantify nudity reduction and document important evaluation caveats. Selective Amnesia reduces detected nudity relative to Stable Diffusion baselines, while NudeNet produces classification errors and false positives.

  • Evaluation Setup: The I2P benchmark contains 4703 images, and the evaluation considers only nudity classified as exposed.Covered and exposed content are handled separately by NudeNet.
  • Evaluation Caveats: NudeNet often misclassifies gender, causing the reported number of female breasts for Selective Amnesia to overestimate the true number.Manual inspection also found six flagged images without exposed female genitalia, indicating false positives.
  • C.1 Forget Nudity: Table 4 counts exposed nudity detected by NudeNet in 1000 images generated from “a photo of a naked person”.Multiple instances per image can make Armpits and Female Breasts counts exceed the number of images.
  • C.1 Forget Nudity: Selective Amnesia drastically reduces detected nudity compared with Stable Diffusion v1.4 and v2.1, while ESD-u achieves the best overall scores.Selective Amnesia particularly outperforms SLD Medium on Female Breasts and Female Genitalia.

D Additional Samples

Additional samples illustrate forgetting across discrete classes, celebrity prompts, nudity, and artistic styles. The visual examples also show how user-defined replacement concepts shape generated outputs.

  • D.1 MNIST, CIFAR10, STL10: MNIST samples compare the original VAE with Selective Amnesia after forgetting digit ‘0’ using λ = 100 with generative replay.The setting corresponds to the hyperparameters reported in Table 1.
  • D.1 MNIST, CIFAR10, STL10: STL10 samples compare the original DDPM with Selective Amnesia after forgetting the ‘airplane’ class using λ = 10 with generative replay.The configuration matches the Table 1 hyperparameters.
  • D.1 MNIST, CIFAR10, STL10: CIFAR10 samples show three Selective Amnesia variants after forgetting the ‘airplane’ class, corresponding to the reported ablations.The figure compares the original model with the ablated variants.
  • D.1 MNIST, CIFAR10, STL10: Removing generative replay significantly harms remembered-class image quality, while λ = 10 produces better quantitative results than λ = 1.The comparison concerns the CIFAR10 ablations summarized in Table 1.
  • D.2 Celebrities: Celebrity examples use prompts involving Brad Pitt and Angelina Jolie while remapping each forgotten identity to middle-aged or clown concepts.The figures extend qualitative comparisons across tuxedo, facial, park, dress, bouquet, and hat prompts.
  • D.2 Celebrities: For a Maleficient prompt without Angelina Jolie’s explicit name, Selective Amnesia generalizes the forgetting effect to the character portrayal.The replacement concepts are “middle aged woman” and “female clown”.
  • D.3 Nudity and Art Styles: Nudity samples use the prompt “a photo of a naked person”, while van Gogh-style samples are redirected toward pop-art elements.The nudity figure notes conservative censoring; the style figure reports loss of the distinct van Gogh style.

E Effects on Other Celebrities

The study examines how forgetting Angelina Jolie affects other celebrities under full-layer and cross-attention-only training. Interference is generally slight, but similarity between concepts and layer choice create a trade-off.

  • Experimental Setup: The experiment compares SA-Full, which trains all layers, with SA-Xattn, which trains only cross-attention layers, while forgetting Angelina Jolie.Both variants remap the concept to “middle aged woman” and test Jennifer Aniston, Oprah Winfrey, and Jackie Chan.
  • Results: SA-Full causes slight changes for Jennifer Aniston but minimal changes for Oprah Winfrey and Jackie Chan.The authors hypothesize stronger interference for Jennifer Aniston because of greater similarity to Angelina Jolie.
  • Results: Training only cross-attention layers minimizes interference, rendering Jennifer Aniston and the other celebrities as accurately as Stable Diffusion v1.4.This setting is associated with explicitly specified concepts such as celebrity names.
  • Trade-off: For prompts that imply a celebrity without naming them, cross-attention-only tuning limits generalization to those prompts.Unconditional layers support generalization to concepts omitted from the prompt.
  • Trade-off: The recommended layer choice balances generalization against interference: all-layer tuning favors generalization, while cross-attention-only tuning favors minimal interference.The authors identify this balance as a trade-off and leave a more precise study for future work.
Loading 2305.10120v2…