Source-linked AI summary
Ablating Concepts in Text-to-Image Diffusion Models
Nupur Kumari, Bingliang Zhang, Sheng-Yu Wang, Eli Shechtman, Richard Zhang, Jun-Yan Zhu
TL;DR
Large-scale text-to-image models trained on web data can reproduce copyrighted content, motivating efficient concept removal without retraining from scratch. The paper ablates target styles, instances, and images by matching their conditional distributions to anchor concepts, and reports successful removal while largely preserving related concepts.
Problem
Web-scale text-to-image training can include copyrighted materials, artistic styles, personal photos, and memorized images, creating a need to remove target concepts without full retraining.
Method
The method minimizes KL divergence by matching a target concept’s conditional image distribution to an anchor concept through model-based or noise-based training objectives.
Results
The method successfully ablates target concepts across 16 tasks while minimally affecting closely related concepts, taking around five minutes per concept.
Takeaways & Limitations
Concept ablation provides a computational tool for removing copyrighted images and artworks from pretrained image-generative models while preserving surrounding concepts.
Takeaways & Limitations
Ablation does not guarantee that a target concept cannot be generated from distant prompts, and slight degradation of surrounding concepts can still occur.
Abstract
from arXiv · showhide
Large-scale text-to-image diffusion models can generate high-fidelity images with powerful compositional ability. However, these models are typically trained on an enormous amount of Internet data, often containing copyrighted material, licensed images, and personal photos. Furthermore, they have been found to replicate the style of various living artists or memorize exact training samples. How can we remove such copyrighted concepts or images without retraining the model from scratch? To achieve this goal, we propose an efficient method of ablating concepts in the pretrained model, i.e., preventing the generation of a target concept. Our algorithm learns to match the image distribution for a target style, instance, or text prompt we wish to ablate to the distribution corresponding to an anchor concept. This prevents the model from generating target concepts given its text condition. Extensive experiments show that our method can successfully prevent the generation of the ablated concept while preserving closely related concepts in the model.
1. Introduction
The paper targets efficient removal of copyrighted concepts, styles, instances, and memorized images from pretrained text-to-image models without retraining from scratch. It matches target-conditioned image distributions to broader anchor concepts while aiming to preserve related concepts.
- Web-scale training datasets often contain copyrighted materials, creators’ artistic oeuvres, and personal photos.
- The paper asks how to prevent target concepts from being generated efficiently while preserving closely related concepts.The motivating setting includes creator opt-out requests that would otherwise require computationally intensive retraining.
- Concept ablation replaces a target concept’s generated images with outputs matching a broader anchor, such as replacing Van Gogh with painting or Grumpy Cat with cat.
- The method minimizes KL divergence between the target-conditioned distribution and an anchor-conditioned distribution using two alternative training objectives.One objective matches model predictions across target and anchor prompts; the other pairs target prompts with anchor-concept images.
- Across 16 ablation tasks, the method removes target concepts while minimally affecting closely related concepts and takes around five minutes per concept.Experiments cover object instances, artistic styles, and memorized images, alongside studies of objectives, parameter subsets, anchors, steps, spelling robustness, and multiple-concept ablation.
2. Related Work
Related work establishes the problem through advances in text-to-image synthesis, training-data memorization, machine unlearning, generative-model fine-tuning, and model editing. The paper distinguishes its approach from prior removal methods that retrain across the entire dataset.
- Modern text-to-image models combine improved architectures, generative techniques, and large-scale datasets to synthesize high-quality images and compose concepts in unseen contexts.
- Because training commonly uses copyright images, these models can mimic artist styles and other copyrighted content.
- A prior GAN-removal method adds redacted data to fake data and uses adversarial loss, but requires retraining on the entire dataset.
- Prior studies report that text-to-image models can generate exact or similar copies of training data under certain text conditions.
- Machine-unlearning research addresses deletion of training data, while generative-model fine-tuning and model editing modify pretrained models for new concepts, tasks, or specific weights.
3. Method
Concept ablation edits a pretrained diffusion model so target prompts produce images from a related anchor distribution, avoiding full retraining. The method compares model-based and noise-based objectives, regularizes updates, and evaluates parameter choices, convergence, and robustness.
- Concept Ablation: Concept ablation prevents generation of a target concept while preserving performance on closely related concepts.A naive strategy that maximizes diffusion loss on target pairs damages surrounding concepts.
- Formulation: The method matches the target distribution p̂Φ(x|c*) to an anchor distribution p(x|c) using KL divergence.The anchor should be a superset of or similar to the target concept, such as Cat for Grumpy Cat.
- Objective Variants: The model-based objective matches predictions for target and anchor prompts, whereas the noise-based objective trains on target prompts paired with generated anchor images.The model-based variant uses the pretrained model’s anchor distribution; the noise-based variant matches Gaussian noise targets.
- Training Objective: The final objective combines concept-ablation loss with standard diffusion loss on anchor pairs as a regularizer.The regularization term helps retain the anchor concept while modifying the target condition.
- Objective Comparison: The model-based variant converges faster than the noise-based variant and better preserves surrounding concepts than loss maximization.Loss maximization causes deterioration of surrounding concepts, while the model-based method is selected for later experiments.
- Parameter Choices: Embedding and cross-attention fine-tuning converge early, but cross-attention is more robust to small spelling mistakes.Embedding-only fine-tuning can be circumvented by slight prompt misspellings that still generate the target concept.
4. Experiments
Experiments evaluate concept ablation on instances, styles, and memorized images using Stable Diffusion. The method removes target concepts while preserving related concepts, with model-based and parameter-selection comparisons supporting the reported results.
- Experimental setup: The evaluation covers four object instances, four artistic styles, and eight memorized-image examples using Stable Diffusion.Instances include Grumpy Cat, Snoopy, Nemo, and R2D2; styles include Van Gogh, Salvador Dali, Claude Monet, and Greg Rutkowski.
- Evaluation metrics: CLIP Score and CLIP accuracy assess target-concept ablation, while higher surrounding-concept scores indicate better preservation.KID compares fine-tuned and pretrained generations, and SSCD measures similarity to memorized images.
- Comparisons and main results: The model-based variant converges faster and performs better or on par with the noise-based variant, so it is used in later experiments.The comparison covers instance ablation and parameter-subset fine-tuning.
- Comparisons and main results: The method maps target instances to anchor concepts and minimally affects related surrounding concepts, including preserving BB8 while ablating R2D2.Fine-tuning text embeddings performs similarly or better than cross-attention fine-tuning but is less robust to small spelling errors.
- Comparisons and main results: Specific artistic styles are successfully ablated while related surrounding styles are minimally affected.Qualitative style results show similar content to the pretrained model without the unique style.
- Comparisons and main results: The fine-tuned model generates varied outputs instead of memorized samples, with full-weight fine-tuning giving the best results.Table 1 reports the percentage of samples with at least 0.5 SSCD cosine similarity to a memorized image.
- Additional Analysis: Using target-concept images for the reverse KL objective performs marginally better on styles but worse on instances than using anchor-concept images.The instance comparison includes lower CLIP Score on surrounding concepts and similar CLIP Score on the target concept.
5. Discussion and Limitations
The method efficiently removes target concepts, but it may leave alternate prompt pathways and slightly degrade related concepts. The authors frame it as a step toward creator opt-out tools.
- Limitations: The method can fail to remove a target concept expressed through a different, distant text prompt.After ablating Van Gogh, the model can still generate Starry Night; explicitly ablating that discovered concept can address it.
- Limitations: Ablating a target concept can sometimes cause slight degradation in surrounding concepts.The paper illustrates degradation of Monet when ablating Van Gogh.
- Limitations: Users with full model-weight access can re-introduce an ablated concept, and iterative prompt optimization may also recover it without weight access.The authors do not guarantee that either route is impossible.
- Discussion: The reverse KL objective performs better on certain styles and can ablate famous paintings that the forward KL objective cannot without additional steps.This comparison is illustrated in Figures 13 and 14.
- Discussion: The work takes a small step toward creator opt-out by providing a computational tool for removing copyrighted images and artworks from generative models.The stated goal is to remove copyrighted content without retraining the full model from scratch.
A. Model-based concept ablation objective
The derivation reduces the model-based KL-divergence objective to matching predicted noise vectors. This follows from the fixed-variance Gaussian structure of diffusion transition distributions.
- Derivation: Minimizing KL divergence between anchor- and target-conditioned noisy-latent joint distributions reduces to an ℓ2 difference between predicted noise vectors.The derivation establishes the simplified training objective.
- Derivation: The derivation expands the KL objective at a particular diffusion time step.The time-step expansion isolates the conditional transition distributions used in the reduction.
- Derivation: An integral over earlier latent variables equals 1 because it integrates a probability distribution over its domain.This removes those variables from the expanded expression.
- Derivation: Diffusion transition distributions have fixed variance and means linear in the latent and predicted noise, so their KL divergence simplifies to squared mean differences.Variance terms are ignored because they are fixed and not learned.
B. Compositional Concept Ablation
Compositional ablation removes a combination such as “kids with guns” while preserving the individual concepts. Additional analyses examine data quantity and prompt diversity.
- Compositional Concept Ablation: The training setup targets the composition “kids with guns” while using regularization for the individual concepts kids and guns.The regularization is intended to preserve each constituent concept.
- Compositional Concept Ablation: The method generates kid images for the “kids with guns” prompt while preserving samples for the individual gun and kid concepts.The paper reports that the target composition is successfully ablated.
- Results: 0.62 CLIP Score for kids is similar to the 0.63 baseline, while guns reaches 0.52 versus the baseline model’s 0.60.These values are reported for generations using “kids with guns” prompts.
- Additional analysis: Training with 200 images produces similar target-concept performance by 100 training steps but is marginally worse on surrounding concepts.The comparison concerns ablation of Grumpy Cat.
- Additional analysis: Using fewer unique prompts leads to slower convergence when generating 1000 training images for Grumpy Cat ablation.The comparison uses 10 and 50 prompts against the standard 200-prompt setting.
C. Additional analysis
Additional analyses compare objectives, parameter settings, real target images, and baselines. The method generally preserves related or unrelated concepts while maintaining target ablation, with convergence and robustness depending on the setup.
- Real target images: Real target-concept images with captions can successfully support ablation, but require manual labeling and anchor-prompt modification and therefore are not the standard setup.The experiment uses real images for Van Gogh and Grumpy Cat.
- Loss objectives: With full-weight fine-tuning, the model-based objective outperforms baseline and noise-based variants, converges faster, and maintains surrounding-concept CLIP metrics.The maintained metrics are average CLIP Score and CLIP Accuracy.
- Loss objectives: With embedding fine-tuning, model-based and noise-based variants perform similarly and better than baseline, but spelling mistakes can still recover the target concept.The paper identifies embedding fine-tuning as non-robust to small spelling errors.
- Baseline comparisons: The method ablates instances while remaining most similar to the pretrained model on anchor and surrounding concepts, outperforming negative-prompt and SLD baselines in surrounding-concept preservation.The comparisons use the diffusers implementation with the recommended SLD-Medium hyperparameters.
- Prompt generation: Using QLoRA instead of ChatGPT yields similar Grumpy Cat results: target CLIP Score 0.651 versus 0.639 and surrounding-concept score 0.801 versus 0.796.The paper states that lower target CLIP Score and higher surrounding-concept score are better.
D. More qualitative samples
The paper presents qualitative samples of ablated memorized images, styles, and instances, alongside surrounding concepts. These comparisons show varied outputs for memorized-image prompts and comparisons with pretrained generations for styles and instances.
- Ablated memorized-image prompts produce significantly varying images compared with the pretrained model.
- Multiple-style ablation compares fine-tuned models with pretrained outputs on target and surrounding concepts.
- Multiple-instance ablation similarly compares target and surrounding concept generations with the pretrained model.
E. Implementation details
The appendix specifies optimization settings, prompt-generation procedures, and evaluation details for concept and memorization ablation experiments.
- Cross-Attention: The cross-attention variant uses batch size 8, learning rate 2 × 10−6, and 100 training steps for qualitative samples.The noise-based variant uses 200 steps, while the loss maximization baseline uses 50 steps.
- Embedding and Full-weights: The embedding variant uses batch size 8, learning rate 1 × 10−5, and 200 training steps for qualitative samples.Full-weight U-Net fine-tuning uses batch size 4.
- Training and test set prompts: ChatGPT generated 210 object-instance captions, with 200 used for training images and 10 reserved for evaluation.For style concepts, the authors used clip-retrieval to collect prompts.
- Memorization prompts: For eight memorization cases, prompt variations were generated until five prompts produced copied images with less than 30% probability.The authors observed that paraphrased prompts could also generate memorized images with high probability.
F. Societal Impacts
Concept ablation can remove learned styles and personal or copyrighted images, but the same capability may be misused to remove desired concepts or alter expected model behavior.
- Benefits: Ablation can remove styles learned without an artist’s approval and remove personal and copyrighted images from a pretrained model.The passage presents these as benefits of the proposed capability.
- Risks: The capability can also be used adversely by removing desired concepts or changing expected model behavior.The example given is ablating Grumpy Cat and generating Garfield instead.
G. Change log
The change log records added evaluations and appendix experiments, while the appendix documents prompts and qualitative comparisons for multiple concepts, styles, and memorized images.
- G. Change log: Version 3 added comparisons to Safe Latent Diffusion and negative prompting, MSCOCO FID results, and an experiment replacing ChatGPT with an open-source LLM.Version 2 updated Figure 2 and fixed a minor CLIP Score and Accuracy calculation bug.
- Evaluation prompts: The appendix lists ten evaluation prompts for style concepts and ten evaluation prompts for instance concepts.For surrounding or anchor instances, the target concept is replaced in the sentence with the corresponding concept.
- Multiple-concept ablation: Figures 23 and 24 show ablation of four instances or four styles while displaying surrounding concepts for comparison.The fine-tuned model is described as preserving pretrained-like generations for surrounding styles.
- Memorized-image ablation: Figures 25–28 compare ablation results for memorized images, including online media, artworks, posters, photographs, and other named images.The figures identify the paired target images shown in each comparison.
- Target, surrounding, and anchor concepts: Figures 29–36 compare target, surrounding, and anchor generations for four artistic styles and four object instances.The style examples are Van Gogh, Monet, Greg Rutkowski, and Salvador Dali; the instance examples are Grumpy Cat, R2D2, Nemo, and Snoopy.
- Surrounding concepts: Surrounding concepts comprise remaining styles plus Jeremy Mann, or four similar instances selected for generation by pretrained Stable Diffusion.These surrounding concepts are used for comparison with each target concept.
- Memorized-image evaluation: Anchor prompts provide captions corresponding to target prompts that generate memorized images.The appendix includes these prompts for evaluating memorized-image ablation.