PaperlayerView exact revision on arXiv

Source-linked AI summary

IMMA: Immunizing text-to-image Models against Malicious Adaptation

Amber Yijia Zheng, Raymond A. Yeh

arXiv:2311.18815v3cs.CV

TL;DR

Fine-tuning text-to-image models can enable harmful or unauthorized content, while licenses and safety filters can be bypassed. IMMA learns model parameters intended to make malicious adaptation more difficult, and the paper reports resilience across several adaptation targets while maintaining model usability.

  • Problem

    Fine-tuning text-to-image models can generate harmful or unauthorized content, while licenses and safety filters do not reliably prevent misuse.

    Source: §1 · p. 1

  • Method

    IMMA learns model parameters that make adaptation to malicious concepts more difficult while maintaining adaptability for other concepts.

    Source: §1 · p. 2

  • Results

    IMMA makes text-to-image models more resilient against adaptation to malicious concepts while maintaining usability across artistic-style mimicry, erased-concept restoration, and personalized-concept learning.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The authors describe IMMA as an encouraging first step toward safer open-source text-to-image models.

    Source: §6 · p. 14

    The main study focuses on single concepts; preliminary multiple-concept results are less comprehensive.

    Source: §5.3 · p. 14

Abstract

from arXiv · show

Advancements in open-sourced text-to-image models and fine-tuning methods have led to the increasing risk of malicious adaptation, i.e., fine-tuning to generate harmful/unauthorized content. Recent works, e.g., Glaze or MIST, have developed data-poisoning techniques which protect the data against adaptation methods. In this work, we consider an alternative paradigm for protection. We propose to ``immunize'' the model by learning model parameters that are difficult for the adaptation methods when fine-tuning malicious content; in short IMMA. Specifically, IMMA should be applied before the release of the model weights to mitigate these risks. Empirical results show IMMA's effectiveness against malicious adaptations, including mimicking the artistic style and learning of inappropriate/unauthorized content, over three adaptation methods: LoRA, Textual-Inversion, and DreamBooth. The code is available at \url{https://github.com/amberyzheng/IMMA}.

1 Introduction

Malicious adaptation can generate harmful or unauthorized content, while licenses and removable safety filters do not reliably prevent it. IMMA shifts protection from training images to model weights, aiming to impede malicious adaptation while preserving usability.

1 Introduction

  • Open-source text-to-image models and quick adaptation methods lower barriers to generating harmful or unauthorized content, including sexual content and unauthorized copying.The paper groups harmful fine-tuning of models under “malicious adaptation.”Source: §1 · p. 1
  • Licenses and safety checks offer limited enforcement because users can disregard licenses and remove safety filters.The paper cites these limitations as motivation for alternative protections.Source: §1 · p. 1
  • IMMA learns model parameters that perform poorly when adapted to malicious concepts, while aiming to preserve adaptability for other concepts.The introduction illustrates the approach with a model that fails to mimic Picasso’s style.Source: §1 · p. 2, Figure 1 · p. 2
  • IMMA makes text-to-image models more resilient to malicious adaptation while maintaining their usability across tested settings.The experiments cover artistic-style mimicry, restoring erased concepts, and learning personalized concepts.Source: §1 · p. 2
  • Unlike data-poisoning methods that modify images, IMMA protects model weights before adaptation and places the protection burden on open-source model releasers.Data-poisoning methods instead require content creators to modify artwork before release.Source: §1 · pp. 1–2

2 Related Work

Prior work addresses generative-AI misuse through detection, content suppression, removal, and image-protection methods. IMMA instead protects model parameters and seeks poor adaptation initializations through a learning-against-adaptation approach.

2 Related Work

  • Earlier misuse research includes detecting unauthorized training images, suppressing inappropriate generations, and removing inappropriate content from diffusion models.The cited approaches address different stages or forms of misuse mitigation.Source: §2 · pp. 2–3
  • GLAZE, MIST, and EUDP protect images by modifying data so adaptation techniques fail, whereas IMMA protects the model against misuse.The paper characterizes image modification as data poisoning and contrasts it with model immunization.Source: §2 · p. 3, Figure 2 · p. 3
  • Meta-learning studies learning aspects of a learning algorithm, such as an initialization suitable for adaptation or hyperparameters.MAML is cited as using a single gradient-unrolling step to approximate hypergradients.Source: §2 · p. 3
  • Unlike work that learns good adaptation initializations, IMMA aims to learn poor initializations that prevent generative-model misuse.The paper situates its approach within learning against an adaptation method.Source: §2 · p. 3

3 Preliminaries

The preliminaries describe text-to-image diffusion models and several ways to adapt or modify them. These include erasing concepts, adding personalized concepts, and using low-rank updates.

3 Preliminaries

  • A text-to-image diffusion model learns a conditional distribution of images given a concept embedding, denoted p(x|c; θ).Its objective can be formulated via a variational lower bound or a denoising score-matching perspective.Source: §3 · pp. 3–4
  • Concept-erasing methods fine-tune a pretrained model so it no longer generates images corresponding to a target concept.The passage describes replacing the target concept with another concept, commonly an empty token.Source: §3 · p. 4
  • Personalization methods add a novel token and train the model to associate it with a new concept using representative images.DreamBooth fine-tunes a subset of parameters, whereas Textual Inversion optimizes the new word token.Source: §3 · p. 4
  • LoRA adapts pretrained weights with a low-rank update, and the paper reports that it can relearn concepts erased from diffusion models.The update is written as Δ = AB, with rank r much smaller than the weight dimensions.Source: §3 · p. 4

4 Approach

IMMA learns immunized parameters from a pretrained model to make adaptation perform poorly on target concepts. The approach also targets re-learning erased concepts and personalization adaptation before model release.

4.1 Model immunization

  • IMMA uses a bi-level program: adaptation minimizes its loss over ϕ, while the outer task seeks parameters θ that yield poor adaptation performance.The set S specifies which pretrained parameters IMMA trains, and U denotes parameters overlapping between θ and ϕ.Source: §4.1 · p. 5
  • In practice, IMMA takes fixed lower-level update steps to limit computation, reuses the previous ϕ initialization, and optimizes only cross-attention layers.The lower-level task is approximated with a single gradient step; the cross-attention choice reflects their role in mixing concept and image features.Source: §4.1 · pp. 5–6
  • When pretrained and adapted parameters overlap, updating the shared parameters with values from ϕ performed better empirically.The paper reports that the algorithm’s re-initialization and overlap-handling steps both improve immunization quality.Source: §4.1 · p. 6

4.2 Applications of model immunization

  • Erased models can re-learn target concepts in a few LoRA epochs, motivating IMMA to make that re-learning more difficult.Fig. 3 presents IMMA’s result against re-learning after concept erasure.Source: §4.2 · p. 6, Figure 3 · p. 6
  • For personalization with DreamBooth or Textual Inversion, IMMA is intended for use before model release so adaptation fails to learn the unique concept.Source: §4.2 · p. 7

5 Experiments

The experiments evaluate IMMA across erased-concept relearning, personalized concepts, and comparisons with alternative protection methods. Across these settings, IMMA generally impedes adaptation to target concepts, though protecting one erased concept can reduce accuracy on other concepts.

5.1 Immunizing erased models against re-learning

  • For erased models, re-learning artist styles yields consistently positive SGR gaps across eight artists, averaging 21.84% for LPIPS, 5.5% for CLIP, and 23.35% for DINO.Source: §5.1 · p. 8
  • Across ten object classes, IMMA lowers similarity metrics during LoRA adaptation, yielding generations of lower quality or less related to the target object.The supplied passages describe the trend but do not provide the average SGR values.Source: Table 2 · p. 8, §5.1 · p. 9
  • After just three LoRA epochs, ESD without IMMA relearns the target concept, with average target-concept accuracy rising from 1.3% to 45.5%; with IMMA, average accuracy remains 2.2%.Source: §5.1 · p. 9
  • After LoRA adaptation on unsafe prompts, detected nudity changes by −80% with IMMA versus −60% without it, relative to SD V1-4.The evaluation uses the remaining 251 unsafe prompts after selecting separate image sets for relearning and IMMA.Source: §5.1 · p. 10, Figure 7 · p. 10

5.2 Immunizing against personalized content

  • Across ten personalization datasets, IMMA prevents learning target concepts with Textual Inversion, DreamBooth, and DreamBooth LoRA in most reported SGR metrics.The exceptions are “furniture chair” and “lighthouse” with DreamBooth.Source: §5.2 · p. 11, Table 4 · p. 11
  • Positive RSGR values across all ten personalization datasets indicate that IMMA preserves adaptability for other concepts while impeding the target concept.The authors visualize the gap between other-concept and target-concept similarity in Fig. 8.Source: §5.2 · p. 11, Table 5 · p. 11, Figure 8 · p. 11

5.3 Additional discussion

  • MIST and IMMA both prevent Textual Inversion from learning a target concept, but JPEG compression weakens MIST while IMMA remains robust when JPEG images are included in training.The comparison reports a user study for MIST and IMMA’s compression robustness in Fig. 10.Source: §5.3 · p. 12, Figure 10 · p. 13
  • Direct maximization damages adapted image quality, omitting either of two algorithm steps permits target learning, and crossed DreamBooth/Textual Inversion immunization remains effective; multi-concept support is less comprehensive.The authors identify support for multiple concepts as future work and report preliminary appendix results.Source: §5.3 · pp. 13–14

6 Conclusion

IMMA shifts protection from images to pre-trained model parameters, using bi-level optimization to make malicious adaptation more difficult. Experiments support model immunization as a promising approach, though the authors describe it as an encouraging first step.

6 Conclusion

  • IMMA protects pre-trained models from adaptation methods by learning an initialization that makes adaptation more difficult.The paper formulates immunization as a bi-level optimization program.Source: §6 · p. 14
  • The authors report extensive experiments on relearning concepts for erased models and adapting personalized content.Source: §6 · p. 14
  • The authors call model immunization a promising paradigm and IMMA an encouraging first step toward combatting malicious adaptation.They hope further advances will contribute to safer open-source text-to-image models.Source: §6 · p. 14

Appendix

The appendix collects additional quantitative and qualitative results, user-study details, and experimental documentation. It also points readers to interactive results and supplementary materials.

Appendix

  • Section A1 adds quantitative results, including multiple-concept adaptation results in Figures A10 and A11.Source: Appendix · p. 19
  • Section A2 provides additional qualitative results and interactive results in HTML with the supplemental materials.Source: Appendix · p. 19
  • Section A3 documents the conducted user study.Source: Appendix · p. 19
  • Section A4 documents experimental details such as model architecture, hyperparameters, and baseline descriptions; code is also included in the supplementary materials and slated for release.Source: Appendix · p. 19

A1.1 Comparison with data poisoning method

The appendix compares MIST with IMMA using CLIP values after Textual Inversion adaptation. MIST appears to prevent learning some personalized concepts in some datasets, but is less robust than IMMA and fails on others.

A1.1 Comparison with data poisoning method

A1.2 Additional quantitative results of IMMA

Additional results report that IMMA makes adaptation to personalized targets and relearning erased concepts more difficult. The reported results also indicate retained adaptability for other concepts in the personalization setting.

A1.2 Additional quantitative results of IMMA

A1.3 IMMA for multiple concepts

Sequential IMMA protected three target concepts against Textual Inversion, with some reduction in similarity for other concepts. Further experiments examine MIST, face cloning, published personalization sets, user-study design, and implementation details.

A1.3 IMMA for multiple concepts

  • After sequential IMMA on castle, chair, and guitar, Textual Inversion could not learn those concepts, though other-concept similarities dropped more than in the single-concept case.The authors identify extending IMMA to multiple concepts as a possibility and the similarity drop as an area for further research.Source: A1.3 IMMA for multiple concepts · p. 20, Figure A10 · p. 29, Figure A11 · p. 30

A2.1 Comparison with MIST

  • MIST-protected images prevented Textual Inversion personalization, but JPEG compression removed that protection and DreamBooth adaptation also defeated it.The MIST attack used strength 16 and 100 iterations.Source: A2.1 Comparison with MIST · p. 20, Figure A12 · p. 30

A2.2 IMMA on preventing cloning of face images

  • After IMMA on a target person, DreamBooth LoRA lost its capacity to generate images of that person’s identity.The experiment used the face-image datasets from Kumari et al. and the prompt “a photo of [V] person.”Source: A2.2 IMMA on preventing cloning of… · p. 20

A2.3 IMMA on datasets from Dreambooth [39] and Textual Inversion [12] directly reported in their paper

  • Across four image sets, generations with IMMA prevented target-concept content for both DreamBooth and Textual Inversion adaptations.The prompts followed those provided with the corresponding papers; the upper block showed DreamBooth and the lower block Textual Inversion.Source: A2.3 IMMA on datasets from Dreambooth… · p. 21

A2.4 Visualization of generation with negative metrics in Tab. 4

A3 User Study

  • The user study compared image quality and reference similarity for erased-style relearning and personalization, and separately compared MIST with IMMA for personalization.Style trials used four randomly selected training-image references per artist; personalization trials showed four reference images per item, with two generations to choose between.Source: A3 User Study · p. 21

A4 Additional Experimental Details

  • Datasets covered ImageNet object relearning, eight artistic styles, and CustomConcept101 personalization; experiments used Stable Diffusion V1-4 and shared hyperparameters, with IMMA-LoRA training taking 6 minutes and 15GB on one Nvidia A30.Evaluation used ViT-B/32 for CLIP, ViT-S/16 for DINO, and AlexNet for LPIPS; the code builds on Diffusers example code.Source: A4 Additional Experimental Details · pp. 21–22
Loading 2311.18815v3…