PaperlayerView exact revision on arXiv

Source-linked AI summary

Prompt Stealing Attacks Against Text-to-Image Generation Models

Xinyue Shen, Yiting Qu, Michael Backes, Yang Zhang

arXiv:2302.09923v2cs.CRcs.LG

TL;DR

Creating high-quality prompts is costly, and the paper asks whether an adversary can infer them from generated images, a threat to prompt engineers’ intellectual property and prompt marketplaces. It analyzes prompt-image data and proposes PromptStealer, which infers subjects and modifiers; the attack outperforms three baselines across four similarity measures.

  • Problem

    Prompt creation is costly, and it remains unclear whether and to what extent an adversary can infer a generated image’s prompt, threatening engineers’ intellectual property and prompt marketplaces.

    Source: §1 · pp. 1–2

  • Method

    The paper analyzes a collected prompt-image dataset and proposes PromptStealer, which infers the subject and predicts modifiers before combining them into a prompt.

    Source: §1 · p. 2

  • Results

    PromptStealer outperforms three baselines across semantic, modifier, image, and pixel similarities; it scores 0.70, 0.43, 0.80, and 0.90, versus CLIP Interrogator’s 0.52, 0.01, 0.77, and 0.89.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The study identifies prompt stealing as a threat to prompt engineers’ intellectual property and prompt marketplaces.

    Source: §8 · p. 15

    PromptShield assumes white-box access to the attack model, and adaptive attacks reduce its defense performance.

    Source: §5.1 · p. 13, §5.5 · p. 14

Abstract

from arXiv · show

Text-to-Image generation models have revolutionized the artwork design process and enabled anyone to create high-quality images by entering text descriptions called prompts. Creating a high-quality prompt that consists of a subject and several modifiers can be time-consuming and costly. In consequence, a trend of trading high-quality prompts on specialized marketplaces has emerged. In this paper, we perform the first study on understanding the threat of a novel attack, namely prompt stealing attack, which aims to steal prompts from generated images by text-to-image generation models. Successful prompt stealing attacks directly violate the intellectual property of prompt engineers and jeopardize the business model of prompt marketplaces. We first perform a systematic analysis on a dataset collected by ourselves and show that a successful prompt stealing attack should consider a prompt's subject as well as its modifiers. Based on this observation, we propose a simple yet effective prompt stealing attack, PromptStealer. It consists of two modules: a subject generator trained to infer the subject and a modifier detector for identifying the modifiers within the generated image. Experimental results demonstrate that PromptStealer is superior over three baseline methods, both quantitatively and qualitatively. We also make some initial attempts to defend PromptStealer. In general, our study uncovers a new attack vector within the ecosystem established by the popular text-to-image generation models. We hope our results can contribute to understanding and mitigating this emerging threat.

1 Introduction

Text-to-image prompts combine a subject with modifiers, and crafting high-quality prompts is costly enough to support a marketplace for prompt engineers. This paper studies whether those prompts can be inferred from generated images and introduces an attack designed to recover both components.

1 Introduction

  • Prompt engineering is iterative and costly, while marketplaces let customers buy prompts corresponding to sample images, making prompt stealing a threat to engineers’ intellectual property and marketplace business models.The paper frames the question as whether an adversary can infer a generated image’s corresponding prompt.Source: §1 · pp. 1–2
  • The study collects 250,000 prompt-image pairs from Lexica, retains 61,467 after preprocessing and de-duplication, and finds that successful stealing should recover both subjects and modifiers.The analysis reports that both prompt components matter for generated-image quality.Source: §1 · p. 2
  • 0.70 semantic, 0.43 modifier, 0.80 image, and 0.90 pixel similarities beat CLIP Interrogator at 0.52, 0.01, 0.77, and 0.89, respectively.PromptStealer also produced images similar to target images, showed similar results on real-world marketplace prompts, and improved with adversary feedback or multiple target images.Source: §1 · p. 2
  • PromptStealer infers a prompt’s subject with a subject generator, predicts its modifiers with a detector, then concatenates them into a stolen prompt.The paper also proposes PromptShield, which adds an optimized image perturbation; its effectiveness requires strong defender assumptions and declines under adaptive attack.Source: §1 · p. 2
  • The paper reports following Lexica’s terms, using its official API, disclosing the research to Lexica and marketplaces, and responsibly disclosing the findings.It says PromptBase acknowledged the findings and launched a watermark editor for prompt engineers.Source: §1 · p. 2

2 Background

Text-to-image models generate digital images from natural-language prompts, and Stable Diffusion is the study’s focus. Prompt marketplaces trade prompts because producing high-quality images remains laborious, and their terms identify prompts as engineers’ intellectual property.

2.1 Text-to-Image Generation Models

  • Stable Diffusion is the paper’s focus because it is prevalent, open-source, and supported by an active community that enables large-scale evaluation.The experiments use the official Stable Diffusion v1.4 model for image generation.Source: §2.1 · p. 2

2.2 The Prompt Marketplace

  • Prompt marketplaces let customers browse sample images, buy the corresponding prompts, then generate similar images or change the subject while retaining a style.The paper reports that producing high-quality images remains a laborious and costly manual process.Source: §2.2 · p. 2
  • PromptBase’s top 50 prompt engineers sold about 45,000 prompts for an estimated $186,525 over nine months; marketplace terms identify prompts as engineers’ intellectual property.The paper says prompt stealing therefore violates those rights and jeopardizes marketplace business models.Source: §2.2 · p. 3

3 Preliminary Analysis

The authors construct Lexica-Dataset and analyze prompt structure and modifier patterns. Their analysis indicates that prompt stealing must recover both subjects and modifiers, which shape generated images.

3.1 Data Collection

  • Lexica-Dataset contains 61,467 deduplicated Stable Diffusion prompt-image pairs, collected by querying Lexica with 5,000 prompts sampled from an 80K-prompt dataset.The authors filter unparsable items, grid images, and images from models other than Stable Diffusion; they decompose prompts at commas into subjects and modifiers, standardize some modifiers, and remove style-evocation phrases.Source: §3.1 · p. 3

3.2 Data Analysis

  • 11 modifiers per prompt on average, while subjects average 56 characters (23.63% of prompt length), motivating recovery of both subjects and modifiers.The analysis also reports that modifiers influence image style and detail, while subjects describe the image’s main content.Source: §3.2 · pp. 3–4, Figure 3 · p. 4
  • Only 7,672 modifiers (9.88%) appear more than ten times; excluding flavor, artist modifiers occupy the largest prompt share and influence image style and detail.The paper reports that 75.33% of modifiers belong to flavor and 14.32% to artist; movement modifiers are used least often, with “fantasy art” appearing 1,567 times.Source: §3.2 · p. 4, Table 1 · p. 4
  • Artist modifier embeddings cluster apart from the other categories, while the other four categories are relatively mixed.The authors use CLIP text embeddings for the top 20 modifiers in each category and visualize them with T-SNE; they note that movement and medium modifiers often accompany each other.Source: §3.2 · pp. 4–5, Figure 4 · p. 5

4 Prompt Stealing Attack

PromptStealer targets both a prompt’s subject and modifiers, evaluating its ability to recover prompts and recreate images from target images. It outperforms three baselines on Lexica-Dataset and is also assessed on transferability to new subjects.

4.1 Threat Model

  • The threat model gives an adversary a target image and black-box model access; success includes text, modifier, image, and pixel similarity.The evaluation also considers whether stolen prompts retain the target image’s style when the subject changes.Source: §4.1 · p. 5

4.2 PromptStealer

  • Given a target image, PromptStealer generates a subject and detects modifiers, then concatenates them into the stolen prompt.The subject generator is fine-tuned on Lexica-Dataset images and subjects; the modifier detector adapts a multi-label classifier to the dataset’s modifier labels.Source: §4.2 · pp. 5–6, Figure 6 · p. 6

4.4 Quantitative Evaluation

  • PromptStealer scores 0.70 semantic, 0.43 modifier, 0.80 image, and 0.90 pixel similarity, versus CLIP Interrogator’s 0.52, 0.01, 0.77, and 0.89.Table 2 compares PromptStealer with three baselines on Lexica-Dataset.Source: Table 2 · p. 7, §4.4 · p. 7

4.5 Qualitative Evaluation

  • On human-rated similarity, PromptStealer scores 4.45, versus 3.20 for ImgCap (FT), 2.95 for CLIP Interrogator, and 1.65 for ImgCap.Two labelers rated images on a five-level scale; average inter-rater Cronbach’s Alpha was 0.90.Source: §4.5 · p. 7
  • In examples, PromptStealer recovers many target modifiers, while incorrect CLIP Interrogator modifiers can steer generated images away from the target style.The paper generates four images per stolen prompt with different random seeds for the examples.Source: §4.5 · pp. 7–8, Figure 7 · p. 8
  • When subjects are replaced, images made from PromptStealer’s stolen prompts adapt to new subjects while retaining features similar to images made from the target prompts.Examples replace “city” with subjects such as “village,” “forest,” and “mountain.”Source: §4.5 · p. 8, Figure 8 · p. 8, Figure 9 · p. 8

4.6 Ablation Study

Ablations favor fine-tuned BLIP and ML-Decoder, while open-world tests, multiple-image strategies, and a GPT-4 comparison examine PromptStealer’s performance across settings.

4.6 Ablation Study

  • Fine-tuned BLIP performs best among the subject generators, ML-Decoder exceeds Query2Label, and 7,672 labels with a 0.6 threshold are suitable settings.Semantic similarity is 0.70 with 7,672 labels, versus 0.68 with 1,966 and 0.67 with 1,109; semantic similarity peaks at 0.71 with a 0.3 threshold.Source: §4.6 · pp. 8–9, Table 3 · p. 9, Table 4 · p. 9

4.7 Adversary in the Loop

  • Manually correcting misidentified subjects, including celebrities or animals, improves attack results in the adversary-in-the-loop examples.The authors also suggest adding a subject detector as future work.Source: §4.7 · p. 9, Figure 10 · p. 9

4.8 Open-World Evaluation

  • On DiffusionDB and real-world traded prompts, PromptStealer outperforms three baselines quantitatively and qualitatively; on DiffusionDB it scores 3.95 in human-rated similarity.The DiffusionDB evaluation uses 1,000 sampled prompt-image pairs; the traded-prompt evaluation uses 10 prompts from PromptBase and PromptDB.Source: Table 5 · p. 9, Table 6 · p. 9, §4.8 · p. 10
  • PromptStealer also recovers some modifiers on unseen Midjourney and DALL·E 2 images, but performs worse than on Stable Diffusion; it performs similarly on Stable Diffusion v1.4 and v1.5, and slightly worse on v2.0.The authors could not run large-scale quantitative tests on Midjourney and DALL·E 2; they report a case study instead.Source: §4.8 · p. 11, Table 7 · p. 11

4.9 Performance on Multiple Target Images

  • With multiple target images, the union strategy performs best among random selection, best-prompt selection, and greedy caption-and-modifier union.Providing multiple target images increases attack performance.Source: §4.9 · p. 11, Table 8 · p. 12

4.10 ChatGPT in Prompt Stealing Attacks

  • On ten Lexica-Dataset test samples, PromptStealer outperforms the GPT Store prompt generator on semantic, modifier, image, pixel, and human-rated similarity.The GPT-4-based prompt generator scores 0.35, 0.00, 0.72, 0.91, and 3.2 on those measures, respectively.Source: §4.10 · p. 12

5 Defense

PromptShield perturbs target images to suppress PromptStealer’s modifier detector while preserving human-perceived utility. It reduces similarity most for artist modifiers, but requires white-box access and its defense weakens under adaptive attack.

5.1 PromptShield

  • PromptShield targets modifier detection by removing artist modifiers from the target set and optimizing a perturbation toward that reduced set with I-FGSM.The method uses 100 iterations and ε = 0.2; the authors focus on artist modifiers because they drive image style and relate to artists’ intellectual property.Source: §5.1 · pp. 12–13

5.3 Quantitative Evaluation

  • Artist-modifier similarity falls from 0.49 to 0.06, while semantic similarity falls from 0.70 to 0.62 and image similarity from 0.80 to 0.71.The overall modifier similarity increases; Table 9 reports PromptShield’s effectiveness against PromptStealer.Source: Table 9 · p. 13, §5.3 · p. 13

5.2 Experimental Settings

  • The evaluation measures defense effectiveness with semantic, modifier, image, and pixel similarity, and utility with MSE plus human-rated similarity.Lower MSE indicates higher utility.Source: §5.2 · p. 13

5.3 Quantitative Evaluation

  • The average MSE between target and shielded images is 0.0007, which the authors interpret as an imperceptible perturbation.Lower MSE indicates higher utility in their evaluation.Source: §5.3 · p. 13, §5.2 · p. 13

5.4 Qualitative Evaluation

  • Human-rated similarity falls from 4.45 to 1.85, and Figure 16 illustrates PromptShield preventing recovery of target artist modifiers in an example.The target and shielded images are described as quite similar, demonstrating the defense’s utility.Source: Figure 16 · p. 13, §5.4 · pp. 13–14

5.3 Quantitative Evaluation

  • PromptShield also gives comparable results with C&W and promising results on CLIP Interrogator.The CLIP Interrogator perturbation is evaluated against that optimization-based baseline attack.Source: §5.3 · p. 13

5.5 Limitations

  • The defense assumes white-box access to the modifier detector; transfer to PromptStealer was not promising, and adaptive training reduced defense performance.The adaptive attack trains the modifier detector with shielded images and ground-truth modifiers.Source: §5.1 · p. 13, §5.5 · p. 14

6 Related Work

Related work covers prompt engineering, image-to-text synthesis, and machine-learning security. The paper positions prompt stealing as a threat that these prior lines of work had not directly investigated.

Prompt Engineering

  • Prompt-engineering studies emphasize that proper modifiers matter for image quality, informing PromptStealer’s focus on recovering modifiers [34] [43] [47].The cited work includes prompt-design guidelines, a modifier taxonomy, and a human-in-the-loop search for useful modifier combinations.Source: §6 · p. 14

Image-to-Text Synthesis

  • Image-captioning systems and CLIP Interrogator can be adapted to prompt stealing, but the authors report that these methods do not reverse prompts satisfactorily [31] [39].CLIP Interrogator searches by iteratively calculating similarity between modifiers and the target image.Source: §6 · p. 14

Security Issues of Machine Learning Models

  • Prior machine-learning security work largely focused on classification models, while text-to-image security research had not examined prompt stealing from generated images.The authors describe their study as addressing this emerging threat.Source: §6 · p. 14

7 Discussion

The discussion connects prompt stealing to intellectual-property and marketplace concerns, outlines implications for stakeholders, and identifies limits in the evaluated model and prompt domains.

Legal Debates / Copyright Laws Concerning Prompts

  • The authors argue that user-curated prompts involve human effort and align more closely with traditional copyright concepts than AI-generated images, while prompt ownership remains legally ambiguous.They note that PromptBase and PromptSea state that creators own prompts on their platforms.Source: §7 · p. 14

Societal Impacts

  • Prompt stealing could enable rapid copying of marketplace prompts, threaten marketplace business models, and motivate defenses using imperceptible perturbations; Lexica-Dataset contains 61,467 prompt-image pairs.The dataset also supports modifier analysis for prompt engineers and training attack models.Source: §7 · p. 14

Limitations & Future Work

  • The attack is mainly evaluated on Stable Diffusion; Midjourney and DALL·E 2 receive case studies rather than quantitative evaluation, and applicability to large-language-model prompts remains open.The authors cite the closed-source status of Midjourney and DALL·E 2 as the reason for leaving quantitative evaluation to future work.Source: §7 · p. 15

8 Conclusion

The conclusion presents prompt stealing as a new threat to text-to-image systems and reports PromptStealer’s advantage over three baselines. It highlights the need to recover both prompt subjects and modifiers.

8 Conclusion

  • PromptStealer outperforms three baseline methods in both quantitative and qualitative experiments.The paper’s conclusion reports this overall comparison without listing metric values.Source: §8 · p. 15
  • The dataset analysis indicates that successful prompt stealing must account for both the target prompt’s subject and its modifiers.The study collected Lexica-Dataset and used it for large-scale measurement.Source: §8 · p. 15
  • The authors frame the work as the first investigation of prompt stealing against text-to-image generation models.They conclude that the study sheds light on threats in the ecosystem of popular text-to-image models.Source: §8 · p. 15

A The Impacts of Modifier Counts on CLIP Interrogator

Reducing CLIP Interrogator’s modifier count from 77,616 to 7,637 produced only minimal changes across four similarity metrics; the paper retains 77,616 modifiers for comparison.

A The Impacts of Modifier Counts on CLIP Interrogator

B PromptShield With C&W

PromptShield using C&W reduces PromptStealer’s similarity scores, with the largest modifier-similarity drop for artist modifiers, while producing perturbations with average MSE 0.08.

B PromptShield With C&W

  • Semantic, modifier, and image similarities fell from 0.70, 0.43, and 0.80 to 0.58, 0.21, and 0.74, respectively, under PromptShield (C&W).Source: B PromptShield With C&W · p. 17
  • Artist-modifier similarity dropped most, from 0.49 to 0.13, while the average MSE between target and shielded images was 0.08.The passage describes the perturbation as invisible.Source: B PromptShield With C&W · p. 17
  • The C&W experiments used 100 iterative steps, a learning rate of 0.05, and a loss trade-off hyperparameter of 0.001.Source: B PromptShield With C&W · p. 17

C PromptShield Against CLIP Interrogator

Against CLIP Interrogator, PromptShield with I-FGSM lowers semantic, modifier, and image similarities with minimal perturbation; examples also show incorrect modifier predictions.

C PromptShield Against CLIP Interrogator

Loading 2302.09923v2…