Source-linked AI summary
Harnessing LLM to Attack LLM-Guarded Text-to-Image Models
Yimo Deng, Huangxun Chen
TL;DR
T2I safety filters block inappropriate prompts, but adversarial prompts seek acceptance while retaining the original intended image. DACA uses LLM agents to rephrase unsafe prompts as benign descriptions of visual components and reassemble them. The paper reports successful one-time and re-use attacks against DALL·E 3 and Midjourney.
Problem
T2I safety filters block inappropriate prompts, while adversarial prompts seek acceptance and generation of the image intended by the original prompt.
Source: §1 · pp. 1–2Method
DACA uses an image ontology and LLM agents to decompose prompts into visual components, rephrase sensitive descriptions benignly, and reassemble them coherently.
Source: §1 · p. 2, §3 · p. 3, §3.3 · p. 4Results
One-time attack success rates reached 76.7% for DALL·E 3 and 64% for Midjourney, while re-use attack rates reached 98% and 84%, respectively.
Source: §1 · p. 2Takeaways & Limitations
The authors characterize the attack as having non-negligible safety implications and call for more defense efforts.
Source: §1 · p. 2Prompt summarization still allowed over 95% bypass success in empirical tests, although some nuanced visual details could be lost.
Source: §5 · p. 8
Abstract
from arXiv · showhide
To prevent Text-to-Image (T2I) models from generating unethical images, people deploy safety filters to block inappropriate drawing prompts. Previous works have employed token replacement to search adversarial prompts that attempt to bypass these filters, but they have become ineffective as nonsensical tokens fail semantic logic checks. In this paper, we approach adversarial prompts from a different perspective. We demonstrate that rephrasing a drawing intent into multiple benign descriptions of individual visual components can obtain an effective adversarial prompt. We propose a LLM-piloted multi-agent method named DACA to automatically complete intended rephrasing. Our method successfully bypasses the safety filters of DALL-E 3 and Midjourney to generate the intended images, achieving success rates of up to 76.7% and 64% in the one-time attack, and 98% and 84% in the re-use attack, respectively. We open-source our code and dataset on [this link](https://github.com/researchcode003/DACA).
1. Introduction
T2I safety filters aim to prevent inappropriate images, but effective adversarial prompts must be both safe and logically coherent. DACA reframes unsafe prompts as benign descriptions of visual components and automates their rephrasing with an ontology-guided, multi-agent workflow.
1. Introduction
- Safety filters intercept prompts to prevent inappropriate outputs, and an effective adversarial prompt must pass both safety and logical-coherence checks.Token replacements can move a prompt across the semantic safety boundary but also make it illogical and liable to rejection.Source: §1 · pp. 1–2
- DACA breaks an unsafe holistic image description into fine-grained, benign component descriptions while preserving logical coherence and the intended visual effect.Its ontology-guided Decomposer, Polisher, and Assembler automate component extraction, sensitive-term rephrasing, and sentence reassembly.Source: §1 · p. 2
- 76.7% and 64% one-time attack success rates, and 98% and 84% re-use attack success rates, were reported for DALL·E 3 and Midjourney, respectively.The paper also reports cost-effective prompt generation, including 28 prompts per dollar with GPT-4 and up to 83 with Qwen-14B.Source: §1 · p. 2
2. Related Work
Prior work on T2I adversarial prompts has used token substitution and other search strategies, while newer safety filters combine harmful-content checks with logical-coherence checks. This section situates DACA among these attack and defense approaches.
2.1. Adversarial Attack
- Earlier T2I attacks used cross-language word combinations, nonsensical tokens, or iterative sensitive-token replacement; DACA instead directly rephrases visual components.Other work also generated safe images and locally edited them.Source: §2.1 · p. 3
2.2. Defense against Adversarial Prompt
- Safety approaches include blocklists and prompt classifiers, while LLM-assisted filters such as those in DALL·E 3 and Midjourney can also reject chaotic or illogical inputs.Figure 2 depicts DACA’s Decomposer, Polisher, and Assembler as its alternative prompt-rephrasing workflow.Source: §2.2 · p. 3, Figure 2 · p. 3
3. Method
DACA uses an image ontology to guide agents that isolate visual components, rephrase sensitive descriptions, and assemble a coherent prompt. Its workflow applies these operations to component nodes and their associations.
3. Method
- DACA isolates key visual components from a target prompt, articulates them benignly, and reassembles them into a safe drawing prompt.Its three agents are the Decomposer, Polisher, and Assembler.Source: §3 · p. 3
3.1. Agent’s Meta-Prompts
- Each agent’s meta-prompt specifies context, the overall task and sub-steps, and an expected output format; an example demonstration and input may also be supplied.The demonstration is optional.Source: §3.1 · pp. 3–4
3.2. Agent Role Specialization
- The Decomposer extracts characters, clothing, actions, belongings, and background; the Polisher rephrases sensitive elements, and the Assembler forms coherent sentences.The authors found a single-agent approach less effective for semantically rich images and intricate details, motivating the division into three roles.Source: §3.2 · p. 4
3.3. Workflow across Agents
- The ontology represents visual components as nodes and associations as edges; the workflow decomposes each node, identifies sensitive elements and replacements, then assembles associated descriptions.The assembled sentences, including sentences for isolated nodes such as the background, are concatenated into the final prompt.Source: §3.3 · p. 4, Figure 3 · p. 4
4. Evaluation
The evaluation measures attack effectiveness and cost efficiency across a curated, multi-category sensitive-prompt dataset. It tests one-time and re-use attacks, image-to-prompt alignment, and attack costs.
4.2–4.3. Attack effectiveness
- 76.6% one-time and 98.0% re-use average bypass rates are reported for Qwen-Max.The one-time rate is averaged across sensitive categories; the re-use evaluation reports strong stability across repeated generations.Source: Table 1 · p. 5, §4.2 · p. 5, §4.3 · p. 6
4.1. VBCDE Dataset
- The VBCDE dataset contains 100 sensitive prompts across five categories, and the evaluation generated 3,600 adversarial prompts using six LLM backbones.The categories are violence, gore, illegal activities, discrimination, and pornographic content; each category has around 20 prompts.Source: §4.1 · p. 5
4.2. One-time Attack against T2I Models
- For one-time attacks, Qwen-Max achieved the highest reported average bypass rate at 76.6%, followed by GPT-4 at 74.8%.Qwen-14B achieved 46.8%; Midjourney's one-time bypass rate was lower than DALL·E 3's, likely because of stricter prompt scrutiny.Source: §4.2 · p. 5
4.3. Re-use Attack against T2I Models
- 50% of re-used prompts bypassed the filter in all 10 attempts, and every re-used prompt bypassed it in at least 6 attempts.Figure 4 plots per-prompt bypass rate against the proportion of evaluated prompts at that rate.Source: Figure 4 · p. 6, §4.3 · p. 6
4.4. Image Generation Quality
- In the re-use evaluation, CLIP similarity scores were close to or above the 0.274 average from 100 accepted benign prompts.The authors report that re-use scores outperform the one-time case and indicate alignment with the original sensitive prompts.Source: §4.4 · p. 6, Figure 5 · p. 6
- As more sentences from an adversarial prompt were fed to DALL·E 3, similarity to the original prompt gradually increased.Figure 6 also shows representative generated images across sensitive categories, agent backbones, and T2I models; certain categories are omitted.Source: §4.4 · pp. 6–7, Figure 6 · p. 7
4.5. Cost Effectiveness
- GPT-4 costs $0.009 fixed and averages $0.035 per attack, while Qwen-14B can support up to 83 attack attempts.The paper estimates that the GPT-4 cost allows approximately 28 attacks for under one dollar.Source: §4.5 · p. 7
5. Limitations and Discussions
The discussion describes the attack’s cost and reuse potential, then identifies limits in summarization-based defenses and the current image ontology. It also connects those findings to safety implications and possible countermeasures.
Safety Implications
- The attack can generate reusable prompts inexpensively without querying the target T2I model during prompt generation.The cost evaluation reports average token usage and expense; the authors say generated prompts can be reused for multiple image generations.Source: §5 · pp. 7–8, Figure 7 · p. 8
Countermeasures
- Summarization-based defenses remain limited: summarized prompts bypass filters with over 95% success, and gradual sentence-by-sentence feeding makes detection harder.Summarization may lose nuanced visual details; the authors plan to study summarization techniques systematically.Source: §5 · p. 8
Evaluation with More Fine-grained Image Ontology
- Generated images for violence, crime, and discrimination align better with the original prompt than images for the other two sensitive categories.The authors attribute this difference to ontology granularity; bloodiness and eroticism involve detailed sensitive elements, such as blood, that were not thoroughly decomposed.Source: §5 · p. 8
- For violence, separating the performer and recipient of an action effectively conceals sensitive semantics; finer ontology specification is proposed for broader categories and models.The authors also report responsible disclosure and hope the method can support red teaming to identify vulnerabilities.Source: §5 · p. 8
6. Conclusion
The conclusion presents an ontology-guided, LLM-piloted multi-agent method for generating adversarial prompts and reports successful bypasses of DALL·E 3 and Midjourney. It also summarizes the evaluation dataset and attack costs.
6. Conclusion
- The Decomposer, Polisher, and Assembler use an image ontology to isolate visual components, describe them in benign terms, and reassemble them into a safe drawing prompt.The resulting prompts objectively describe component appearances while aiming to bypass safety filters.Source: §6 · p. 8
- The evaluation dataset covers five censored topics and contains 100 sensitive prompts and 3,600 corresponding adversarial prompts.Source: §6 · p. 8
- The method successfully bypasses close-box safety filters of DALL·E 3 and Midjourney.Source: §6 · p. 8
- With GPT-4 as the agent backbone, $1 generates 28 adversarial prompts.Source: §6 · p. 8