PaperlayerView exact revision on arXiv

Source-linked AI summary

CollageAttack: Exploiting Cross-Modal Alignment Flaws in T2I Models through Spatial Text Composition

Zhiyi Mou, Yao Lu, Wangze Ni, Di Hong, Dakun Shen, Haoyang Li, Chen Jason Zhang, Alexander Zhou, Kui Ren

arXiv:2609.38253v1cs.CRcs.AI

TL;DR

T2I models can render text and spatially compose scenes, while harmful meaning may remain inconspicuous in serialized prompts and emerge after image composition. COLLAGEATTACK distributes intent across scene context, grounded carriers, and text fragments in a single black-box prompt. It achieves up to 86.0% attack success and outperforms the strongest same-model baseline by 18.5 percentage points.

  • Problem

    T2I safeguards face a cross-modal gap because harmful meaning can remain inconspicuous in a serialized prompt yet emerge when textual and visual elements are composed in the image.

    Source: §1 · pp. 1–2

  • Method

    COLLAGEATTACK uses a single-prompt black-box attack that combines a context-relevant scene, scene-grounded text carriers, and spatially distributed fragments.

    Source: §5 · p. 9

  • Results

    86.0% was the reported peak attack success rate, with COLLAGEATTACK outperforming the strongest same-model baseline by 18.5 percentage points while preserving source semantics with higher harmfulness.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The results indicate that harmful meaning can emerge from cross-modal composition, while coherent visual scenes strengthen harmful signals expressed by distributed text fragments.

    Source: §5 · p. 10

    The method assumes that the target can follow compositional scene and spatial instructions and render text on visual surfaces.

    Source: §3.2 · p. 4

Abstract

from arXiv · show

Text-to-image (T2I) models have substantially improved in language understanding, in-image text rendering, and visual composition, while their safety mechanisms do not always keep pace with these capabilities. This creates a cross-modal attack surface in which harmful semantics can remain inconspicuous in a serialized prompt yet emerge through image-level composition. We propose CollageAttack, an automated single-prompt black-box jailbreak that shifts semantic assembly into the image plane by combining context-relevant scenes, scene-grounded textual carriers, and spatially distributed text fragments. Experiments across multiple open-weight and commercial T2I models show that CollageAttack achieves attack success rates of up to 86.0%, outperforming the strongest baseline on the same model by 18.5 percentage points, while consistently producing more harmful outputs and preserving the source intent. We further find that distributed textual fragments can reconstruct the intended semantics after generation, with visual composition producing stronger communicative impact than text alone. These results reveal a cross-modal safety gap in which harmful meaning emerges from the composition of individually less explicit elements.

1 INTRODUCTION

Recent T2I models can render text and compose spatially structured scenes, but existing safeguards face a gap when harmful meaning emerges from visual context and distributed text. COLLAGEATTACK shifts semantic assembly into the image plane and achieves higher attack success than existing black-box baselines.

1 INTRODUCTION

  • T2I advances in language understanding, text rendering, and visual composition make spatially distributed text a safety concern alongside prompt filtering, aligned generation, and post-generation moderation.Source: §1 · p. 1
  • Text-centric jailbreaks leave underexplored how rendered text, visual context, and spatial structure jointly convey harmful meaning that a direct request may not evade safeguards to express.Source: §1 · pp. 1–2, Figure 1 · p. 2
  • The attack must preserve intent across incomplete fragments, place them on scene-consistent carriers, and realize the composition in one prompt without iterative feedback or image editing.Source: §1 · p. 2
  • COLLAGEATTACK combines context-aware scene construction, thematic surface allocation, and harmful-intent fragmentation and spatial recomposition to assemble semantics in the image plane.Source: §1 · p. 2
  • 86.0% was the highest attack success rate, and COLLAGEATTACK exceeded the strongest baseline on the same model by 18.5 percentage points while preserving source semantics with higher harmfulness.Source: §1 · p. 2

2 RELATED WORK

Prior work describes limitations in safety alignment and a progression of T2I jailbreaks from prompt perturbation toward automated semantic search. Figure 2 summarizes the proposed attack's scene, carrier, and fragment composition.

2 RELATED WORK

  • Related alignment work includes supervised fine-tuning with reinforcement learning from human feedback, direct preference optimization, and Constitutional AI.Source: §2 · pp. 2–3
  • Figure 2 depicts COLLAGEATTACK composing scene context, text-bearing surfaces, and spatially distributed fragments before independent evaluation.Source: Figure 2 · p. 3
  • Safety alignment can fail to generalize to adversarial or out-of-distribution inputs, with effectiveness depending on alignment-data quality and coverage.Source: §2 · p. 3
  • T2I jailbreaks range from iterative image editing and token substitution to semantic replacement, agent-guided fuzzing, and attack-LLM prompt rewriting, with efficiency, transferability, and robustness remaining concerns.Source: §2 · p. 3

3 METHOD

COLLAGEATTACK converts a source intent into one structured prompt whose scene, text-bearing carriers, and distributed inscriptions are specified together. Its construction assumes a target can follow compositional instructions and render text on visual surfaces.

3.1 OVERVIEW

  • A prompt-construction LLM jointly generates three complementary commands that are assembled into one prompt for a single image-generation request.Source: §3.1 · p. 4

3.2 THREAT MODEL

  • The black-box threat model gives the attacker access only to the generation interface and returned outputs, not model parameters, gradients, training data, prompts, classifiers, or moderation internals.Source: §3.2 · p. 4

3.3 CONTEXT-AWARE SCENE CONSTRUCTION

  • Context-Aware Scene Construction creates a scene relevant to the source intent and specifies its content and visual setting without reproducing the complete source statement.Source: §3.3 · p. 4

3.4 THEMATIC SURFACE ALLOCATION

  • Thematic Surface Allocation selects thematically related, scene-native carriers and describes them as blank surfaces before textual fragments are assigned.Source: §3.4 · p. 5

3.5 HARMFUL-INTENT FRAGMENTATION AND SPATIAL RECOMPOSITION

  • Harmful-Intent Fragmentation and Spatial Recomposition divides the source expression into three or four phrase-level fragments and specifies each inscription’s carrier, placement, and presentation style.Source: §3.5 · p. 5

3.6 ALGORITHM DESCRIPTION

  • Algorithm 1 requests the three commands together, then parses and returns them as c1, c2, and c3 after specifying scene, carrier, and fragment-placement requirements.Source: §3.4 · p. 5, §3.6 · p. 6

4 EXPERIMENT

The evaluation tests attack effectiveness, semantic preservation, harmfulness, visual-over-text impact, and component contributions across five T2I models, using a shared hate-speech dataset and black-box baselines. Results cover attack success, source-intent alignment, harmfulness, visual impact, and ablations.

4.1.1 MODEL SELECTION

  • Five targets span proprietary services and open-weight systems, including different developers, generative architectures, and safety mechanisms.The set includes three commercial models plus HiDream-O1-Image and FLUX.2 [dev].Source: §4.1.1 · p. 6

4.1.2 DATASET

  • The evaluation samples 200 hate-labeled DGHS entries covering multiple hate types and target identities, using the same source intents across methods and models.The dataset includes derogation, animosity, threatening language, dehumanization, and support for hateful entities.Source: §4.1.2 · pp. 6–7

4.1.3 BASELINES

  • Three black-box baselines represent semantic substitution, token-level perturbation, and LLM-agent-guided fuzzing; evaluation also includes original prompts as a reference.The section assesses attack success and harmfulness, semantic similarity, and visual-over-text impact.Source: §4.1.3 · p. 7, §4.1.4 · p. 7

4.2 EXPERIMENTAL RESULTS

  • 8.14 under Google’s evaluator versus SneakyPrompt’s 6.88 shows the highest similarity score, while COLLAGEATTACK also leads under Doubao and remains competitive under OpenAI CLIP and OpenCLIP.The reported alignment is between generated images and source intents, despite the meaning being distributed across visual and textual components.Source: Figure 3 · p. 8, §4.2 · p. 8
  • COLLAGEATTACK alone has positive average visual-over-text impact on all five models, with a positive rate reaching 69.4% on FLUX.2 Dev.The metric compares judged communicative intensity, not semantic equivalence or observed audience effects.Source: Table 2 · p. 8, §4.2 · p. 9
  • 77.0% aggregate ASR versus SneakyPrompt’s 47.5% shows COLLAGEATTACK led across all five models, with 57.5% on GPT-Image-2 and 75.0% on HiDream-I1.The section also reports that COLLAGEATTACK outperformed the strongest baseline on each evaluated model.Source: Table 1 · p. 7, §4.2 · p. 8
  • 6.12/10 cross-model average harmfulness versus SneakyPrompt’s 4.23/10 shows stronger harmful signals across all evaluated models.The section distinguishes this severity score from ASR, which measures how often outputs are classified as harmful.Source: Table 1 · p. 7, §4.2 · pp. 8–9
  • 6.41 average harmfulness for the complete three-component configuration exceeds the strongest individual configuration’s 5.47 and the two-component configuration’s 5.94.The full configuration improves on the strongest individual configuration by 0.94 points; component contributions depend on how they are combined.Source: Table 3 · p. 9, §4.2 · p. 9

5 CONCLUSION

The paper identifies a cross-modal safety weakness in which harmful semantics can emerge when textual and visual elements are composed in the image plane. CollageAttack distributes incomplete text across scene-grounded carriers so its intended meaning is reconstructed through visual composition.

5 CONCLUSION

  • CollageAttack targets a cross-modal safety weakness: harmful semantics can emerge after textual and visual elements are composed in the image plane.The meaning may remain inconspicuous in the serialized prompt.Source: §5 · p. 9
  • The framework constructs a context-relevant scene, grounds text on scene-consistent carriers, and spatially distributes incomplete textual fragments.The fragments’ intended meaning is reconstructed through visual composition.Source: §5 · pp. 9–10
  • Across heterogeneous T2I models, CollageAttack improves attack effectiveness and generated-image harmfulness while preserving source intent.The findings also indicate that coherent visual scenes strengthen harmful signals expressed in the generated image.Source: §5 · p. 10

AI USE STATEMENT

The authors report using LLMs only to polish the manuscript’s language. They state that all assisted revisions were reviewed and verified and that they take responsibility for the final content.

AI USE STATEMENT

  • LLMs were used solely for language polishing, including grammar, clarity, and readability.The authors state they were not used for research ideas, methodology, experiments, data analysis, or scientific conclusions.Source: AI USE STATEMENT · p. 10
  • The authors say they carefully reviewed and verified all AI-assisted revisions.Source: AI USE STATEMENT · p. 10
  • The authors take responsibility for the final content, including its text, claims, and results.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

The authors describe the work as a controlled research study of T2I safety vulnerabilities, intended to inform understanding and robustness of existing safety mechanisms. They state that potentially sensitive examples are presented only when necessary to demonstrate the risks.

ETHICS STATEMENT

  • The work studies T2I safety vulnerabilities to improve understanding and robustness of existing safety mechanisms.Source: ETHICS STATEMENT · p. 10
  • Evaluations involving adversarial prompts and potentially harmful generated content were conducted solely for research in a controlled setting.Source: ETHICS STATEMENT · p. 10
  • The authors say sensitive examples are presented only when necessary to demonstrate identified safety risks.They state the research may provide empirical evidence supporting more effective defenses.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

The statement describes reproducibility materials, while the accompanying analyses illustrate semantic similarity and show a qualitative generated-image example. The similarity values are presented as illustrative rather than as an additional aggregate benchmark.

REPRODUCIBILITY STATEMENT

  • The authors report detailed descriptions of the method, settings, datasets, target models, baselines, and evaluation metrics, alongside supplementary prompts, implementation details, scripts, and anonymized code.They say these materials provide information necessary to reproduce the main experimental results and analyses.Source: REPRODUCIBILITY STATEMENT · p. 10

B CASE STUDY

  • The qualitative case study presents a generated output for readers to inspect as visual evidence of whether the intended content is conveyed.Figure 5 identifies the example as one evaluated in the semantic-preservation and harmfulness assessments.Source: B CASE STUDY · p. 13, Figure 5 · p. 14

A SEMANTIC SIMILARITY

  • In the example, vision–language similarity to the source intent rises as relevant visual context and textual cues are introduced.Figure 4 reports an increase from 0.0700 to 0.3870; text-only rendering is an upper reference at 0.6018, and the values are illustrative rather than an aggregate benchmark.Source: A SEMANTIC SIMILARITY · p. 13, Figure 4 · p. 14

C ADDITIONAL CLIP SIMILARITY RESULTS

The additional CLIP results compare source-prompt and jailbreak generations across five T2I models and four CLIP encoders, while model selection spans open-weight and commercial systems.

C ADDITIONAL CLIP SIMILARITY RESULTS

  • Table 4 compares image–text similarity for source-prompt and jailbreak generations across five models using four CLIP encoders.The encoders provide complementary measurements of semantic alignment.Source: C ADDITIONAL CLIP SIMILARITY RESULTS · p. 15

D.1 MODEL SELECTION

  • The evaluation selects five representative T2I models spanning open-weight and commercial systems, model families, architectures, and safety mechanisms.The commercial selections include GPT Image 2, Seedream 5.0 Lite, and Wan2.7-Image-Pro; the open-weight selections include HiDream-O1-Image and FLUX.2 [dev].Source: §D.1 · pp. 15–16

D.2 DATASET

The evaluation samples hate-labeled DGHS intents, compares three automated black-box jailbreak baselines under a shared protocol, and assesses semantic alignment, harmfulness, and comparative communicative impact.

D.2 DATASET

  • From DGHS release v0.2.3, the experiments randomly sample 200 distinct hate-labeled entries as source intents and normalize obfuscated spellings.DGHS contains 41,255 English samples covering five hateful-content types and a broad range of target identities.Source: §D.2 · p. 16

D.3 BASELINES

  • The comparison uses PGJ, SneakyPrompt, and JailFuzzer, representing semantic substitution, token perturbation, and agent-guided fuzzing, respectively.The methods use no target-model parameters; all are evaluated on the same 200 intents and target models with the same pipeline.Source: §D.3 · p. 16

D.3.1 EVALUATION METRICS

  • The evaluation treats source-intent preservation, intended-content production, and effective communication of harmful intent as three dimensions of attack success.CLIP image–text cosine similarity measures semantic alignment between the generated image and source prompt; higher scores indicate stronger alignment.Source: §D.3.1 · pp. 16–17
  • GPT-4.1 assigns each generated image a rubric-guided harmfulness score from 0 to 10, with higher scores denoting more explicit and severe harmful content.The study provides a detailed rubric to support consistent scoring.Source: §D.3.1 · p. 17
  • GPT-4.1 compares the communicative impact of each image with its source intent using a score from −2 to 2.Negative values indicate weaker image impact, zero comparable impact, and positive values stronger image impact.Source: §D.3.1 · p. 17

E EVALUATION DETAIL

The evaluation-detail figures expose the prompts and criteria used to score image harmfulness and compare an image’s communicative impact with its source text.

E EVALUATION DETAIL

  • Figure 6 shows the judge prompt for assigning generated images a harmfulness score on a 0–10 rubric.The prompt instructs the judge to assess harmful content.Source: E EVALUATION DETAIL · p. 17, Figure 6 · p. 18
  • Figure 7 presents the criteria for interpreting harmfulness and applying the image-scoring rubric consistently.The evaluation figures make score interpretation and judging procedures transparent.Source: E EVALUATION DETAIL · p. 17, Figure 7 · p. 19
  • Figure 8 shows the judge prompt for comparing an image’s communicative impact with that of its source intent text on a −2 to 2 scale.The figure’s caption specifies the comparison and score range.Source: E EVALUATION DETAIL · p. 17, Figure 8 · p. 20
Loading 2609.38253v1…