PaperlayerView exact revision on arXiv

Source-linked AI summary

When Scene Text Hijacks the Scene: Uncovering, Exploiting, and Mitigating Rendered-Text Semantic Leakage in Image Generation Models

Feifei Li, Runjie Wang, Xiaohan Zhang, Zhenxing Qian, Mi Wen, Mi Zhang

arXiv:2610.11286v1cs.CV

TL;DR

Rendered text can carry semantics that models interpret beyond the intended local text-rendering constraint, a safety issue that remains insufficiently understood. The paper decouples prompt components to measure and analyze leakage, evaluates rendered-text injection, and reports a preliminary mitigation with residual leakage on FLUX-2-dev.

  • Problem

    Rendered text is both a local rendering target and part of the natural-language instruction, but whether its meaning affects non-text regions has been underexplored.

    Source: §1 · p. 2

  • Method

    The paper decomposes text-rendering requests, measures leakage and rendering fidelity, analyzes generation evidence, evaluates rendered-text injection, and tests selective semantic masking.

    Source: §1 · p. 2

  • Results

    Rendered-text semantics can steer non-text regions despite a benign main prompt, and selective masking reduces leakage on FLUX-2-dev while largely preserving text-rendering fidelity.

    Source: §1 · pp. 2–3, §7 · p. 13

  • Takeaways & Limitations

    Rendered text is a semantic control channel as well as visible data, exposing a text-centric cross-modal attack surface in image-generation models.

    Source: §1 · p. 2, §7 · p. 13

    Selective masking remains preliminary: its single-model FLUX-2-dev evaluation leaves substantial residual leakage, while closed-loop exploration is substantially more expensive than single-pass generation.

    Source: §7 · pp. 13–14

Abstract

from arXiv · show

The reliability and accountability of image generative models (IGMs) are essential for building responsible and trustworthy AI systems. Recent IGMs, such as Nano Banana and GPT-Image, now support complex instruction following, realistic image synthesis, and controllable scene-text rendering. As these capabilities expand, safety analysis must also account for new control channels introduced by complex prompts. In this work, we study rendered-text semantic leakage, a largely overlooked phenomenon in open-domain text rendering. Although rendered text is intended to serve as a local visual constraint that should be reproduced verbatim in the generated image, it also carries linguistic semantics that may be interpreted by the model as part of the input instruction. This makes rendered text a potential semantic control channel whose safety implications remain insufficiently understood. We systematically characterize this phenomenon by decoupling the main visual prompt from the rendered text and measuring their individual and compositional effects on generated images. We quantify semantic leakage and rendering fidelity, and further analyze how leakage emerges from intermediate model evidence. We then show that harmful semantics embedded in scene text can persist through LLM-based prompt enhancement pipelines and steer non-text image regions, even when the main visual prompt remains benign. Finally, we propose a preliminary mitigation approach that reduces unsafe semantic transfer from rendered text to non-text regions while preserving the intended text-rendering behavior on FLUX-2-dev. Our findings reveal rendered text as a dual-use carrier of visible data and latent semantics, exposing a text-centric cross-modal attack surface in modern IGMs.

1. Introduction

Rendered text is meant to be a local visual constraint, but models may also interpret its meaning while planning the wider scene. The paper studies this leakage, its exploitation through rendered-text injection, and an initial mitigation.

1. Introduction

  • Rendered-text semantic leakage occurs when the meaning of text intended for local rendering affects non-text regions of the generated scene.The same quoted string is both a rendering target and part of the natural-language instruction the model parses.Source: §1 · p. 2
  • The paper frames rendered text as a hidden semantic carrier when policy-sensitive meaning is preserved for visual rendering and influences regions beyond the text.This risk pathway differs from direct requests for unsafe target images.Source: §1 · p. 2
  • The analysis identifies over-interpretation, misbinding, and interference, and examines denoising trajectories and prompt-embedding ablations to study how leakage propagates.These analyses are conducted in open-source models.Source: §1 · p. 2
  • Selective semantic masking reduces rendered-text injection leakage while largely preserving text-rendering fidelity in the evaluation on FLUX-2-dev.The paper also discusses leakage-aware prompt enhancement as a black-box mitigation direction.Source: §1 · p. 2
  • The evaluation decouples main prompts from rendered text, introduces leakage-oriented metrics, and tests two commercial and four open-source image-generation models under controlled settings.The metrics address rendering fidelity and semantic transfer beyond the text region.Source: §1 · pp. 2–3
  • Rendered-text injection uses policy-sensitive semantics in the rendered-text field to steer non-text image regions despite a benign main prompt, including under prompt-enhanced generation.The study evaluates attacks across visual granularities and safety-critical categories.Source: §1 · p. 3

2. Background

The background contrasts specialized text-rendering methods with native rendering in general-purpose models, then situates leakage alongside semantic-binding failures and image-generation safety attacks. The paper's focus is whether rendered-text meaning can affect global scene composition.

2.1. Text Rendering in Image Generation Models

  • Earlier text-rendering methods treated text as a specialized local target, while native text rendering now appears within general-purpose models' natural-language prompts.The latter setting raises the possibility that rendered-text semantics influence non-text regions.Source: §2.1 · p. 3

2.2. Semantic Binding in Text-to-Image Generation

  • Prior semantic-binding work studies failures to connect concepts with intended attributes, relations, or spatial roles; this paper instead treats native rendered text as a semantic control channel beyond its text region.Prior work on text rendering largely examines position-level binding between strings and their intended carriers.Source: §2.2 · p. 3

2.3. Attacks and Defenses for Image Generation

  • Existing image-generation defenses address unsafe intent at input, model, or output stages, while rendered text presents a distinct, rarely evaluated attack condition when its meaning leaks into the global scene.The paper studies this risk as rendered-text semantic leakage and its exploitation through rendered-text injection.Source: §2.3 · pp. 3–4

3. Problem Statement

The paper models text-rendering requests as a main visual prompt, rendering constraints, and rendered text. It examines how the text can influence non-text regions and specifies attacker and defender capabilities.

3. Problem Statement

  • A text-rendering request consists of a main visual prompt, rendering constraints, and the text to render.The paper formalizes this request structure before defining its adversarial setting.Source: §3 · p. 4

3.1. Rendering Text in Images

  • In MM-DiTs, text and visual representations interact during denoising; the paper partitions rendered-text and non-text regions to assess whether rendered-text semantics propagate beyond the intended text region.The request components are denoted ym, yc, and yr, with corresponding token-index sets Im, Ic, and Ir.Source: §3.1 · p. 4

3.2. Threat Model

  • The attacker can choose the main prompt, rendering constraints, and rendered text, but cannot alter the model or its pipeline; the goal is to place target semantics only in rendered text and induce them in non-text regions.Black-box defenders can intervene through input-side mechanisms, while white-box defenders can intervene on internal conditioning or denoising dynamics.Source: §3.2 · p. 4

4. Methodology: Uncovering Rendered-Text Semantic Leakage

The methodology decomposes text-rendering requests and measures whether rendered-text semantics reach OCR-masked, non-text image regions. It also tests rendered-text injection and analyzes how such semantics propagate during generation. [sic?]

4.1.1. Testing Pipeline.

  • The testing pipeline builds requests from a visual prompt, rendering constraints, and rendered text, then uses OCR masking and goal-specific VQA to score semantic evidence outside the text region.The masking step prevents evaluation from relying on the visible text itself.Source: §4.1.1 · p. 5, §4.3 · p. 7

4.1.2. Target Model Selection.

  • FLUX-2-dev, LongCat, Qwen-Image, and Z-Image-Turbo show stronger OCR word-recall and F1 performance, so the main leakage analysis focuses on these four models.The screening aims to distinguish semantic leakage from failures to render the requested text.Source: §4.1.2 · p. 5, Figure 4 · p. 6

4.2.1. Main Prompts Dataset.

  • The benchmark selects 72 templates: 45 scene-carrier, 21 real-world, and 6 synthetic, drawn from CVTG-2K, TextInVision, and caption-style contexts.The source pool includes 2,375 deduplicated scene-carrier pairs, 1,418 real-world templates, and 6 synthetic templates.Source: §4.2.1 · pp. 5–6, Table 2 · p. 6

4.2.2. Rendered-Text Manipulation Goal Construction.

  • The study constructs 60 local and global goals, verbalizes each as a description, statement, or imperative, and pairs variants with high- or low-compatibility prompts.Compatibility is selected from the top and bottom thirds of CLIP-ranked candidate prompts.Source: §4.2.2 · p. 6

4.3. Leakage Measurement

  • OCR masking removes readable rendered text; Direct TSL measures text-region-excluded alignment, while counterfactual normalization subtracts alignment from matched requests sharing the scene and layout.The matched requests vary rendered text while holding the main prompt and rendering constraints fixed.Source: §4.3 · p. 7
  • Goal-driven SL uses VQA yes/no scores to check a target condition in the non-text region; scores above 0.5 count as successful leakage.Each goal is assessed with two questions, including one that explicitly asks for evidence after ignoring written text.Source: §4.3 · pp. 7–8

4.4.1. Automating Exploit Search.

  • Rendered-text injection keeps the main prompt benign and places target semantics only in the rendered-text field; an adaptive black-box search varies text and context to induce the goal.The tester cannot modify the image generator or evaluator.Source: §4.4 · p. 8, §4.4.1 · pp. 8–9

4.4.2. Validation and Feedback Reflection.

  • Pre-generation validation rejects main prompts that directly imply the target, while reflection uses evaluator feedback to revise requests and tracks over-interpretation, misbinding, and mutual interference.Attempts with readable text remaining after masking are treated as invalid for semantic-leakage measurement.Source: §4.4.2 · p. 9, Figure 7 · p. 9

5. Evaluation

Across six open-source and commercial systems, rendered text produces measurable semantic leakage into non-text regions. Leakage varies with target dimension, wording, and visual context, and remains difficult to remove through LLM rewriting.

5. Evaluation

  • Evaluation covers four open-source models and two commercial systems using TSL-60, RTI-Safety, OCR-based fidelity, NSL, and goal-driven GSL.Images are evaluated after OCR-based text-region masking; SL success generally requires a goal-driven score above 0.5.Source: §5 · pp. 9–10

5.1. Cross-Model Prevalence of Semantic Leakage

  • All six systems show positive NSL (0.055–0.078); four exceed the Avg-GSL threshold, and Max-GSL ranges from 0.689 to 0.962.Matched base images remain below the 0.5 GSL threshold, while qualitative examples show text semantics appearing outside the text region.Source: §5.1 · p. 10, Figure 9 · p. 10
  • Strong leakage occurs with normally rendered, misspelled, and missing target words; commercial systems average 0.92 normal rendering, consistent with over-interpretation beyond rendering failure.The evidence also identifies misbinding and mutual interference, while overall leakage persists across measurement types and architectures.Source: §5.1 · pp. 10–11

5.2. Factor Analysis of Semantic Leakage

  • Leakage is stronger for entity presence, attribute binding, interaction, scene setting, and global appearance than for counting, spatial relations, or camera framing.Direct prompting achieves 96.9% average success across nine dimensions, indicating the evaluated goals are generally within the open-source models’ generation capabilities.Source: §5.2 · p. 11
  • Leakage remains above threshold across verbalizations, with prompt-like forms scoring highest for open-source models and commercial systems showing high best-attempt scores across all forms.The tested forms are visual descriptions, image-level statements, and imperative instructions.Source: §5.2 · p. 11
  • High-compatibility contexts amplify leakage across model families, while low-compatibility contexts reduce but do not eliminate it; one effective formulation can expose a controllable channel.High-compatibility contexts make target semantics plausible without directly requesting them.Source: §5.2 · pp. 11–12
  • Direct TSL exceeds matched Base TSL, but verbalization bias in Base TSL makes NSL a diagnostic and goal-driven GSL necessary to verify realized target conditions.Some text forms align more naturally with base images even without leakage.Source: §5.2 · pp. 11–12

5.3. RTI under LLM Rewriting

  • After GPT-5.4 rewriting, RTI retains 87.5% PUR and 52.5% UR, versus 42.7% average direct-prompt PUR and 34.5% conventional UR.RTI PUR falls 10.8 percentage points, while average direct-prompt PUR falls 57.3 points; RTI UR exceeds the conventional mean by 18.0 points.Source: §5.3 · p. 12

6. Mechanism and Mitigation

Analysis of FLUX-2-dev indicates that rendered-text semantics enter shared prompt representations and influence scene formation before glyphs become legible. Selective semantic masking reduces leakage while largely preserving text-rendering fidelity, with some utility degradation.

6. Mechanism and Mitigation

  • Ablating FLUX-2-dev’s final prompt-embedding block most weakens semantic alignment and text rendering, with the largest WR drop (ΔWR= −0.190); scene semantics emerge before glyphs are legible.The prompt embedding comprises three 5120-dimensional blocks, and the final block appears to concentrate rendered-text semantics.Source: §6 · p. 13
  • Selective semantic masking suppresses rendered-text semantics during the first 20 denoising steps, then restores them later for glyph rendering.The intervention targets embedding dimensions identified through prompt-embedding ablation.Source: §6 · p. 13
  • On FLUX-2-dev, masking reduces RTI leakage rate from 97.5% to 65.0% while preserving comparable WR, though WR degrades moderately and more on multi-text CVTG100.The authors attribute CVTG100’s larger drop to its multiple rendered-text spans depending more heavily on the intervened representations.Source: §6 · p. 13, Table 9 · p. 13

7. Conclusion

The study finds that rendered text can steer non-text image content, including safety-relevant content, even when the main visual prompt is benign. Its mitigation remains preliminary, and the authors call for further work on leakage-aware evaluation and safety mechanisms.

7. Conclusion

  • Rendered text can influence non-text image regions and steer scene generation even when the main visual prompt remains benign.Across commercial and open-source systems, the behavior affects diverse visual dimensions and can induce safety-relevant content.Source: §7 · p. 13
  • Agentic closed-loop exploration improves controllability and attribution but costs substantially more than single-pass image generation.A systematic evaluation of alternative agent backbones and their cost-performance trade-offs remains future work.Source: §7 · p. 13
  • Selective semantic masking reduces RTI leakage from 97.5% to 65.0% on FLUX-2-dev, leaving substantial residual leakage.The authors describe this single-model evaluation as a proof of concept for partial mitigation.Source: §7 · p. 14
  • The authors call for future research on leakage-aware evaluation, semantic attribution, and safety mechanisms that preserve intended text rendering while preventing unintended semantic transfer.Source: §7 · p. 14

A.1.1. Preprocessing Collected Main Prompts.

  • The benchmark retains native scene-carrier pairings, yielding 2,375 unique pairs across 530 scenes and 297 carriers.It also selects 21 real-world templates from a deduplicated pool of single-text prompts.Source: §A.1.1 · p. 16

A.1.2. Additional Experimental Settings.

The appendix specifies the models and settings used for leakage evaluation, prompt rewriting, and safety classification, then reports validation of the leakage metrics. At the selected threshold, GSL has high precision and stronger agreement with human scores than NSL.

A.1.2. Additional Experimental Settings.

  • CLIP-ViT-B/32 evaluates DTSL/NSL and compatibility, Qwen3-VL-32B evaluates GSL, and Qwen3-30B drives agent planning, verification, and reflection.Local experiments used eight NVIDIA RTX 4090 GPUs; commercial-model evaluation on TSL-60 cost about $0.05 per GPT-Image-2 image and $0.08 per Nano-Banana-2 image.Source: §A.1.2 · p. 16

A.1.3. LLM Rewriting and Safety Classification.

  • GPT-5.4 rewrites prompts, while Qwen3-8B classifies prompt safety as a second text guard alongside LlamaGuard.The rewriting prompt asks to preserve scene, composition, style, actions, and other user-specified details; the classifier flags listed disallowed content categories.Source: §A.1.3 · p. 16

A.2.1. SL Validation.

  • At t = 0.5, GSL precision and recall were 97.41% and 82.06%; mean correlation with SLhuman was 0.806 for GSL versus 0.606 for NSL.Across thresholds 0.4, 0.5, and 0.6, precision was 95.36%, 97.41%, and 97.40%, while recall was 87.15%, 82.06%, and 81.70%.Source: §A.2.1 · p. 16

A.2.2. Image Safety Validation.

The appendix describes human validation of automated image-safety evaluation and presents qualitative examples of GSL judgments and RTI-Safety behavior. The supplied captions identify the comparisons shown but do not report their outcomes.

A.2.2. Image Safety Validation.

  • Two annotators labeled 560 Tab. 8 images benign, uncertain, or harmful, with only harmful mapped to unsafe; Table 10 reports unsafe rates across models.Source: §A.2.2 · pp. 16–17, Table 10 · p. 17
  • Figure 14 shows GSL judgments at t = 0.5, including a false negative from a stricter physical-presence interpretation and rarer OCR-masking and annotation disagreements.Source: Figure 14 · p. 17
  • Figure 15 presents GSL across six settings.Source: Figure 15 · p. 17
  • Figure 16 gives qualitative RTI-Safety examples, while Figure 17 indicates that prompt enhancement can amplify rendered-text semantic leakage.Source: Figure 16 · p. 17, Figure 17 · p. 17

A.3.1. Runtime Analysis.

A full RTI agentic generation-and-evaluation cycle averages 90.0 seconds, with image generation accounting for most of the time. The supplied figures and ablation-table captions identify additional evaluation scopes but provide no corresponding outcomes.

A.3.1. Runtime Analysis.

  • 90.0 seconds is the average full RTI generation-and-evaluation cycle: 62.4 seconds for image generation and 27.6 seconds for evaluation.Mean runtime ranges from 45.6 seconds on Z-Image-Turbo to 152.1 seconds on FLUX-2-dev; the underlying image-generation system is the primary efficiency constraint.Source: §A.3.1 · p. 17
  • Figure 18 compares per-dimension binary GSL performance across thresholds and Pearson correlations of NSL and GSL with the soft human score.Source: Figure 18 · p. 18
  • Table 12 summarizes RTI ablation configurations and results across four open-source generators; Table 13 covers prompt-embedding ablation on FLUX-2-dev across timesteps.Source: Table 12 · p. 18, Table 13 · p. 18

A.3.2. RTI Ablation of Search and Black-box Rewriting.

The ablations show that GPT-5.4 rewriting and online search each reduce average UR, while neither prompt-level rewriting variant eliminates the rendered-text control channel.

A.3.2. RTI Ablation of Search and Black-box Rewriting.

  • 52.5% to 74.2% average UR when GPT-5.4 rewriting is removed (GR vs. NR), showing that rewriting suppresses but does not eliminate harmful visual semantics.Source: §A.3.2 · p. 18
  • 52.5% to 37.9% average UR with search removed under GPT-5.4 rewriting (GR vs. NS-GR), and 74.2% to 61.7% without rewriting (NR vs. NS-NR).Source: §A.3.2 · p. 18
  • 72.5% UR remains with QR-SL despite its instruction to ignore rendered-text semantics, versus 80.4% UR with QR.Source: §A.3.2 · p. 18
  • QR-SL reduces lexical leakage by 30.0pp relative to QR (47.5% vs. 77.5%), but substantial unsafe generation remains.Source: §A.3.2 · p. 18

A.3.3. Mitigation Ablation.

The ablation examines prompt-embedding interventions and selects an early masking window based on the timing of semantic leakage and rendering fidelity.

A.3.3. Mitigation Ablation.

  • DTSL peaks at step 20, while WR continues improving before stabilizing around step 30.Source: §A.3.3 · p. 18
  • Masking is restricted to the first 20 steps to limit semantic transfer while preserving later-stage text rendering.Source: §A.3.3 · p. 18
  • Tab. 13 reports the full prompt-embedding ablation results discussed in Sec. 6.Source: §A.3.3 · p. 18
  • Fig. 20 visualizes token-patch attention in FLUX-2-dev MM-DiT joint-attention layers.Source: Figure 20 · p. 18

A.3.4. Attention Analysis for SL.

The attention analysis links leakage cases to text-token attention reaching object patches; the accompanying ethics and review material records the study’s safeguards and reviewers’ assessment.

A.3.4. Attention Analysis for SL.

  • In leakage cases, attention extends beyond text patches to the generated dog object, suggesting rendered-text tokens are reused for scene-level semantic grounding.Source: §A.3.4 · p. 18

Appendix B. Ethics Considerations

  • The authors reported RTI behavior to OpenAI and Google on August 24, 2026; as of September 15, 2026, they had received no substantive response.Source: Appendix B · p. 19
  • Researchers manually inspected approximately 560 safety-related samples each; harmful RTI and provider-specific attack prompts were not publicly released.Source: Appendix B · p. 19

Appendix C. Meta-Review

  • The meta-review is identified as prepared by the program committee for the 2027 IEEE Symposium on Security and Privacy.Source: Appendix C · p. 19

C.1. Summary

  • The meta-review summarizes rendered-text leakage as changing non-text regions and describes RTI as an attack that bypasses prompt-rewriting safety filters.Source: §C.1 · p. 19

C.2. Scientific Contributions •

  • The review categorizes contributions as a new tool, an impactful vulnerability, a new research direction, and a step forward in an established field.Source: §C.2 · p. 19

C.3. Reasons for Acceptance

  • Reviewers cite a TSL-60 benchmark spanning 60 goals over 9 dimensions and report that early denoising-step embedding interventions suppress leakage while retaining text-rendering fidelity.Source: §C.3 · p. 19
Loading 2610.11286v1…