Source-linked AI summary
Typographic Attack Against VLM-based AI-generated Image Detection
Eunmin Lee, Jungwoo Kim, Jong-Seok Lee
TL;DR
VLMs can use semantic cues to classify AI-generated images, but typographic overlays may bias authenticity judgments, and evidence on this vulnerability has been limited. This paper systematically evaluates attacks across VLM families, modes, and directions. It finds differing vulnerability by mode and direction, and that higher clean accuracy does not ensure lower attack success.
Problem
Typographic attacks may bias VLM-based AIGI judgments, but prior evidence included only one VLM and one file-path overlay for fake-to-real evasion.
Source: §1 · p. 1Method
The study evaluates four adapted typographic attacks and a random-text control across detection-oriented, open-weight, and commercial VLMs in both attack directions.
Source: §1 · p. 1Results
Reasoning modes generally show higher ASR, attack effectiveness varies markedly by direction, higher clean accuracy does not ensure lower ASR, common image transformations do not reliably mitigate attacks, and truth-aligned typography recovers selected errors.
Source: §5 · p. 4Takeaways & Limitations
VLM-based AIGI detectors need to distinguish overlaid authenticity claims from visual evidence.
Source: §5 · p. 4The experiments identify greater vulnerability in reasoning modes but do not fully explain why they are often more susceptible.
Source: §5 · p. 4
Abstract
from arXiv · showhide
Vision-language models (VLMs) are increasingly used for AI-generated image (AIGI) detection, providing natural-language explanations for authenticity judgments. However, their ability to interpret text within images may also expose these judgments to misleading semantic cues. We systematically evaluate typographic attack strategies across detection-oriented, open-weight, and commercial VLMs, considering both real-to-fake and fake-to-real attacks. Our results show that reasoning modes generally exhibit greater vulnerability than direct modes and that attack effectiveness exhibits pronounced directional asymmetry. Moreover, larger models tend to exhibit higher clean detection accuracy but also higher attack success rates. We further examine attack robustness under image and text transformations and investigate whether overlays indicating the correct class can aid error correction. Together, these analyses characterize how typographic attacks influence authenticity judgments and expose limitations of current VLM-based AIGI detection systems.
1. INTRODUCTION
VLMs support AIGI classification and explanations, but authenticity judgments may be vulnerable to misleading text embedded in images. This work systematically evaluates typographic attacks across models, directions, and transformations, finding vulnerability patterns that vary by inference mode and attack direction.
1. INTRODUCTION
- A prior study tested one VLM and one file-path overlay only for fake-to-real evasion, leaving attack variation across models, designs, and directions unclear.The paper frames its evaluation as the first systematic assessment of typographic vulnerability in VLM-based AIGI detection.Source: §1 · p. 1
- The study evaluates four adapted attacks against detection-oriented, open-weight, and commercial VLMs in both directions, alongside a random-text control.It also examines image and text transformations and applies correct-class overlays to initially misclassified images.Source: §1 · p. 1
- Reasoning modes are generally more vulnerable than direct modes, effective cues differ by attack direction, and larger models can have higher clean accuracy while also showing higher attack success rates.The paper also assesses persistence under JPEG compression, downsampling, multilingual perturbations, and typographical errors, and probes whether correct-class overlays can aid error correction.Source: §1 · p. 1
2. RELATED WORK
Prior VLM-based detectors combine authenticity classification with natural-language explanations, while prior typographic attacks manipulate models through misleading text and graphical cues. The cited AIGI-detection demonstration was limited to a file-path overlay against GPT-4o.
2. RELATED WORK
- VLM-based detectors use artifact descriptions, reinforcement learning, or planning and self-reflection to support authenticity detection and explanations.Examples include FakeVLM, Ivy-xDetector, BusterX++ [8], and Veritas [9].Source: §2 · p. 1
- Prior typographic attacks target VLMs with misleading labels, deceptive descriptions, instructions, and non-class text or graphical cues; an early AIGI-detection demonstration used file-path overlays against GPT-4o [18].This paper evaluates such vulnerabilities across general-purpose and detection-oriented VLMs.Source: §2 · p. 1
3. EXPERIMENTAL SETTINGS
The experiments compare adapted typographic attacks across detector families and datasets, using controlled image subsets and shared overlay settings. They also specify inference procedures and how attack success is counted.
3.1. Attack Methods
- Four adapted attacks—Class Label, Instruction, File Path, and Logo—are compared with a random-text control.The control uses two fixed-length words, while the attack-specific overlays include authenticity labels, response directives, target-label paths, or selected logos.Source: §3.1 · p. 2
3.2. VLM Detectors
- The evaluated detectors comprise specialized AIGI detectors, open-weight VLMs, and commercial VLMs, with direct and reasoning modes compared for models supporting both.The groups include Ivy-xDetector, Veritas++, BusterX++, Qwen3.8-24B, GLM-4.6V-Flash, GPT-5.4, and Claude Sonnet 5.Source: §3.2 · p. 2
3.3. Evaluation Datasets
- Ivy-Fake evaluation uses 1,250 real and 1,250 fake test images, while the GenImage subset has 909 real and 1,000 fake images, each at least 512 pixels in width and height.Clean-correct Ivy-Fake analyses use 100 real and 100 fake images (200 total); each model-specific clean-misclassified subset contains 100 images.Source: §3.3 · p. 2
3.4. Implementation Details
- Text is placed top-center at 0.06 × min(W, H) pixels and 0.95 opacity; locally hosted models use greedy decoding, and ASR counts attacks on clean-correct images.Invalid or incomplete outputs count as unsuccessful attacks.Source: §3.4 · p. 2
4. ANALYSIS
Across VLM families, typographic attack success varies with inference mode and attack direction. Scale, input transformations, correct-class overlays, and rendering choices further characterize that vulnerability.
4.1. Overall Results
- 42 of 48 matched comparisons favored higher ASR in reasoning mode, while Class Label and Instruction consistently outperformed Random Text across evaluated model families.Instruction also outperformed Random Text with the same character count and rendering settings, indicating that attack-specific semantics contribute beyond visual occlusion.Source: §4.1 · p. 2
- File Path generally achieved the highest real-to-fake ASR, whereas Instruction or Logo was strongest for fake-to-real; no model family was uniformly robust.File Path was more effective for real-to-fake attacks across models except Ivy and Veritas++; examples suggest models may accept paths implying AI generation but dismiss authenticity claims.Source: §4.1 · p. 3, Figure 2 · p. 3
4.2. Model Scale and Vulnerability
- The 4B Qwen model achieved 90% accuracy on real images and 50% on fake images, while the 27B model achieved 80% and 88%, respectively.Either the 9B or 27B model attained the highest ASR in seven of ten settings; vulnerability increased more from 4B to 9B, with less consistent differences from 9B to 27B.Source: §4.2 · pp. 3–4, Table 3 · p. 3
4.3. Attack Robustness
- JPEG compression and downsampling raised ASR, but accompanying clean-image errors suggest weakened visual evidence may explain the increase; text-perturbation effects varied by attack and direction.Translation sharply reduced real-to-fake ASR for Class Label and Instruction, while Instruction and File Path retained substantial fake-to-real success across languages; typos progressively weakened Instruction but left File Path comparatively stable.Source: §4.3 · p. 4, Figure 3 · p. 4
4.4. Amicable Aid
- All four correct-class aids exceeded Random Text in average recovery, with Instruction best on Ivy at 74.0% and Class Label best on Veritas++ at 61.0%.Their respective advantages over Random Text were 36.0 pp and 46.0 pp, supporting the finding that semantic agreement matters beyond text presence alone.Source: §4.4 · p. 4, Table 4 · p. 4
4.5. Ablation on Typography Attributes
- Pooled ASR rose from 22.69% to 36.63% as font-size fraction increased from 0.02 to 0.10, and from 12.50% to 34.31% as opacity increased from 0.05 to 0.95.Position had a modest, model-dependent effect, with pooled ASR ranging from 34.31% to 37.94%; a font-size fraction of 0.10 frequently caused rendering failures.Source: §4.5 · p. 4, Table 5 · p. 4
5. CONCLUSION
The evaluation finds substantial typographic susceptibility across VLM-based AIGI detectors, with vulnerability varying by inference mode and attack direction. Higher clean accuracy does not ensure lower attack success, and transformations do not reliably mitigate attacks; the discussion notes that the reasons for reasoning modes’ susceptibility remain unexplained.
5. CONCLUSION
- Reasoning modes generally exhibit higher attack success rates, and attack effectiveness varies markedly by direction.The discussion says the experiments do not fully explain why reasoning modes are often more susceptible or what determines attack success and failure.Source: §5 · p. 4
- Higher clean accuracy does not ensure lower attack success, and common image transformations fail to reliably mitigate attacks.Source: §5 · p. 4
- Truth-aligned typography can recover selected detection errors, while detectors need to distinguish overlaid authenticity claims from visual evidence.Source: §5 · p. 4
- The experiments do not fully explain why reasoning modes are often more susceptible or what determines attack success and failure.The discussion identifies these mechanisms as subjects for further investigation.Source: §5 · p. 4