PaperlayerView exact revision on arXiv

Source-linked AI summary

Imperceptible Jailbreaking against Large Language Models

Kuofeng Gao, Yiming Li, Chao Du, Xin Wang, Xingjun Ma, Shu-Tao Xia, Tianyu Pang

arXiv:2510.05025v1cs.CLcs.AIcs.CR

TL;DR

Textual jailbreaks have generally relied on visible prompt changes, leaving imperceptible textual attacks less explored than image attacks. This paper appends invisible variation-selector suffixes and optimizes them with chain-of-search; it reports high success across four aligned LLMs and 100% ASR in prompt injection.

  • Problem

    Textual jailbreaks generally modify malicious questions visibly, unlike image attacks that often use imperceptible perturbations.

    Source: §1 · p. 1

  • Method

    The method appends tokenizer-encoded but visually invisible variation selectors to malicious questions and optimizes suffixes through a bootstrapped chain-of-search pipeline.

    Source: §1 · p. 1

  • Results

    The method achieves high attack success rates against four aligned LLMs without visible prompt modifications and reaches 100% ASR in prompt injection.

    Source: §1 · p. 2, §4.4 · p. 9

  • Takeaways & Limitations

    Invisible variation selectors constitute a class of adversarial characters that can be used to jailbreak aligned LLMs and generalize to prompt injection.

    Source: §1 · p. 2, §5 · p. 10

    The invisible suffixes could potentially be mitigated by perplexity-based filtering, while harmful outputs may be detected by harmfulness probes or output filters.

    Source: D POTENTIAL LIMITATIONS AND FUTURE… · p. 15

Abstract

from arXiv · show

Jailbreaking attacks on the vision modality typically rely on imperceptible adversarial perturbations, whereas attacks on the textual modality are generally assumed to require visible modifications (e.g., non-semantic suffixes). In this paper, we introduce imperceptible jailbreaks that exploit a class of Unicode characters called variation selectors. By appending invisible variation selectors to malicious questions, the jailbreak prompts appear visually identical to original malicious questions on screen, while their tokenization is "secretly" altered. We propose a chain-of-search pipeline to generate such adversarial suffixes to induce harmful responses. Our experiments show that our imperceptible jailbreaks achieve high attack success rates against four aligned LLMs and generalize to prompt injection attacks, all without producing any visible modifications in the written prompt. Our code is available at https://github.com/sail-sg/imperceptible-jailbreaks.

1 INTRODUCTION

Textual jailbreaks have generally relied on visible edits, but variation selectors can add tokenizer-encoded characters without changing how a prompt appears on screen. The paper pairs these invisible suffixes with a chain-of-search pipeline and reports high attack success rates across four aligned LLMs, including in prompt injection.

1 INTRODUCTION

  • Existing textual jailbreaks generally insert human-perceptible characters, unlike image attacks that often use imperceptible perturbations.This holds for both manually designed templates and automated algorithms.Source: §1 · p. 1
  • Variation selectors form invisible suffixes that occupy textual space and can be encoded by LLM tokenizers despite leaving the rendered prompt unchanged.The paper repurposes these Unicode characters, originally designed to specify glyph variants, as adversarial suffixes.Source: §1 · pp. 1–2
  • The chain-of-search pipeline optimizes suffixes by maximizing the likelihood of target-start tokens, then reuses successful suffixes and tokens on previously failed questions.The procedure starts from randomly initialized suffixes and candidate target-start tokens and repeats across multiple search rounds.Source: §1 · pp. 1–2
  • The attacks achieve high success rates against four aligned LLMs without visible prompt changes and also generalize to prompt injection.The paper describes the prompts as visually identical to the original malicious questions while being tokenized differently.Source: §1 · p. 2, Figure 1 · p. 2

2 RELATED WORK

Prior jailbreaks include both manually crafted prompts and automated optimization, while prior imperceptible text attacks have targeted tasks other than jailbreaking aligned LLMs. This paper uses variation selectors as an alternative invisible perturbation for inducing unsafe outputs.

2 RELATED WORK

  • Manual jailbreaks use crafted prompts, whereas automated approaches use optimization or LLM assistance; GCG optimizes suffixes and I-GCG uses multi-coordinate updates and easy-to-hard initialization.Simple adaptive attacks combine a crafted prompt with random search to elicit an affirmative token.Source: §2 · p. 2
  • Variation selectors leave standard characters visually unchanged, providing an invisible alternative to the visible suffixes used by simple adaptive attacks and I-GCG.Simple adaptive attacks use a crafted template and visible suffix; I-GCG appends a visible non-semantic suffix.Source: §2 · p. 3, Figure 2 · p. 3
  • Prior imperceptible text attacks use encoding perturbations, homoglyphs, or control characters for tasks such as machine translation, toxicity classification, sentiment analysis, and AI-text detection.This work instead targets jailbreaks of aligned LLMs by inducing unsafe outputs.Source: §2 · p. 3

3 IMPERCEPTIBLE JAILBREAKS

The method exploits variation selectors that can remain invisible in ordinary text while being retained and encoded by LLM tokenizers. It optimizes suffixes through random search and bootstraps successful suffixes and target-start tokens across failed questions.

3 IMPERCEPTIBLE JAILBREAKS

  • The paper introduces imperceptible jailbreaks using Unicode variation selectors and a chain-of-search pipeline to construct prompts.The method appends invisible variation-selector suffixes to malicious questions.Source: §3 · p. 3

3.1 VARIATION SELECTORS

  • Unicode defines 256 invisible variation selectors, and applying them to standard characters such as alphabetic letters produces no visible difference on screen.The selectors include 16 original codepoints (U+FE00–U+FE0F) and 240 supplementary codepoints (U+E0100–U+E01EF).Source: §3.1 · pp. 3–4
  • For over 90% of selectors, token ID length is 3 for Llama-3.1-Instruct-8B and 4 for other LLMs.The table reports token ID length proportions for 256 variation selectors under different tokenizers.Source: Table 2 · p. 4
  • Variation selectors persist through copying and pasting and are typically encoded by LLM tokenizers as fixed multi-token blocks, allowing invisible suffixes to alter model inputs.The paper concatenates multiple selectors to malicious questions to form imperceptible jailbreaks.Source: §3.1 · p. 4

3.2 CHAIN OF SEARCH

  • Random search modifies M contiguous selectors at random positions and accepts changes that raise the target-start token’s log-likelihood; successful suffixes and target tokens seed later rounds on failed questions.The pipeline initializes suffixes and candidate target-start tokens, searches for T iterations per question, and repeats the bootstrap for R rounds.Source: §3.2 · p. 4
  • Unlike simple adaptive attacks, this method needs instance-specific suffix initialization and target-start tokens; unlike I-GCG, it uses random search and invisible selectors rather than gradient-guided optimization and visible suffixes.Figure 3 contrasts the visible suffixes in both baselines with the visually unchanged prompts produced by the proposed method.Source: §3.2 · p. 5, Figure 3 · p. 5

4 EXPERIMENTS

Across four aligned LLMs, the invisible-suffix method achieves higher ASRs than GCG, TAP, and PAP while prompts remain visually unchanged; it also reaches 100% ASR in prompt injection. The experiments examine model-specific search behavior, suffix settings, attention, and embeddings.

4.1 EXPERIMENTAL SETUPS

  • Evaluation uses 50 AdvBench questions across four open-source models, five jailbreak baselines, and unmodified and random-suffix controls, with GPT-4 requiring a perfect 10/10 score for success.The suffix length is 800 selectors for three models and 1,200 for Llama-3.1; each iteration changes 10 contiguous selectors, with five search rounds.Source: §4.1 · pp. 5–6

4.2 MAIN RESULTS

  • The method achieves higher ASRs than GCG, TAP, and PAP across four models while leaving prompts visually unchanged.Both unmodified questions and randomly inserted variation selectors yield significantly lower ASRs.Source: Table 3 · p. 6, §4.2 · pp. 6–7

4.3 ABLATION STUDY

  • Across successful jailbreaks, Vicuna-13B-v1.5’s “To”, “Here”, and “Sure” collectively account for over 50% of all successful jailbreaks; Llama-2-Chat-7B has dominant “Sure” and “Here”, while Llama-3.1-Instruct-8B favors “To” and “I”.Mistral-7B-Instruct-v0.2 often begins task-structured responses with “1” or “Title”.Source: §4.3 · p. 7
  • Most successes occur by rounds one or two for Vicuna and Mistral, but rounds two to four for Llama-2 and Llama-3.1; Llama-3.1 also needs more restarts.Vicuna tends to succeed within three restarts, whereas Llama-2 and Mistral typically require up to ten.Source: §4.3 · p. 7, Figure 5 · p. 7, Figure 6 · p. 7
  • For Mistral-7B-Instruct-v0.2, L=800 and M=10 yield the highest log-likelihood improvement, while Llama-3.1-Instruct-8B benefits from L=1,200 with M=10, balancing search capacity and optimization efficiency.Source: §4.3 · p. 8, Figure 7 · p. 8

4.4 DISCUSSIONS

  • Under jailbreak prompts, attention shifts from harmful phrases such as “how to hack” toward the invisible suffixes.The original question receives uniformly low attention under the jailbreak prompt.Source: §4.4 · p. 8, Figure 8 · p. 8
  • t-SNE visualizations show separated embedding clusters for original malicious questions and jailbreak prompts across the four models.The paper reports that invisible variation selectors alter internal embedding features despite their imperceptible appearance.Source: §4.4 · p. 8, Figure 9 · p. 9
  • 100% ASR is reported for prompt injection, including success in the first round with randomly initialized suffixes.Evaluation used 50 samples, a single “Spam” target-start token, one inference without restarts, and measured whether the attacker’s task displaced the user’s task.Source: Table 4 · p. 9, §4.4 · p. 9

5 CONCLUSION

The conclusion presents invisible variation selectors as a way to create visually unchanged jailbreak prompts and describes the chain-of-search optimization used to construct suffixes. Experiments across four aligned LLMs report high attack success rates, and the paper frames the findings as a vulnerability in current alignment mechanisms.

5 CONCLUSION

  • Invisible variation selectors are appended to malicious questions to create adversarial suffixes without visible changes.The conclusion characterizes these attacks as a novel class of jailbreaks.Source: §5 · p. 10
  • The chain-of-search pipeline reuses successful suffixes and target-start tokens across multiple rounds of random search.The conclusion says this bootstrapped approach enables transfer of adversarial components across diverse malicious questions.Source: §5 · p. 10
  • Experiments on four aligned LLMs achieve high attack success rates while the resulting prompts remain visually identical to their original forms.The paper describes this as a vulnerability in current LLM alignment mechanisms.Source: §5 · p. 10

ETHICS STATEMENT

The ethics statement says the study aims to raise awareness of limitations in current alignment and was conducted in controlled laboratory settings. It acknowledges potential misuse while supporting transparency as a means of promoting responsible research and secure deployment.

ETHICS STATEMENT

  • All experiments were conducted in controlled laboratory settings using publicly available models and benchmarks.The study involved no human subjects or private data.Source: ETHICS STATEMENT · p. 10
  • The authors identify potential misuse risks and state that they do not endorse real-world deployment of the attacks.Source: ETHICS STATEMENT · p. 10
  • The authors argue that transparency can foster responsible research and promote secure LLM deployment.They state that the work’s primary objective is to raise awareness of current alignment limitations.Source: ETHICS STATEMENT · p. 10

A ALGORITHM OUTLINE

Algorithm 1 initializes a suffix and target-start-token pool, then searches over mutated invisible suffixes for each malicious question. It retains successful suffix–token pairs across rounds and returns prompts that produce harmful outputs.

A ALGORITHM OUTLINE

  • The algorithm takes malicious questions, search iterations, an effective-pair pool, initial target-start tokens, and suffix and mutation settings as inputs.Its output is the set of successful jailbreak prompts.Source: A ALGORITHM OUTLINE · p. 13
  • For each question and search iteration, it mutates a span of M variation selectors, appends the suffix, and queries the model for the target token’s first-position log-probability.A mutation is accepted when that log-probability increases.Source: A ALGORITHM OUTLINE · p. 13
  • When a prompt produces harmful output, the algorithm adds its suffix–token pair to the next-round pool and adds the prompt to the output set.It removes the completed question from the remaining question set before continuing the chain of search.Source: A ALGORITHM OUTLINE · p. 13

B SYSTEM PROMPTS

The appendix documents the system prompts used for semantic judging, jailbreak tasks, and prompt-injection tasks across the evaluated models.

B SYSTEM PROMPTS

  • GPT-4 is used as the semantic judge, with gpt-4o-mini-2024-07-18 as the default evaluation version.Source: B SYSTEM PROMPTS · p. 13
  • Tables 6–8 give the jailbreak-task prompts for Vicuna-13B-v1.5, Llama-2-Chat-7B, Llama-3.1-Instruct-8B, and Mistral-7B-Instruct-v0.2.Source: B SYSTEM PROMPTS · p. 13
  • For prompt-injection tasks, all four models use the system prompt shown in Table 9, following Liu et al. (2024).Source: B SYSTEM PROMPTS · p. 13

C GENERATION EXAMPLES

The section presents example outputs for jailbreak and prompt-injection attacks against four aligned LLMs, alongside the prompts used to elicit them.

C GENERATION EXAMPLES

  • Across four representative aligned LLMs, the malicious question and its imperceptible jailbreak variant remain visually identical in the examples.The examples respond to a request for step-by-step instructions to hack into a corporate network.Source: C GENERATION EXAMPLES · p. 13, Table 10 · p. 15
  • Table 11 gives four aligned LLMs’ example outputs to an injection that changes a sentiment-classification request into a spam-or-not-spam task.Source: C GENERATION EXAMPLES · p. 13
  • The Llama-3.1-Instruct-8B example begins by identifying the corporate network and describing its address and protocols.Source: C GENERATION EXAMPLES · p. 15
  • The Mistral-7B-Instruct-v0.2 example also begins by identifying the target network through its IP address or domain name.Source: C GENERATION EXAMPLES · p. 15
  • The example network-hacking response includes gathering information, scanning for open ports and services, and exploiting vulnerabilities.Source: C GENERATION EXAMPLES · p. 15

D POTENTIAL LIMITATIONS AND FUTURE DIRECTIONS

The authors note that suffix filtering and output filtering could mitigate the attack, and identify adaptive attacks against these defenses as future work.

D POTENTIAL LIMITATIONS AND FUTURE DIRECTIONS

Related papers

Loading 2510.05025v1…