Improved Few-Shot Jailbreaking Can Circumvent Aligned Language Models and Their Defenses

• Adversarial suffixes can be effective but are often semantically meaningless and vulnerable to perplexity filters and SmoothLLM.LLM-assisted attacks produce semantic requests that can bypass perplexity filtering but may struggle under strict evaluation conditions.Source: §1 · p. 1
• Many-shot jailbreaking can achieve high ASRs but requires long contexts that remain unavailable to many open-source models.This motivates improving few-shot attacks within limited context sizes.Source: §1 · p. 2
• I-FSJ combines automatically generated harmful demonstrations, target-template special tokens, and demo-level random search to optimize few-shot jailbreak prompts.The search operates over demonstrations rather than individual tokens, preserving semantic inputs.Source: §1 · p. 2
• I-FSJ achieves ASRs above 80% (mostly above 95%) on aligned Llama-2-7B and Llama-3-8B, retaining above-95% ASRs in most cases under different jailbreaking defenses.Source: §1 · p. 2
• The paper presents I-FSJ as an automated baseline that removes extensive human labor from few-shot jailbreaking research.The contribution is positioned as a strong automated baseline for future work.Source: §1 · pp. 2–3

Jailbreaking attacks

• Jailbreaking attacks exploit weaknesses in aligned LLMs through manually designed prompts, automatically generated prompts, adversarial suffixes, and poisoned or benign fine-tuning.The literature attributes vulnerability to conflicting capability and safety objectives, training gaps, and adversarial-example weaknesses.Source: §2 · p. 3

Jailbreaking defenses

• Jailbreaking defenses include safety-context prompts, perplexity filters, input perturbations, harmful-output detection, and adversarial training against optimized suffixes.Some defenses also augment models with safety experts or interpolate safety and original prediction distributions.Source: §2 · p. 3

3.1 Preliminaries

• The attack seeks a prompt P that makes a target model produce harmful content for a harmful request, as judged by JUDGE(LLM(P), G) = YES.The setup assumes access to harmful requests that aligned models normally recognize and refuse.Source: §3.1 · p. 3
• Few-shot jailbreaking uses harmful in-context demonstrations, but vanilla FSJ is ineffective on some well-aligned models and many-shot variants require unavailable long contexts.ICL supplies task examples before a new query, allowing the model to infer a contextual pattern.Source: §3.1 · pp. 3–4

3.2 Improved strategies

• I-FSJ constructs one reusable pool of harmful request-response demonstrations by prompting helpful-inclined models such as Mistral-7B.The pool is built once and reused against multiple models and defenses.Source: §3.2 · p. 4
• Injecting target-template tokens such as [/INST] between user and assistant messages and between generated steps addresses near-zero vanilla-FSJ ASRs on strongly aligned Llama models.The design is motivated by the model’s conversation template and a hypothesized tendency to generate after [/INST].Source: §3.2 · p. 4
• Demo-level random search replaces one demonstration at a random position and accepts changes that reduce the loss of generating a target initial token such as “Step”.The batch algorithm uses output logits rather than gradients, and an ensemble variant seeks robustness to input perturbations such as SmoothLLM.Source: §3.2 · pp. 4–5, §3.1 · p. 4
• ≥80% ASRs are maintained at 8 shots across SmoothLLM perturbation types, although ASRs decrease as perturbation percentages increase.The figure reports LLM-based and rule-based ASRs for q ∈ {5, 10, 15, 20} across three trials.Source: Figure 2 · p. 5

4.1 Implementation details

• The evaluation covers multiple aligned 7B model families, two ASR metrics, seven defense mechanisms, and 50 harmful requests from AdvBench.The metrics are rule-based detection and a fine-tuned Llama Guard classifier.Source: §4.1 · pp. 5, 7

4 Empirical studies

• I-FSJ measures attack success rates against safety-aligned LLMs using 50 harmful requests, with both rule-based and LLM-based metrics, after just 3 random restarts.Source: Table 1 · p. 6

4.2 Jailbreaking attacks on aligned LLMs

• I-FSJ generalizes across aligned models: OpenChat-3.5, Starling-LM-7B, and Qwen1.5-7B-Chat reach nearly 100% ASRs with either demo-level random search or injected special tokens.Stronger-aligned Llama-2-7B-Chat and Llama-3-8B-Instruct require combining both techniques, yet exceed 90% ASRs within limited context windows.Source: §4.2 · p. 7
• On GPT-4, I-FSJ achieves over 90% rule-based and over 80% LLM-based ASRs with only 1-shot or 2-shot demonstrations.Demo-level random search and the injected special token </text> consistently improve GPT-4 attack success.Source: §4.2 · p. 7

4.3 Jailbreaking attacks on Llama-2-7B-Chat + jailbreaking defenses

• Against Llama-2-7B-Chat defenses, demo-level random search raises ASRs to nearly 100% in 4-shot and 8-shot settings, with 4 shots sufficient for most defenses.The PPL filter and SafeDecoding do not reduce the attack under the reported settings, while ICD is effective mainly in the 2-shot setting.Source: §4.3 · pp. 8–9
• I-FSJ remains robust to perturbations: SmoothLLM limits 2-shot ASRs to at most 10%, but 8-shot I-FSJ exceeds 85% against all tested variants.At 8 shots, ASRs remain at least 80% across perturbation types and rates.Source: §4.3 · p. 9
• I-FSJ can jailbreak both a target model and Llama Guard by adapting demonstrations to the Guard model’s conversation template.This propagative variant addresses output-detection defenses rather than only the target model’s refusal behavior.Source: §4.2 · p. 8, §4.3 · p. 9

4.4 Further analysis

• ASRs increase with pool size and shot count before saturating: 32 demonstrations already yield over 90% ASRs at 8 shots, while 8 shots approaches 100%.The 2-shot setting starts around 25.4% and reaches about 61.6%, whereas 4-shot performance rises from about 88.0% to roughly 97.8%.Source: §4.4 · pp. 9–10, Figure 4 · p. 9

• I-FSJ automates demonstration-pool creation, uses special tokens from the target system template, and applies demo-level random search to achieve high ASRs.The method is designed particularly for well-aligned open-source LLMs with limited context sizes and avoids extensive human labor.Source: §5 · p. 10

Broader Impacts

• I-FSJ could be misused against deployed systems, motivating robust and adaptive defenses against evolving attack methods.The paper frames this as a broader-impact concern and places the work within an ongoing attack–defense dynamic.Source: A Broader Impacts and Limitations · p. 15

Limitations

• The method primarily targets open-source LLMs because it assumes access to the target model’s conversation template and exploitable special tokens.For closed-source models such as GPT-4 and Claude, the template is usually unknown, and effectiveness remains a future research question.Source: A Broader Impacts and Limitations · p. 15
• Obfuscating or randomizing system-prompt special tokens could reduce I-FSJ effectiveness and require continual attack adaptation.Source: A Broader Impacts and Limitations · p. 15

B.1 Demo-level random search for SmoothLLM

• The SmoothLLM-specific search objective evaluates multiple perturbations per iteration to select demonstrations that are both effective and perturbation-robust.Algorithm 2 uses batch demo-level random search over a demonstration pool and ensembles K perturbations during optimization.Source: A Broader Impacts and Limitations · p. 16

B.1 Demo-level random search for SmoothLLM

• The adaptive objective evaluates K perturbations at each iteration to optimize demonstrations for both jailbreak effectiveness and robustness.Source: §B.1 · p. 16

B.2 The setup of metrics

• The metric setup uses a refusal-keyword list and an LLM-based judge whose goal and response fields are filled with the request and model output.The refusal keywords follow Zou et al. [74], while the judging prompt follows Chao et al. [10].Source: §B.2 · p. 16, Figure 6 · p. 16, Figure 7 · p. 17

B.3 Defenses

• The defense evaluation includes Self-Reminder, ICD, perplexity filtering, retokenization, SmoothLLM, Safe Decoding, and Llama Guard.These defenses respectively modify safety prompts, use rejecting demonstrations, filter by perplexity, alter tokenization, perturb inputs, interpolate safety-model predictions, or detect unsafe outputs.Source: §B.3 · p. 18
• Llama Guard requires the attacker to bypass both the target model and an output filter that classifies responses as safe or unsafe.The attack demonstrations are adapted to the guard model’s conversation template.Source: §B.3 · p. 18, Figure 10 · p. 19

B.4 Demonstration pool construction

• The demonstration pool contains 520 harmful request–response pairs generated with Mistral-7B-Instruct-v0.2 from AdvBench requests.The requests are formatted using Andriushchenko et al. [2], and each generated response uses a maximum of 256 new tokens.Source: §B.4 · p. 19, Figure 11 · p. 20

B.5 The prompt template for attacking GPT-4

• The GPT-4-1106-Preview attack uses a prompt template modified from Andriushchenko et al. [2].The appendix also specifies an assistant-turn token sequence for this template.Source: Figure 12 · p. 21, §B.5 · p. 22
• Table 5 reports 8-shot I-FSJ attack success rates on AdvBench and HarmBench for Llama-2-7B-Chat, with optional similarity filtering and decoding lengths of 100 or 512.Source: Table 5 · p. 22

The effect of similar harmful behaviors

• Most generations have below 0.5 textual similarity to in-context demonstrations, indicating that I-FSJ produces novel generations rather than simply replicating them.Filtering demonstrations with similarity scores of 0.5 or higher does not remove I-FSJ’s effectiveness.Source: §C.1 · p. 22, Figure 13 · p. 23

The effect of more diverse test cases

• I-FSJ maintains its effectiveness on HarmBench, showing that its performance extends beyond AdvBench’s limited scale.The evaluation uses both AdvBench and HarmBench to assess more diverse test cases.Source: §C.1 · p. 22

The effect of decoding length

• I-FSJ maintains its effectiveness when decoding length increases from 100 to 512 new tokens.The longer-decoding evaluation covers both AdvBench and HarmBench.Source: §C.1 · p. 22

The effect of using correct special tokens

• Using [INST] or Qwen1.5B’s special tokens is ineffective, demonstrating that injecting the target model’s correct special tokens is important.The comparison is conducted on Llama-2-7B-Chat.Source: §C.1 · p. 22

The number of necessary query times

• 88 queries on AdvBench and 159 on HarmBench suffice for I-FSJ to achieve nearly 100% ASRs on Llama-2, whereas comparison methods trade off queries and effectiveness.PAIR uses 33.8 queries for 10% ASR, while GCG requires 256K queries for 54% ASR on AdvBench.Source: §C.1 · p. 22, Figure 14 · p. 23

Compare our method with ICA

• I-FSJ with 2 shots reaches 68% ASR against Llama-2 on AdvBench, compared with 58% for ICA with 10 shots.The comparison is reported against Wei et al.’s ICA results [62].Source: §C.2 · p. 23
• Against GPT-4, I-FSJ with 2 shots reaches 94% ASR, compared with 46% for ICA with 10 shots.The reported comparison is attributed to Wei et al. [62].Source: §C.2 · p. 23
• I-FSJ with 8 shots achieves ASRs comparable to ICA with 64 shots, yielding an 8× efficiency improvement.Both methods use approximately 64-token demonstrations in a 4096-token context window, with similar harmful requests filtered out.Source: §C.2 · p. 23, Table 6 · p. 23

C.3 Smooth LLM loss curves

• SmoothLLM loss steadily decreases as the demo-level optimization step increases, indicating the effectiveness of the proposed method.The trend is shown in Figure 15 for Insert, Swap, and Patch variants.Source: §C.3 · p. 24
• The surrounding evaluation reports attack success rates across multiple defenses on Llama-2-7B-Chat using rule-based and LLM-based measures with three random restarts.The supplied table descriptions identify the evaluation structure but do not provide the individual ASR values.Source: Table 8 · p. 25, Table 7 · p. 25