PaperlayerView exact revision on arXiv

Source-linked AI summary

LLM-Virus: Evolutionary Jailbreak Attack on Large Language Models

Miao Yu, Junfeng Fang, Yingjie Zhou, Xing Fan, Kun Wang, Shirui Pan, Qingsong Wen

arXiv:2501.00055v1cs.CRcs.AIcs.CL

TL;DR

Jailbreak research faces gaps in transparency, transferability, and computational cost, alongside limitations in existing evolutionary attacks’ operators, scope, and evaluation. LLM-Virus uses LLM-guided evolutionary search and transfer-learning techniques to evolve jailbreak templates. The paper reports competitive or better performance than several baselines across multiple safety benchmarks.

  • Problem

    Existing evolutionary jailbreak methods have limited operators and attack settings, small malicious datasets, and do not address evolution time costs or comprehensively compare against traditional attacks.

    Source: §I · p. 2

  • Method

    LLM-Virus uses LLMs as evolutionary operators to search jailbreak templates and incorporates transfer learning through Local Evolution and Generalized Infection.

    Source: §I · p. 2

  • Results

    LLM-Virus performs comparably or better than several baselines across multiple safety benchmarks.

    Source: §VI · p. 7

  • Takeaways & Limitations

    The experiments report state-of-the-art attack success rates and lower time costs, with comparisons to traditional LLM attacks.

    Source: §I · p. 2

Abstract

from arXiv · show

While safety-aligned large language models (LLMs) are increasingly used as the cornerstone for powerful systems such as multi-agent frameworks to solve complex real-world problems, they still suffer from potential adversarial queries, such as jailbreak attacks, which attempt to induce harmful content. Researching attack methods allows us to better understand the limitations of LLM and make trade-offs between helpfulness and safety. However, existing jailbreak attacks are primarily based on opaque optimization techniques (e.g. token-level gradient descent) and heuristic search methods like LLM refinement, which fall short in terms of transparency, transferability, and computational cost. In light of these limitations, we draw inspiration from the evolution and infection processes of biological viruses and propose LLM-Virus, a jailbreak attack method based on evolutionary algorithm, termed evolutionary jailbreak. LLM-Virus treats jailbreak attacks as both an evolutionary and transfer learning problem, utilizing LLMs as heuristic evolutionary operators to ensure high attack efficiency, transferability, and low time cost. Our experimental results on multiple safety benchmarks show that LLM-Virus achieves competitive or even superior performance compared to existing attack methods.

I. INTRODUCTION

Safety-aligned LLMs can still be bypassed by jailbreaks, while existing evolutionary approaches face limits in operator design, attack scope, and evaluation. LLM-Virus uses LLM-guided evolutionary search and transfer-learning techniques to improve jailbreak performance, transferability, and time cost.

I. INTRODUCTION

  • Safety alignment is increasingly important as LLMs are deployed in applications and multi-agent systems, yet jailbreak methods can bypass built-in safeguards.Direct malicious queries are typically rejected by safety-aligned models.Source: §I · p. 1
  • Earlier evolutionary attacks were limited by narrow operators, restricted attack settings, small malicious datasets, and missing time-cost analysis or comprehensive baselines.The cited examples include attacks focused on toxicity or smaller, less secure models and datasets of 100 samples.Source: §I · p. 2
  • LLM-Virus treats jailbreaks as viral infection: templates evolve under the host LLM’s safety selection, while malicious queries act as functional proteins.The population is initialized with human-written templates, and LLMs serve as evolutionary operators.Source: §I · p. 2
  • LLM-Virus achieves state-of-the-art attack success rates and lower time costs, with experiments comparing it against traditional LLM attacks.The paper also reports comparisons with other evolutionary attacks.Source: §I · p. 2
  • LLM-based mutation and crossover explore a wider template space, while Local Evolution and Generalized Infection target lower time costs and greater transferability.Templates can embed different queries rather than being tied to one malicious query.Source: §I · p. 2

II. RELATED WORKS

Related work spans prompt-based jailbreak attacks, evolutionary search, and uses of LLMs as evolutionary operators. The preliminaries formalize black-box inference attacks in which a template is combined with a malicious query to seek an affirmative response.

II. RELATED WORKS

  • Prior jailbreak work includes human-designed templates, optimized prefixes or suffixes, and sequence-to-sequence or multi-agent prompt modification; this paper combines LLM-enhanced EAs with jailbreak search.Evolutionary methods for LLM attacks include mutation, crossover, fitness evaluation, and template evolution.Source: §II · p. 2
  • EAs iteratively improve populations through selection, mutation, and crossover, while some research uses LLMs to enhance these evolutionary operators.Prior work also applies EAs within LLM tasks such as prompt engineering and structure search.Source: §II · p. 2

III. PRELIMINARIES

  • The threat model is an inference-time attack in which a jailbreak template is combined with a malicious query to elicit a harmful affirmative response.Directly querying the malicious content is normally rejected by well-guarded LLMs.Source: §III · p. 3
  • The text operator ⊕ combines text by concatenation or insertion, including placing one text as a prefix or suffix of another.This operator describes how a template and query can be combined in the attack formulation.Source: §III · p. 3

IV. LLM-VIRUS FRAMEWORK

LLM-Virus applies evolutionary algorithms to search for jailbreak templates, using an LLM as an evolutionary operator. It frames the attack as viral evolution and selects initial templates for stealthiness, diversity, and cheapness.

IV. LLM-VIRUS FRAMEWORK

  • LLM-Virus uses evolutionary algorithms to optimize jailbreaks, with an LLM serving as an evolutionary operator.The framework is inspired by biological virus evolution.Source: §IV · p. 3
  • The target is a virus-like pair: template j is mutating genetic material, while query qi performs the jailbreak attack on the host.The host’s safety mechanisms provide selection pressure.Source: §IV · p. 3
  • The objective combines jailbreak-template attack success with constraints such as text length.The toxicity term measures attack success, while additional requirements are represented by constraint functions.Source: §IV · p. 3

A. Strain Collection

  • Strain Collection filters existing jailbreak templates to initialize the search, favoring stealthiness, semantic diversity, and conciseness.Stealthiness means the template alone, without the malicious query, should not be rejected by the host LLM.Source: A. Strain Collection · p. 4

B. Local Evolution

Local Evolution reduces work across malicious queries by evolving templates on a clustered representative subset. It uses an LLM for fitness evaluation and crossover and mutation, then applies Generalized Infection to assess templates across queries.

B. Local Evolution

  • LLM-Virus clusters the malicious-query set and evolves templates on a representative subset to reduce time and computational overhead.The paper frames this as transfer learning between LLMs that share knowledge structures and modes of thinking.Source: B. Local Evolution · p. 4
  • During Local Evolution, an LLM evaluates attack success and performs crossover and mutation because mathematical operators do not handle text variables effectively.The execution pipeline takes the host, operator LLM, query set, human-designed templates, generation limit, population size, and expected success rate as inputs.Source: B. Local Evolution · p. 4
  • The pipeline initializes templates through Strain Collection, evolves generations, and uses Generalized Infection to decide whether to continue or retain the resulting templates.If average infection success is below the expected rate, the current templates seed another round; otherwise they become the output.Source: B. Local Evolution · p. 4

1) Fitness:

The paper evaluates jailbreak success using rejection-keyword detection and an LLM judge. The judge distinguishes the attacked model from the evaluating model and receives a designed system prompt.

2) Fitness:

  • Jailbreak success is evaluated with rejection-keyword detection and LLM discrimination.Examples of rejection keywords include “Sorry” and “can’t”.Source: 1) Fitness · p. 4
  • In the LLM-based evaluation, Le is the evaluating model and Lh is the attacked model.A designed system prompt guides Le in judging whether the virus succeeds.Source: 1) Fitness · p. 4
  • The evaluation prompt is designed to guide the judge’s determination of jailbreak success for virus (j, qi).The paper also uses rejection-keyword detection as a separate evaluation method.Source: 1) Fitness · p. 4

2) Crossover/Mutation:

LLM-Virus uses heuristic crossover and mutation to search beyond fixed word-level and paragraph-level operations. Tailored prompts direct these operators toward specified textual properties, and higher temperature can increase diversity.

2) Crossover/Mutation:

  • Heuristic crossover and mutation extend prior operators [13], [20] by directing LLM search across words and paragraphs toward specified properties.The approach builds on LLM-based crossover and mutation in text [23] and.Source: 2) Crossover/Mutation · p. 4
  • Tailored prompts Pc and Pm guide the LLM in few-shot evolution toward specified properties, producing a new jailbreak template.The resulting template is denoted jnew ∈ T.Source: 2) Crossover/Mutation · p. 4
  • Increasing the crossover or mutation LLM’s temperature can produce more diverse tokens and thereby greater diversity.Source: 2) Crossover/Mutation · p. 4

3) Selection:

LLM-Virus prioritizes toxicity, then stealthiness and cheapness, in selection. Its transfer-learning stages test and exploit template transferability while reducing evolution cost, with |Dr| controlling the overhead–transferability trade-off.

3) Selection:

  • Selection ranks toxicity first, followed by stealthiness and cheapness, using keyword ranking rather than a multi-objective evolutionary algorithm.The ordered fitness criteria support common selection strategies.Source: 3) Selection · p. 5

C. Generalized Infection

  • After Local Evolution, Generalized Infection tests evolved templates’ transferability from Dr to D using Eq. 6 to calculate success rate.The paper attributes easier transfer in this setting to Dr’s clustering-based representativeness and similar LLM defenses across harmful queries.Source: C. Generalized Infection · p. 5

D. Trade-off between Cost and Transferability

  • Eq. 10 and 11 show that transfer learning reduces evolution cost, with |Dr| adjustable to balance overhead and transferability.For transfer learning, each round generates N offspring and uses 2N LLM queries for crossover and mutation, plus |Dr|×N for fitness evaluation.Source: D. Trade-off between Cost and… · p. 5

V. EXPERIMENT

The experiments use closed- and open-source host LLMs, two harmful-query benchmarks, multiple ASR evaluators, and GPT-4o for evolutionary operations. Baselines are evaluated on both benchmarks, and LLM-Virus results are averaged over three runs.

A. Experimental Setups

  • The host models include GPT, Claude 2, and Gemini series, plus Llama, Vicuna, and Gemma series; GPT-4o performs crossover and mutation.The host set includes closed- and open-source models.Source: A. Experimental Setups · p. 5
  • AdvBench and HarmBench provide 520 and 400 harmful-behavior instances, respectively; KMeans selects cluster-center-nearest queries into Dr for Local Evolution.The queries are embedded using all-MiniLM-L6-v2 before clustering.Source: A. Experimental Setups · p. 5
  • Baselines are evaluated on HarmBench and AdvBench, and LLM-Virus performance is averaged across three runs to account for evolution randomness.The comparison includes traditional and evolutionary methods.Source: A. Experimental Setups · p. 6
  • The evaluation uses rejection-keyword ASRk, classifier-based ASRc, and GPT-4o-classified ASRl as binary attack-success measures.ASRc is the HarmBench fitness function, while ASRl is used for AdvBench evolution.Source: A. Experimental Setups · p. 6
  • Evolution uses N = 10 offspring and G = 10 iterations, with GPT-4o at temperature 1 for mutation, crossover, and fitness evaluation.Mutation probability is 0.5; parents have equal selection chance, crossover probability is 1, and elitism selects the next generation.Source: A. Experimental Setups · p. 6

B. Local Evolution Dynamic

During Local Evolution, average ASRl increases across generations, while template length also shifts toward the optimization target.

B. Local Evolution Dynamic

  • ASRl rises from 11.5 to 26.9 for GPT-4o and from 20.8 to 29.3 for Claude-3.5; GPT-3.5-Turbo increases from 54.6 to 100.0.The passage reports both average ASRl and template length progressively optimizing toward the target as generations increase.Source: B. Local Evolution Dynamic · p. 6

C. Generalized Infection Performance

On the full harmful-query dataset, LLM-Virus outperforms baselines across HarmBench and AdvBench, while its evolved templates also transfer across hosts. The reported evaluations further describe perplexity and time cost.

1) Toxicity:

  • LLM-Virus achieves the best HarmBench result on 3 of 5 closed-source models and all open-source models, with ASRc 1.46× and 1.61× the second-best on Gemini-Pro and Llama-3.1-70B.Across three Llama-3.1 scales, average ASRc is 44.2 for LLM-Virus versus 1.37 for AutoDAN; on AdvBench, GPT-3.5-Turbo ASRl exceeds twice BlackDAN’s.Source: 1) Toxicity · p. 6

2) Transferability:

  • Transferred templates show average ASRl of 73.7 on GPT-3.5-Turbo and 49.3 on Llama-3.1-70B, while transfers to Claude-3.5-Haiku fail unless evolved on that host.GPT-4o-mini templates transfer at ASRl 54.2 on GPT-4o and 75.0 on Llama-3.1-8B, exceeding their original-host values of 31.7 and 34.6.Source: 2) Transferability · p. 6, Figure 4 · p. 7

3) Perplexity & Time:

  • Average perplexity is 45.1 and 46.5, closer to manually written jailbreak texts at 23.0 than GCG at 1532.2; GPT-3.5-Turbo template generation costs 1.2 minutes per harmful action.The time figure uses five parallel workers; time can be further reduced with lower |Dr|/|D| or greater parallelism.Source: 3) Perplexity & Time · p. 7, Table IV · p. 7

D. Ablation Study

LLM-Virus is reported to perform comparably or better than several baselines across multiple safety benchmarks; ablations examine its settings and modules, and a case study shows gains in attack success and prompt length after evolution.

D. Ablation Study

  • At temperature 2, ASRc fluctuates around 47.7; at temperature 0, improvement is slower, reaching the reported outcome at Generation 8.The authors emphasize the importance of an appropriate temperature for LLM evolution.Source: D. Ablation Study · p. 7
  • Removing mutation lowers final ASRc from 80.8 to 66.9, while removing crossover lowers it from 80.8 to 71.5.Replacing heuristic operators with normal operators slightly lowers early-generation ASRc but produces character lengths around 2× longer; Figure 5 presents the ablation study.Source: D. Ablation Study · p. 7, Figure 5 · p. 7
  • Removing Strain Collection drops ASRc 26.8% (80.8 →59.2), while population sizes N=20 and N=5 are not better than base N=10 in average performance.Figure 5 shows the ablation results in its top panel.Source: D. Ablation Study · p. 7, Figure 5 · p. 7

E. Case Study

  • The case-study template ends with ASRc 57.2% higher and character length 29.0% lower than the first-generation template.The authors describe the last-generation template as evolved from the first-generation one; the case study appears in Figure 5’s bottom panel.Source: E. Case Study · p. 7, Figure 5 · p. 7

VI. CONCLUSION

  • LLM-Virus performs comparably or better than several baselines across multiple safety benchmarks, according to the paper’s conclusion.The authors also report ablations of tailored settings and components.Source: §VI · p. 7

Related papers

Loading 2501.00055v1…