PaperlayerView exact revision on arXiv

Source-linked AI summary

AutoJailbreak: Exploring Jailbreak Attacks and Defenses through a Dependency Lens

Lin Lu, Hai Yan, Zenghui Yuan, Jiawen Shi, Wenqi Wei, Pin-Yu Chen, Pan Zhou

arXiv:2406.03805v1cs.CR

TL;DR

Jailbreak methods often optimize locally, with attacks tuning specific components and defenses targeting specific prompt types. AutoJailbreak analyzes dependencies among attack and defense strategies and builds ensemble frameworks. The authors report that its attacks and defenses outperform existing approaches.

  • Problem

    Black-box jailbreak attacks optimize particular attack components, while defenses often target specific prompt types, limiting their scope.

    Source: §1 · pp. 1–2

  • Method

    AutoJailbreak uses DAG-based dependency analysis to develop ensemble attacks and defense strategies, alongside an evaluation framework.

    Source: §1 · p. 2, §3 · p. 4

  • Results

    AutoJailbreak’s ensemble attacks reliably break all tested models, and its ensemble defense significantly enhances jailbreak robustness compared with existing approaches.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The authors propose AutoJailbreak as a minimal test for new jailbreak attacks and defenses, rather than claiming it is the ultimate framework.

    Source: §1 · p. 2

    The threat model is black-box: attackers lack prior knowledge of victim-model parameters and sampling details, while defenders cannot access model parameters.

    Source: §2.3 · p. 4

Abstract

from arXiv · show

Jailbreak attacks in large language models (LLMs) entail inducing the models to generate content that breaches ethical and legal norm through the use of malicious prompts, posing a substantial threat to LLM security. Current strategies for jailbreak attack and defense often focus on optimizing locally within specific algorithmic frameworks, resulting in ineffective optimization and limited scalability. In this paper, we present a systematic analysis of the dependency relationships in jailbreak attack and defense techniques, generalizing them to all possible attack surfaces. We employ directed acyclic graphs (DAGs) to position and analyze existing jailbreak attacks, defenses, and evaluation methodologies, and propose three comprehensive, automated, and logical frameworks. \texttt{AutoAttack} investigates dependencies in two lines of jailbreak optimization strategies: genetic algorithm (GA)-based attacks and adversarial-generation-based attacks, respectively. We then introduce an ensemble jailbreak attack to exploit these dependencies. \texttt{AutoDefense} offers a mixture-of-defenders approach by leveraging the dependency relationships in pre-generative and post-generative defense strategies. \texttt{AutoEvaluation} introduces a novel evaluation method that distinguishes hallucinations, which are often overlooked, from jailbreak attack and defense responses. Through extensive experiments, we demonstrate that the proposed ensemble jailbreak attack and defense framework significantly outperforms existing research.

1 Introduction

Jailbreak research faces a local-optimization problem: attacks improve individual components while defenses often target specific prompt types. AutoJailbreak analyzes dependencies across attacks and defenses, proposes automated frameworks, and reports stronger attack and defense performance than existing approaches.

1 Introduction

  • Existing attack methods optimize specific components and defenses often target particular prompt types, leaving jailbreak research in a local-optimization trap.The authors describe this as failing to genuinely enhance LLM robustness.Source: §1 · pp. 1–2
  • AutoAttack groups black-box attacks into genetic-algorithm and adversarial-generation frameworks, analyzes dependencies among their optimization schemes, and proposes an ensemble for each.The ensembles are named Ensemble Attack-GA and Ensemble Attack-Gen.Source: §1 · p. 2
  • AutoDefense analyzes defense dependencies and combines pre-generation and post-generation defenses through a mixture-of-defenders approach.It is intended to resist varied carefully crafted jailbreak prompts.Source: §1 · p. 2
  • AutoEvaluation examines off-topic responses and compares three evaluation methods with human evaluation, selecting LLM-as-a-Judge as its primary experimental criterion.The methods are keyword matching, classifier-based evaluation, and LLM-as-a-Judge.Source: §1 · p. 2
  • AutoJailbreak’s ensemble attack reliably breaks all tested models, while its ensemble defense improves robustness beyond defenses aimed at specific prompt types.The authors say the framework should become a minimal test for new attacks and defenses, not that it is the ultimate attack and defense.Source: §1 · p. 2

2 Background

The background distinguishes black-box attack construction and defense timing, reviews gaps in jailbreak evaluation and benchmarks, and specifies a black-box threat model for both attackers and defenders.

2.1 Black-box Jailbreak Attacks

  • Dynamic black-box attacks iteratively optimize malicious prompts through genetic-algorithm mutation or an LLM red-teaming assistant, while the study excludes long-tail encoding and transferable attacks.Static human-designed templates are straightforward but can be mitigated through alignment fine-tuning.Source: §2.1 · pp. 2–3

2.2 Jailbreak Defenses and Evaluation

  • Existing evaluation methods can miss unsafe content after an apparent rejection or responses that do not address the query, and benchmarks based on existing attacks may not represent the strongest attacks.Methods include keyword matching, classifiers, and LLM-as-a-judge; systematic examination of these evaluation issues remains limited.Source: §2.2 · p. 3

2.3 Threat Model

  • The black-box attacker interacts with a victim model through API queries without prior knowledge of its parameters, token probabilities, or sampling hyperparameters.Defenders may modify the prompt or response but lack model-parameter knowledge, and should resist attacks using malicious semantics and adversarial suffixes.Source: §2.3 · p. 4

3 AutoJailbreak

AutoJailbreak uses DAG-based dependency analysis to combine jailbreak attack and defense strategies, alongside an evaluation framework that distinguishes harmful, off-topic, and aligned responses. Its evaluation comparison favors LLM-as-a-Judge over keyword matching and a binary classifier.

3 AutoJailbreak

  • DAG nodes represent optimization solutions, and directed edges indicate whether an endpoint solution enhances or diminishes jailbreak outcomes relative to its starting point.The authors exclude minor variations such as hyperparameter changes within the same generic framework from the analysis.Source: §3 · p. 4

3.1 AutoAttack

  • AutoAttack combines optimizations into two ensembles: GA-based attacks build on AutoDAN-GA, while adversarial-generation attacks build on Tastle.The GA ensemble incorporates GPTFuzzer, SMJ, and OpenSesame techniques; the generation ensemble draws on PAIR, TAP, and SBJ, with an evaluator filtering off-topic prompts.Source: §3.1 · p. 5

3.2 AutoDefense

  • AutoDefense routes malicious prompts by type to one of two experts, targeting adversarial suffixes or malicious semantics.The experts combine prompt-modification strategies, and the ensemble also filters harmful outputs after generation and uses a voting judge to select a response.Source: §3.2 · pp. 5–6

3.3 AutoEvaluation

  • The evaluation compares keyword matching, a fine-tuned binary classifier, and LLM-as-a-Judge using true positive and false positive rates.The classifier is trained on samples from five toxicity datasets.Source: §3.3 · p. 7
  • LLM-as-a-Judge significantly outperforms the binary classifier and keyword matching methods in the TAP attack evaluation.Table 1 reports TPRs and FPRs for the three methods.Source: §3.3 · p. 7, Table 1 · p. 7
  • AutoEvaluation checks responses first for harmful content and then, when responses are clean, whether they answer the user’s query.It classifies outcomes as jailbreak, hallucination, or alignment.Source: §3.3 · p. 7

4 Experiment

Experiments compare ensemble attacks and defenses across malicious behaviors, multiple LLMs, and dynamic, static, and white-box attacks. The ensemble methods perform strongly across these settings, with defense results varying by attack type.

4.1 Experimental Setup

  • The experiments cover 96 malicious behaviors, eight open- and closed-source LLMs, and jailbreak, hallucination, and alignment rates.The dataset has general and target sections; hallucination and alignment rates are measured among failed jailbreak instances.Source: §4.1 · p. 7

4.2 Main Results

  • Ensemble Attack-Gen led on five models, trailed leading attacks by 4.2% and 6.2% on the others, and beat baselines on GPT-4 and Claude by 50% and 36.5%.The comparison uses PAIR and TAP as baselines; GPT-4 served as the attack model for LLaMa-2, LLaMa-3, and Claude.Source: §4.2 · p. 8, Table 2 · p. 8
  • Ensemble Attack-GA achieved a jailbreak rate above 90% on four LLMs, reaching 99% on Vicuna, Qwen, and Mistral.It also had slightly lower hallucination rates than the baseline methods; the authors link this to semantic features in seed selection and mutation.Source: §4.2 · p. 8
  • AutoDefense outperformed baseline defenses on almost all LLMs and almost always limited successful jailbreaks to single digits against ensemble attacks.The authors also report reduced hallucination rates, attributing this to post-generation rejection of malicious requests.Source: §4.2 · p. 8, Table 3 · p. 9
  • Against static attacks, AutoDefense constrained the jailbreak rate of most attacks to below 1%, while Spell-check and Monto-Carlo Sampling were nearly ineffective against malicious semantics.The tested static attacks included system prompts and malicious templates, including multi-turn and special-format approaches.Source: §4.2 · pp. 8–9, Table 4 · p. 10
  • On white-box gradient-optimized attacks against LLaMa-2, AutoDefense significantly outperformed almost all baseline defenses.Spell-check and Monto-Carlo Sampling performed better than Intention Analysis; Summarize achieved complete defense but had a higher hallucination rate.Source: §4.2 · p. 9, Table 5 · p. 10

5 Conclusion

The conclusion presents AutoJailbreak as a causal-analysis framework for black-box attack and defense optimization, combining ensemble methods and an evaluation system that accounts for hallucinations. Its experiments support using the framework to assess robustness across attack types, while the authors position it as a foundation for further work.

5 Conclusion

  • AutoJailbreak analyzes dependencies among black-box attack and defense optimization schemes, combining attack methods and defenses into ensemble approaches.AutoAttack groups attacks into GA-based and adversarial-generation frameworks; AutoDefense combines defense algorithms through a mixture-of-defenders mechanism.Source: §5 · p. 10
  • Tests on static and dynamic attacks found that the ensemble defense improved victim-model robustness beyond a single attack type.The conclusion also reports strong attack effects across eight common LLMs.Source: §5 · p. 10
  • The authors argue that ensemble attacks can improve robustness assessment and present AutoJailbreak as a foundation for further attack and defense research.They also incorporate hallucinations into their evaluation of jailbreak-generated content.Source: §5 · p. 10

A Summarization of Existing Jailbreak Attacks and Defenses

This section summarizes existing jailbreak attacks and defenses, with Table 6 listing related works in the categories covered by the paper.

A Summarization of Existing Jailbreak Attacks and Defenses

B Prompt Template

The appendix provides prompt templates for classifying jailbreak prompts, evaluating assistant responses, and applying defense and selection procedures. It also includes semantic-similarity scoring and examples of malicious and benign queries.

B Prompt Template

  • A separate prompt rates semantic similarity between two sentences on a scale from 1 to 10.The attack-GA template identifies semantic similarity as a criterion.Source: B Prompt Template · p. 20
  • The malicious-prompt classifier returns YES for jailbreak prompts and NO for benign queries, illustrated by a bomb-making request disguised as code and a home-cooking question.The template asks for only YES or NO.Source: B Prompt Template · p. 21
  • Defense templates refine answers to be harmless, correct jailbreak-prompt grammar, or summarize the prompt while retaining its intent.The output-filter prompt calls for refusal with reasons and risks when an answer contains malicious content, plus a constructive harmless alternative.Source: B Prompt Template · pp. 20–22
  • The voting judge ranks candidate answers by harmful content first, then directness and whether a refusal explains its reasons.The template directs the judge to return the better answer without explanation.Source: B Prompt Template · p. 22
  • A response-evaluation prompt labels harmful information True and safe information False, while another marks unrelated replies as hallucinations and direct answers or refusals as non-hallucinations.The example treats a warning that does not directly address a surveillance prompt as a hallucination.Source: B Prompt Template · p. 23

C Dataset of Malicious Behavior

The dataset presents 96 distinct malicious behaviors across general and target categories. Its general section spans 14 broad categories, while the target section lists 10 more refined categories.

C Dataset of Malicious Behavior

  • The dataset contains 96 distinct malicious behaviors, divided into general and target sections.The behaviors are shown as examples of the dataset evaluated in the work.Source: C Dataset of Malicious Behavior · p. 24
  • The general section covers 14 categories, including theft, surveillance, false personation, drugs, weapons, human trafficking, suicide, and discrimination.The listed categories also include privacy disclosure, robbery, torture, sexual content, and infringement of the public interest.Source: C Dataset of Malicious Behavior · pp. 24–25
  • The target section lists 10 refined categories: finance, industry, cybersecurity, academic, mental health, education, traffic, media, medicine, and government.Examples include market manipulation, intellectual-property theft, data forgery, and election manipulation.Source: C Dataset of Malicious Behavior · p. 26

Related papers

Loading 2406.03805v1…