PaperlayerView exact revision on arXiv

Source-linked AI summary

Improved Techniques for Optimization-Based Jailbreaking on Large Language Models

Xiaojun Jia, Tianyu Pang, Chao Du, Yihao Huang, Jindong Gu, Yang Liu, Xiaochun Cao, Min Lin

arXiv:2405.21018v2cs.LGcs.CLcs.CR

TL;DR

Optimization-based jailbreaks can match a simple target response without eliciting harmful content, limiting GCG’s attacking efficiency. The paper proposes I-GCG, combining harmful-guidance target templates with adaptive multi-coordinate updates and easy-to-hard initialization. It reports superiority across various benchmarks and nearly 100% attack success on all four evaluated LLMs.

  • Problem

    A simple target template such as “Sure” can be matched even when the model’s subsequent response refuses the malicious request, and GCG’s attacking efficiency remains unsatisfactory.

    Source: §1 · p. 2

  • Method

    I-GCG combines diverse target templates with harmful guidance, adaptive multi-coordinate updates, and easy-to-hard suffix initialization.

    Source: §1 · p. 3

  • Results

    Across various benchmarks, I-GCG demonstrates superiority, and it achieves nearly 100% attack success rate on all four evaluated LLMs.

    Source: §1 · p. 3, §5 · p. 10

  • Takeaways & Limitations

    The reported results show that combining target-template and optimization improvements can produce an efficient optimization-based jailbreak method.

    Source: §1 · p. 3, §5 · p. 10

    The authors identify room to improve transferability and to explore better harmful guidance designs and more general suffix initialization.

    Source: §4.5 · p. 9

Abstract

from arXiv · show

Large language models (LLMs) are being rapidly developed, and a key component of their widespread deployment is their safety-related alignment. Many red-teaming efforts aim to jailbreak LLMs, where among these efforts, the Greedy Coordinate Gradient (GCG) attack's success has led to a growing interest in the study of optimization-based jailbreaking techniques. Although GCG is a significant milestone, its attacking efficiency remains unsatisfactory. In this paper, we present several improved (empirical) techniques for optimization-based jailbreaks like GCG. We first observe that the single target template of "Sure" largely limits the attacking performance of GCG; given this, we propose to apply diverse target templates containing harmful self-suggestion and/or guidance to mislead LLMs. Besides, from the optimization aspects, we propose an automatic multi-coordinate updating strategy in GCG (i.e., adaptively deciding how many tokens to replace in each step) to accelerate convergence, as well as tricks like easy-to-hard initialisation. Then, we combine these improved technologies to develop an efficient jailbreak method, dubbed I-GCG. In our experiments, we evaluate on a series of benchmarks (such as NeurIPS 2023 Red Teaming Track). The results demonstrate that our improved techniques can help GCG outperform state-of-the-art jailbreaking attacks and achieve nearly 100% attack success rate. The code is released at https://github.com/jiaxiaojunQAQ/I-GCG.

1 Introduction

The section frames optimization-based jailbreaks as vulnerable to a mismatch between matching a simple target response and producing harmful content. It introduces I-GCG, combining harmful-guidance templates with optimization and initialization changes; the authors report nearly 100% attack success across four LLMs.

1 Introduction

  • The introduction situates these methods amid vulnerable alignment safeguards and distinguishes expertise-based, LLM-based, and optimization-based jailbreaks.Optimization-based methods use model gradient information to produce prompts; GCG is cited as a high-performing example.Source: §1 · p. 2
  • A suffix can match the simple target response yet still be followed by refusal, which the authors attribute to insufficient information in the optimization goal.Prior optimization-based methods target a response of the form “Sure, here is + Rephrase(Q)”.Source: §1 · p. 2
  • I-GCG uses diverse target templates with harmful self-suggestion or guidance, rather than relying on the single “Sure” template.The proposed target response is structured as “Sure, + Harmful Template, here is + Rephrase(Q)”.Source: §1 · pp. 2–3
  • Nearly 100% attack success rate on all four evaluated LLMs is reported for the proposed I-GCG.The method combines the proposed techniques to improve jailbreak efficiency.Source: §1 · p. 3
  • Its optimization changes include adaptively replacing multiple tokens per step and initializing suffixes from simple harmful requests before tackling challenging ones.The authors describe the update strategy as accelerating convergence and the easy-to-hard initialization as boosting efficiency.Source: §1 · p. 3

2 Related work

Related work groups jailbreaks by how prompts are produced: manually, with another LLM, or through optimization. It also places I-GCG’s target-template change in contrast with GCG’s single-template objective.

2 Related work

  • Expertise-based methods use expert knowledge to create adversarial prompts manually, including hand-crafted prompts studied across categories and scenarios.The cited studies include work on real-world prompts, prompts in the wild, cross-language attacks, and LLMs’ programmatic capabilities.Source: §2 · p. 3
  • LLM-based methods use another LLM to generate or iteratively refine jailbreak prompts, including black-box approaches such as PAIR, TAP, and BRT.The passage also describes a method that transforms harmful prompts into expressions viewed as harmless.Source: §2 · p. 3
  • Optimization-based methods use gradients from white-box models; related approaches include AutoDAN, MAC, Probe-Sampling, AdvPrompter, and AmpleGCG.GCG generates suffixes by maximizing the likelihood of a beginning string in a response.Source: §2 · pp. 3–4
  • Unlike GCG’s single “Sure” target template, I-GCG uses diverse target templates containing harmful guidance to form its optimization goal.The contrast concerns how the target response is designed, not the prompt-generation category.Source: Figure 2 · p. 4

3 Methodology

The method optimizes a suffix appended to a malicious question so the model’s initial output approaches a target response, and I-GCG adds harmful guidance and improved suffix-search strategies. It combines adaptive multi-token updates with easy-to-hard initialization; the proposed initialization converges faster than random-token initialization.

3 Methodology

  • The methodology represents the malicious question and suffix as a concatenated prompt and optimizes the suffix so the model’s initial output approaches a predefined target response.The response sequence is represented by its conditional probability given the preceding input tokens.Source: §3 · p. 4

3.1 Formulation of the proposed method

  • The proposed suffix initialization promotes faster jailbreak convergence than random-token initialization, while I-GCG’s target adds harmful information to the original optimization goal.The method applies discrete-token optimization to a prompt formed from the malicious question and suffix, steering the model toward a target response.Source: §3.1 · pp. 4–5, §3 · p. 4

3.2 Automatic multi-coordinate updating strategy

  • GCG’s one-token-per-iteration updates are inefficient; the proposed strategy ranks single-token candidates, combines top candidates into multi-token suffixes, and selects the lowest-loss candidate.The number of tokens to replace is adapted at each step.Source: §3.2 · pp. 5–6

3.3 Easy-to-hard initialization

  • Easy-to-hard initialization first generates a suffix for an easier malicious question, then uses it to initialize attacks on other questions.The described seed question is randomly selected from the fraud category.Source: §3.3 · pp. 6–7

4 Experiments

The experiments compare I-GCG with jailbreak baselines, test its component techniques, and examine their effects on attack performance and efficiency. They also report remaining scope for improving suffix initialization and transferability.

4.1 Experimental settings

  • Experiments use 50 representative AdvBench requests selected from 520 objectives, plus HarmBench, and assess attack success with template-based, ChatGPT-3.5-based, and manual checks.The comparison uses four 7B-class threat models and baselines including GCG, MAC, AutoDAN, Probe-Sampling, Advprompter, PAIR, and TAP.Source: §4.1 · p. 7

4.2 Hyper-parameter selection

  • Figure 7 tracks loss values over attack iterations for different hyperparameters, while Table 1 presents AdvBench comparisons with state-of-the-art jailbreak methods.The supplied passages describe the visual scopes but do not state specific hyperparameter outcomes or table values.Source: Figure 7 · p. 7, Table 1 · p. 8

4.3 Comparisons with other jailbreak attack methods

  • 100% attack success rate across all four LLMs, with approximately 100% on the NeurIPS 2023 Red Teaming Track and robust LLAMA2-7B-CHAT, where prior methods reached approximately 56%.On MISTRAL-7B-INSTRUCT-0.2, AutoDAN achieved approximately 96% while the proposed method achieved approximately 100%.Source: Table 1 · p. 8, Table 2 · p. 8, §4.3 · p. 8

4.4 Ablation study

  • On LLAMA2-7B-CHAT and AdvBench, the ablation finds harmful guidance contributes more to effectiveness, while the update strategy and suffix initialization contribute more to efficiency; combining all three performs best.The ablation uses ASR and average iterations, with GCG as the baseline.Source: §4.4 · p. 9, Table 4 · p. 9

4.5 Discussion

  • On LLAMA2-7B-CHAT, using different harmful guidance examples achieved nearly 100% attack success rate; the authors conclude that adding harmful guidance to the optimization goal could facilitate finding the jailbreak space.Source: §4.5 · p. 9
  • The efficient techniques improve jailbreak convergence on LLAMA2-7B-CHAT and AdvBench, with suffix initialization performing better; generating its prior-guided initialization first requires the update strategy.The comparison tracks average loss across jailbreak iterations.Source: §4.5 · p. 9, Figure 8 · p. 9
  • The authors identify better harmful-guidance design, more general suffix initialization, and improved transferability as remaining areas for exploration.They point to transferability-enhancing methods summarized in [12].Source: §4.5 · p. 9

5 Conclusion

The conclusion presents I-GCG as an optimization-based jailbreak method combining diverse target templates, adaptive multi-coordinate updates, and easy-to-hard initialization. Experiments across various benchmarks are reported to demonstrate its superiority.

5 Conclusion

  • I-GCG combines diverse target templates, adaptive multi-coordinate updates, and easy-to-hard initialization to improve optimization-based jailbreak performance.The paper reports extensive experiments on various benchmarks to demonstrate the method's superiority.Source: §5 · p. 10

6 Impact statement

The impact statement notes that jailbreak prompts may elicit harmful text, while framing vulnerability discovery as research intended to support stronger safeguards. It also specifies that the victim models used have publicly available weights.

6 Impact statement

  • The proposed jailbreak techniques may generate harmful texts, but the authors frame vulnerability discovery as supporting future human-preference safeguards and defense approaches.The victim LLMs used in the paper are open-source models with publicly available weights.Source: §6 · p. 10

A Algorithm of The Proposed Method

Algorithm 1 takes an initial suffix and malicious question through candidate-substitution steps to produce an optimized suffix.

A Algorithm of The Proposed Method

B Implement of I-GCG on NeurIPS 2023 Red Teaming Track

The appendix specifies the target response and optimization settings used for I-GCG on the NeurIPS 2023 Red Teaming Track.

B Implement of I-GCG on NeurIPS 2023 Red Teaming Track

C Details of Used Threat Models

The appendix describes four threat models used in the comparison experiments, including their training or alignment backgrounds where provided.

C Details of Used Threat Models

D Details of Jailbreak Evaluation Settings

The evaluation combines refusal-signal screening, a ChatGPT-3.5 behavior check, and manual review of responses.

D Details of Jailbreak Evaluation Settings

  • Responses passing the template check are evaluated by a ChatGPT-3.5 classifier using criteria for unambiguous, non-minimal, harmful outputs.For coding behaviors, the response must contain related code artifacts; responses that only discuss or react to the behavior do not count.Source: D Details of Jailbreak Evaluation… · p. 15
  • The authors manually review responses to check evaluation accuracy.Source: D Details of Jailbreak Evaluation… · p. 15

Related papers

Loading 2405.21018v2…