PaperlayerView exact revision on arXiv

Source-linked AI summary

CodeChameleon: Personalized Encryption Framework for Jailbreaking Large Language Models

Huijie Lv, Xiao Wang, Yuansen Zhang, Caishuang Huang, Shihan Dou, Junjie Ye, Tao Gui, Qi Zhang, Xuanjing Huang

arXiv:2402.16717v1cs.CLcs.AIcs.CR

TL;DR

Jailbreaks remain a challenge for aligned LLMs, whose safety the paper hypothesizes involves intent recognition followed by response generation. CodeChameleon pairs personalized encryption with embedded decryption in a code-completion format. Across seven major LLMs, it reports 77.5% average ASR, a 29.8% increase over the best existing method.

  • Problem

    Aligned LLMs remain susceptible to jailbreak attacks that circumvent their safety protocols.

    Source: §1 · p. 1

  • Method

    CodeChameleon encrypts queries with personalized functions and embeds decryption instructions in a code-completion prompt.

    Source: §1 · pp. 1–2

  • Results

    Across seven major LLMs, CodeChameleon achieves 77.5% average ASR, a 29.8% increase over the best existing method.

    Source: §5 · p. 6

  • Takeaways & Limitations

    The paper reports that CodeChameleon successfully evades LLM intent recognition across seven major LLMs.

    Source: §5 · p. 6

    Evaluation was limited to Llama2-Chat, Vicuna, and GPT models, with analysis and ablations focused on open-source Llama2; broader experiments are needed.

    Source: §6 · p. 7

Abstract

from arXiv · show

Adversarial misuse, particularly through `jailbreaking' that circumvents a model's safety and ethical protocols, poses a significant challenge for Large Language Models (LLMs). This paper delves into the mechanisms behind such successful attacks, introducing a hypothesis for the safety mechanism of aligned LLMs: intent security recognition followed by response generation. Grounded in this hypothesis, we propose CodeChameleon, a novel jailbreak framework based on personalized encryption tactics. To elude the intent security recognition phase, we reformulate tasks into a code completion format, enabling users to encrypt queries using personalized encryption functions. To guarantee response generation functionality, we embed a decryption function within the instructions, which allows the LLM to decrypt and execute the encrypted queries successfully. We conduct extensive experiments on 7 LLMs, achieving state-of-the-art average Attack Success Rate (ASR). Remarkably, our method achieves an 86.6\% ASR on GPT-4-1106.

1 Introduction

Aligned LLMs are vulnerable to jailbreaks, and the paper hypothesizes that safety involves intent recognition followed by response generation. CodeChameleon uses personalized encryption and embedded decryption to bypass recognition while preserving query understanding.

1 Introduction

  • The proposed safety hypothesis separates aligned LLM behavior into intent security recognition and response generation.A successful attack must bypass recognition while allowing the model to understand and execute the query.Source: §1 · p. 1
  • CodeChameleon reformulates queries as code-completion tasks and uses personalized encryption to conceal malicious intent from recognition.Embedded decryption functions assist the model in understanding encrypted content during inference.Source: §1 · pp. 1–2
  • Across 7 LLMs, CodeChameleon reaches 77.5% average ASR, 28.9% above the optimal baseline, and 86.6% on GPT-4-1106.The paper also reports higher ASR than the baseline across four encryption and decryption functions on Llama2-Chat models.Source: §1 · p. 2
  • The paper proposes a safety-mechanism hypothesis and CodeChameleon, reporting state-of-the-art average ASR across 7 LLMs.The introduction characterizes the method as consistently bypassing existing safety mechanisms.Source: §1 · p. 2

2 Related Work

Related work groups jailbreaks into human-designed, optimization-based, and long-tail distributed encoding approaches. Prior work also uses code formats for varied problem-solving and structured tasks.

2 Related Work

  • Existing jailbreaks include human-designed prompts, optimization-based attacks, and long-tail encoding such as Base64, ciphers, and low-resource languages.Optimization-based methods include gradient, genetic-algorithm, and edit-based approaches.Source: §2 · p. 2

Code-style instructions for Different Tasks

3 Methodology

CodeChameleon encrypts unsafe queries into uncommon formats, then places the encrypted query and decryption instructions in a code-style prompt. Its methodology combines custom transformations, embedded decryption, and a code-completion task.

3.1 Encryption

  • CodeChameleon uses custom Python functions to transform queries into formats unlikely to appear during safety alignment.Four functions alter word order or structure using reverse order, word length, odd and even positions, and binary trees.Source: §3.1 · p. 3

3.2 Decryption

  • Four Python-based decryption functions are embedded in code blocks rather than executed, drawing on LLM code comprehension to recover the original query.The decrypted query is intended to be understood and executed by the model.Source: §3.2 · p. 3

3.3 Code Completion Task

  • The code-completion task places queries in a code-style format that the paper says is absent from most RLHF datasets.The ProblemSolver template uses analyze_problem for decryption, followed by generate_plan and generate_solution; task instructions prompt these calls.Source: §3.3 · pp. 3–4

3 Methodology

  • 77.5% average ASR is reported for CodeChameleon in Table 1, which compares its attack success rates with baselines.The caption says the best result for each model is bolded.Source: Table 1 · p. 4

4 Experiment

CodeChameleon is evaluated against jailbreak baselines on harmful-query benchmarks across seven LLMs. Results cover its overall ASR, encryption and instruction-style variants, and experiments probing decryption and response generation.

4.1 Setup

  • Evaluation uses 820 harmful queries from AdvBench, MaliciousInstruct, and ShadowAlignment across seven LLMs, including Llama2-Chat, Vicuna, GPT-3.5, and GPT-4.The open-source set includes three Llama2-Chat sizes and two Vicuna-v1.5 sizes; the proprietary models are GPT-3.5-1106 and GPT-4-1106.Source: §4.1 · p. 4

Baseline

  • Baselines span optimization-based attacks GCG, AutoDAN, and PAIR, plus manually constructed Jailbroken, cipher-based CipherChat, and multilingual attacks.ASR is assessed by whether a model neither refuses nor redirects its response in line with OpenAI’s security policy; GPT-4 evaluates these criteria.Source: Baseline · p. 4

4.2 Results

  • 77.5% average ASR; CodeChameleon achieves the highest ASR for Llama2 and GPT models, while Vicuna reaches 67.1%, essentially matching three non-optimized baselines.The paper reports CodeChameleon surpasses other methods, including white-box attacks, for Llama2 and GPT series models.Source: §4.2 · pp. 4–5
  • Larger models do not consistently show lower ASR: the reduction is 12% from Llama2-7B-Chat to 70B and 3.3% from Vicuna-7B to 13B.The paper also reports CodeChameleon leverages code capability, with comparatively poor performance on Vicuna models.Source: §4.2 · p. 5

4.3 Analysis & Ablation

  • Across four encryption/decryption functions, all circumvent safety mechanisms, with Length and BinaryTree producing the strongest concealment and most effective results.The comparison is on the Llama2-Chat series.Source: §4.3 · p. 5, Table 2 · p. 5
  • Encryption improves ASR by 24% for text-style and 63% for code-style instructions; omitting decryption lowers ASR by 39%, 47%, and 30.5% across three models.Code-style instructions also achieve higher ASR than text-style instructions across models and settings, especially on 7B and 13B models.Source: Figure 3 · p. 5, §4.3 · p. 6

4.4 Exploration of the CodeChameleon Mechanism

  • GPT-based evaluation finds decrypted queries have goals similar to the originals more than 90% of the time, despite often differing in exact wording.The paper contrasts rule-based exact-match evaluation with GPT-based assessment of objective or semantics.Source: §4.4 · p. 6, Table 3 · p. 6
  • When given a positive response beginning, Llama2-Chat generates harmful content in over 90% of instances, supporting the paper’s intent-recognition hypothesis.The malicious queries in this experiment are not encrypted.Source: §4.4 · p. 6, Table 4 · p. 6

5 Conclusion

The paper hypothesizes that LLMs detect intent before generating responses and introduces CodeChameleon to encrypt and decrypt queries in forms difficult to detect. Across seven major LLMs, it reports a 77.5% average ASR, a 29.8% increase over the best existing method.

5 Conclusion

  • 77.5% average attack success rate (ASR), a 29.8% increase over the best existing method, was achieved across seven major LLMs.The paper also reports that CodeChameleon successfully evades LLMs’ intent recognition.Source: §5 · p. 6
  • CodeChameleon encrypts and decrypts queries into a form difficult for LLMs to detect.Source: §5 · p. 6
  • The paper hypothesizes that LLM safety mechanisms first detect intent and then generate responses.Source: §5 · p. 6

6 Limitation

The authors identify insufficient variety among evaluated LLMs as a limitation and call for more comprehensive experiments to validate CodeChameleon’s effectiveness and universality.

6 Limitation

  • The evaluation used an insufficient variety of LLMs, limiting the study’s experimental scope.Source: §6 · p. 7
  • Experiments covered only Llama2-Chat, Vicuna, and GPT models, while analysis and ablation studies focused on open-source Llama2.Source: §6 · p. 7
  • The authors call for more comprehensive experiments to validate CodeChameleon’s effectiveness and universality.Source: §6 · p. 7

A.1 Encryption Function

The appendices describe four encryption-function designs, a corresponding decryption function that assists inference, and the components of the resulting jailbreak prompts.

A.1 Encryption Function

  • Four encryption functions use reverse order, word length, odd and even positions, or binary tree structure; an encrypted-query example appears in Table 5.Figure 4 shows the specific encryption-function implementation.Source: §A.1 · p. 9

A.2 Decryption Function

  • The corresponding decryption function is not executed, but assists LLMs in understanding the encrypted query during inference.Figure 5 is identified as showing the specific implementation.Source: §A.2 · p. 9

A.3 Complete Jailbreak Prompts

  • A jailbreak prompt contains a task description, decryption function, and code template; the encryption function is applied beforehand and omitted from the prompt.Figure 6 illustrates a complete prompt example.Source: §A.3 · p. 9

B.1 GPT-4 Based Evaluation

The evaluation uses GPT-4 as a safety evaluator against OpenAI’s user policy and checks its judgments against human ratings. It also includes a text-style instruction for comparison.

B.1 GPT-4 Based Evaluation

  • GPT-4’s evaluation ability was prompted for use as a safety evaluator.Source: §B.1 · p. 9
  • OpenAI’s user policy served as the benchmark for assessing whether LLM outputs violated that policy.Source: §B.1 · p. 9
  • 93% agreement was observed between GPT-4-based evaluation and human judgments on 100 GPT-4-1106 outputs under CodeChameleon.Three authors rated outputs in two categories, with majority vote determining the final results.Source: §B.1 · p. 9

B.2 Text-style Instruction

  • A text-style instruction was additionally designed for comparison.Source: §B.2 · p. 9

B.3 Decryption Capability Assessment

The decryption-capability assessment uses GPT-4 to judge whether decrypted queries match their originals. Its prompts instruct models to decrypt the query before planning a solution.

B.3 Decryption Capability Assessment

  • GPT-4 judges the consistency of decrypted queries with their original queries in the decryption-capability assessment.The prompt used for this judgment is shown in Figure 10.Source: §B.3 · p. 9, Figure 10 · p. 16
  • The examples include reverse-order encryption and decryption functions and an encrypted query whose original asks for hazardous-device guidance.Table 5 presents encrypted-query examples from different encryption functions.Source: §B.3 · pp. 10–11, Table 5 · p. 13
  • The task instructions direct the model to recover the original problem with the decryption function, then plan subtasks and their solutions.The prompt also supplies a decryption function.Source: §B.3 · p. 13
  • The prompt includes OpenAI prohibited-use policies and instructs models to refuse disallowed requests.The listed policy categories include illegal activity, child exploitation, hateful or violent content, and malware generation.Source: §B.3 · pp. 14–15
Loading 2402.16717v1…