PaperlayerView exact revision on arXiv

Source-linked AI summary

Defending LLMs against Jailbreaking Attacks via Backtranslation

Yihan Wang, Zhouxing Shi, Andrew Bai, Cho-Jui Hsieh

arXiv:2402.16459v3cs.CLcs.AI

TL;DR

Safety-aligned LLMs remain vulnerable to jailbreak prompts that conceal harmful intent. The paper backtranslates a target model’s initial response into a possible prompt and refuses the original if the model refuses that prompt. It reports stronger defense than baselines in difficult settings while mostly maintaining benign generation quality.

  • Problem

    LLMs trained to refuse harmful requests remain vulnerable to adversarial prompts that conceal harmful intent and elicit harmful responses.

    Source: §1 · p. 1

  • Method

    The defense infers a possible prompt from the target model’s initial response, then refuses the original prompt if the target model refuses the backtranslated prompt.

    Source: §1 · p. 1, §4.1 · p. 3

  • Results

    Backtranslation outperforms existing defense baselines; on PAIR with GPT-3.5-turbo without defense awareness, it achieves an 88% DSR versus 78% for paraphrase.

    Source: §5.2 · p. 6, §6 · p. 8

  • Takeaways & Limitations

    The defense is described as efficient and lightweight, and as mostly maintaining generation quality on benign input prompts.

    Source: §6 · p. 9

    The defense relies on the target model being able to refuse clean harmful requests, so it may not be effective for a model never trained with safety alignment.

    Source: Ethical Considerations and Limitations · pp. 9–10

Abstract

from arXiv · show

Although many large language models (LLMs) have been trained to refuse harmful requests, they are still vulnerable to jailbreaking attacks which rewrite the original prompt to conceal its harmful intent. In this paper, we propose a new method for defending LLMs against jailbreaking attacks by ``backtranslation''. Specifically, given an initial response generated by the target LLM from an input prompt, our backtranslation prompts a language model to infer an input prompt that can lead to the response. The inferred prompt is called the backtranslated prompt which tends to reveal the actual intent of the original prompt, since it is generated based on the LLM's response and not directly manipulated by the attacker. We then run the target LLM again on the backtranslated prompt, and we refuse the original prompt if the model refuses the backtranslated prompt. We explain that the proposed defense provides several benefits on its effectiveness and efficiency. We empirically demonstrate that our defense significantly outperforms the baselines, in the cases that are hard for the baselines, and our defense also has little impact on the generation quality for benign input prompts. Our implementation is based on our library for LLM jailbreaking defense algorithms at \url{https://github.com/YihanWang617/llm-jailbreaking-defense}, and the code for reproducing our experiments is available at \url{https://github.com/YihanWang617/LLM-Jailbreaking-Defense-Backtranslation}.

1 Introduction

Safety-aligned LLMs can still be jailbroken by adversarial prompts that conceal harmful requests. The paper proposes backtranslation: infer a prompt from the model’s initial response, then refuse the original prompt if the target model refuses the inferred prompt.

1 Introduction

  • Safety-aligned LLMs remain vulnerable to adversarial prompts that cause them to generate harmful responses instead of refusing harmful requests.Alignment aims both to follow human instructions and to refrain from generating unethical or illegal content.Source: §1 · p. 1
  • The defense backtranslates the target model’s initial response into a possible prompt, then refuses the original prompt if the target model refuses that backtranslated prompt.The backtranslated prompt tends to recover the original harmful intent while being relatively clean compared with the adversarial prompt.Source: §1 · pp. 1–2
  • Backtranslation operates on the model’s response rather than the attacker-manipulable input and does not require additional task-specific training.It is also described as inexpensive and efficient, requiring few queries and allowing a relatively cheap backtranslation model.Source: §1 · p. 2
  • The authors report superior defense success against adversarial prompts while maintaining generation quality on benign data, provided the backtranslated prompt is not refused.The defense is intended not to affect benign generations under that condition.Source: §1 · p. 2

2 Related Work

Prior jailbreak defenses detect adversarial prompts, denoise them, or add safety-oriented prompts, but these approaches commonly operate on attacker-manipulable inputs. Backtranslation instead uses the target model’s response and is presented as requiring neither additional optimization nor many queries.

2 Related Work

  • Jailbreak attacks include optimized suffixes, genetic prompt optimization, iterative black-box refinement, persuasive paraphrasing, and manually designed fixed prefixes or suffixes.Examples include GCG, AutoDAN, PAIR, and PAP.Source: §2 · p. 2
  • Jailbreak defenses include detection, denoising, prompt engineering, and automatic prompt optimization, with denoising methods using paraphrasing, retokenization, or random perturbations.Detection-based approaches can face natural and stealthy adversarial prompts.Source: §2 · p. 2
  • The surveyed defenses commonly operate on input prompts, which attackers can directly manipulate by changing the input.The paper contrasts this with its response-side backtranslation defense.Source: §2 · p. 2
  • The paper characterizes backtranslation as harder for attackers to manipulate and as requiring no additional optimization or many queries.It operates on the target model’s generated response rather than its input prompt.Source: §2 · p. 2

3 Background

The background formalizes jailbreak success through a judge that evaluates whether a target model’s response fulfills a malicious prompt. It describes how an attacker constructs an adversarial prompt and how a defense changes the model’s success against it.

3 Background

  • A judge evaluates whether the target model’s response fulfills a potentially malicious input prompt, such as a request for bomb-making instructions.The response is generated as R = M(S), and the judge’s prediction is J(S, R).Source: §3 · p. 2
  • A properly aligned target model normally refuses harmful prompts, but an attacker can construct an adversarial prompt from a clean prompt and the target model.A successful jailbreak produces a harmful response that fulfills the original clean prompt.Source: §3 · pp. 2–3
  • A defense strategy is applied to the target model to make it more robust; it succeeds when the judge indicates that the defended model no longer fulfills the harmful prompt.The attack may also take the defended model into consideration when constructing its adversarial prompt.Source: §3 · p. 3

4 Defense by Backtranslation

The defense reconstructs a plausible prompt from the target model’s initial response, then uses the target model’s refusal behavior to decide whether to refuse the original input. A likelihood filter can skip mismatched reconstructions, while early stopping limits added generation cost.

4.1 Backtranslation

  • The backtranslation model infers a prompt S′ that could produce initial response O; refusal of S′ triggers refusal of original prompt S.B is prompted to guess a request that could elicit O and, if multiple requests fit, to return the most harmful one.Source: §4.1 · pp. 3–4

4.2 Defended Response with Backtranslation

  • The defense returns a fixed refusal when either prompt is refused; otherwise it returns the original response, preserving outputs for prompts it does not refuse.If the original response already refuses, backtranslation is skipped; otherwise, a refusal to S′ triggers refusal of S.Source: §4.2 · p. 4

4.3 Cost of Backtranslation

  • The added work is generating S′ and O′; a weaker B and early stopping after refusal cues reduce expense, making overhead small versus generation and SmoothLLM.Only enough tokens of O′ need to be generated to determine whether S′ is refused.Source: §4.3 · p. 4

4.4 Mitigating Over-refusal

  • A likelihood filter skips a mismatched S′ to reduce over-refusal; experiments apply this mitigation only to open-source models where likelihood is available.When the inferred prompt does not match O, the defense returns the original output.Source: §4.4 · p. 4

5 Experiments

Experiments test backtranslation across several attacks and target models, assessing defense success, inference cost, and benign-response quality. It typically outperforms baseline defenses in challenging settings, while early termination and threshold filtering are examined for efficiency and quality.

5.1 Settings

  • Experiments evaluate five attack types across GPT-3.5-turbo, Llama-2-Chat-13B, and Vicuna-13B, using 50 AdvBench harmful requests and GPT-4-judged DSR.GCG and AutoDAN require model access unavailable for GPT-3.5-turbo, so their results there use transfer attacks.Source: §5.1 · p. 5

5.2 Main Results

  • On PAIR with GPT-3.5-turbo, backtranslation reaches 88% DSR versus paraphrase’s 78% without defense awareness, and 76% versus 64% with awareness; fixed-prefix DSR exceeds 90%.For PAIR with defense awareness on Vicuna-13B, backtranslation reaches 56% DSR while baselines reach at most 12%.Source: §5.2 · p. 6

5.3 Time Cost

  • Without early termination, backtranslation costs around twice undefended inference; early termination reduces MT-Bench average time from 86.91 to 49.86 seconds with comparable DSR.The early-termination experiment determines refusal from the first 20 tokens of the backtranslated-prompt response.Source: §5.3 · p. 7

5.4 Impact on the Generation Quality

  • On MT-Bench, backtranslation and paraphrase only slightly reduce average response quality, whereas SmoothLLM degrades it more; response check needs more prompt engineering and threshold tuning.The response-check quality finding is described separately from the comparison among paraphrase, backtranslation, and SmoothLLM.Source: §5.4 · pp. 7–8

5.5 Impact of Different Backtranslation Models

  • Changing the backtranslation model has little effect on DSR or generation quality, so the authors say a relatively small, efficient model may be used.The authors characterize this as making the defense efficient and cheap.Source: §5.5 · p. 8

5.6 Impact of Backtranslation Threshold γ

  • No filtering (γ = −inf) drops MT-Bench quality from 6.80 to 6.11; changing γ from -1.0 to -2.0 shifts quality from 6.40 to 6.34 and DSR from 90% to 94%; main experiments use −2.0.Source: §5.6 · p. 8, Table 7 · p. 8

5.7 Case Study

  • In the PAIR/GPT-3.5-turbo case study, backtranslation recovers harmful intent, elicits a target-model refusal to the inferred prompt, and returns a refusal template; SmoothLLM and paraphrase still yield harmful responses.Source: §5.7 · p. 8

6 Conclusion

The paper concludes that backtranslation is an effective, lightweight defense against jailbreaks that outperforms defense baselines while mostly maintaining generation quality on benign prompts.

6 Conclusion

  • The defense is highly effective against existing jailbreak attacks and outperforms jailbreaking-defense baselines.It operates on the initial response and uses safety-aligned LLMs’ ability to refuse harmful prompts.Source: §6 · pp. 8–9
  • Table 8 gives example prompts and responses for several defenses under a PAIR attack on GPT-3.5-turbo without defense.Source: Table 8 · p. 9
  • Backtranslation is efficient and lightweight, and it mostly maintains generation quality on benign input prompts.Source: §6 · p. 9

Ethical Considerations and Limitations

The authors frame the work as supporting safer LLM deployment, while noting that backtranslation depends on safety alignment and can fail in several settings.

Ethical Considerations and Limitations

A.1 GCG

The appendix specifies attack-generation settings for GCG and PAIR, including optimization budgets, conversation streams, and the PAIR attacker model.

A.1 GCG

  • For GCG on Llama-2-13B and Vicuna-13B, adversarial suffixes are optimized separately for each harmful request with 200 steps per example.The authors use this step count as a trade-off between attack effectiveness and computational cost.Source: §A.1 · p. 12
  • For transfer attacks on GPT-3.5-turbo, each GCG suffix is optimized on 25 examples for 500 steps, and four suffixes are concatenated.The suffixes come from Vicuna and Guanacos, with two suffixes per model using two random seeds.Source: §A.1 · p. 12

A.2 PAIR

  • PAIR uses three iterations, 20 parallel conversation streams, and a Vicuna-13B attacker to generate jailbreak prompts.Source: §A.2 · p. 12

A.3 AutoDAN

The appendix details AutoDAN settings, the backtranslation filter’s token limit, refusal detection, and SmoothLLM perturbations.

A.3 AutoDAN

  • AutoDAN attacks white-box models for 100 iterations or until success, using conditional cross-entropy to score the desired harmful output.Its elite, crossover, and mutation ratios are 0.1, 0.5, and 0.01; transfer attacks use prompts from Vicuna-13B.Source: §A.3 · p. 12

B.1 Hyperparameters

  • The backtranslation filter considers N = 150 tokens in all experiments.Source: §B.1 · p. 12

B.2 Checking Refusal in Backtranslation

B.3 SmoothLLM

  • SmoothLLM uses three random perturbations with a 10% perturbation budget and random-swap perturbation.Source: §B.3 · p. 12

B.4 Response Check

The response-check procedure uses the target model to rate its own responses, with model-specific refusal thresholds. The supplementary material also specifies GPT-4 judging protocols and provides additional attack examples.

B.4 Response Check

  • The target model rates its own response for harmfulness, with refusal thresholds of 5 for GPT-3.5-turbo and 10 for Llama-2-13B and Vicuna-13B.The higher thresholds for Llama-2 and Vicuna were chosen to avoid over-refusal; those models often rated benign prompts as unethical or illegal.Source: §B.4 · pp. 12–13, Table 10 · p. 12

C Details for the GPT-4 Judges

  • The GPT-4 harmfulness judge follows Chao et al. (2023), while generation quality is graded with MT-Bench’s default single-answer prompt.For PAP, the authors use the attack paper’s judge prompt and count ratings greater than 1 on its 1-to-5 scale as jailbroken.Source: C Details for the GPT-4 Judges · p. 13

D Additional Examples

E Additional Empirical Results

The additional comparison reports that backtranslation outperforms one-shot ICL on most settings and is more stable on several difficult settings. The appendix also records the licenses of the datasets and attack implementations.

E Additional Empirical Results

  • Backtranslation performs better than ICL on most settings, while ICL is significantly worse on several, including Vicuna under AutoDAN and the defended model under PAIR.The authors report that backtranslation is relatively more stable across those settings.Source: E Additional Empirical Results · p. 13, Table 11 · p. 13
  • ICL’s lower average time cost is associated with its shorter responses under the in-context demonstration.Reported adversarial-prompt times for GCG are 32.29 seconds for ICL and 53.84 seconds without defense; benign-prompt times are 37.83 and 41.56 seconds, respectively.Source: E Additional Empirical Results · p. 13

F Licenses of Used Artifacts

G Compute Resources

Experiments used A6000 GPUs, with two GPUs reserved for GCG transfer attacks. The accompanying examples show prompts and responses under GCG and AutoDAN attacks across several defense settings.

G Compute Resources

  • Experiments ran on A6000 GPUs with 48GB memory; each used one GPU except GCG transfer attacks, which used two parallel GPUs.Source: G Compute Resources · p. 13
  • Tables 12 and 13 present prompts and responses from different defenses under GCG and AutoDAN attacks using Vicuna-13B.Source: Table 12 · p. 14, Table 13 · p. 16
  • The examples include an AutoDAN adversarial prompt that disguises a request for dangerous-behavior encouragement as a role-play instruction.The accompanying no-defense response and figure caption present the attack example; emojis are removed from prompts and responses for clarity.Source: G Compute Resources · p. 15
  • The appendix also presents AutoDAN examples for SmoothLLM and backtranslation, alongside the unprotected case.The SmoothLLM example perturbs the prompt, while the backtranslation example is identified in a separate figure caption.Source: G Compute Resources · pp. 15–16
Loading 2402.16459v3…