PaperlayerView exact revision on arXiv

Source-linked AI summary

Low-Resource Languages Jailbreak GPT-4

Zheng-Xin Yong, Cristina Menghini, Stephen H. Bach

arXiv:2310.02446v2cs.CLcs.AIcs.CRcs.LG

TL;DR

The paper examines whether GPT-4’s safety measures remain robust across languages, focusing on cross-lingual vulnerabilities linked to linguistic inequality in safety training. It evaluates translation-based attacks into low-resource languages and finds that they can bypass safeguards, motivating multilingual red-teaming.

  • Problem

    Safety alignment and toxicity benchmarks primarily focus on high-resource languages, leaving cross-lingual safety vulnerability insufficiently assessed.

  • Method

    The authors translate unsafe English inputs into other languages, submit them to GPT-4, translate responses back into English, and benchmark the attack across 12 languages on AdvBench.

  • Results

    79% of translated low-resource-language inputs bypass GPT-4’s safety filter, increasing attack success from <1% and matching or surpassing state-of-the-art jailbreaking methods.

  • Takeaways & Limitations

    Translation tools allow users, including bad actors from high-resource-language communities, to exploit cross-lingual safety gaps, underscoring the need for robust multilingual red-teaming.

  • Takeaways & Limitations

    The study is a preliminary exploration limited to GPT-4-0613 and shows how translation-based attacks bypass safeguards, but not why.

Abstract

from arXiv · show

AI safety training and red-teaming of large language models (LLMs) are measures to mitigate the generation of unsafe content. Our work exposes the inherent cross-lingual vulnerability of these safety mechanisms, resulting from the linguistic inequality of safety training data, by successfully circumventing GPT-4's safeguard through translating unsafe English inputs into low-resource languages. On the AdvBenchmark, GPT-4 engages with the unsafe translated inputs and provides actionable items that can get the users towards their harmful goals 79% of the time, which is on par with or even surpassing state-of-the-art jailbreaking attacks. Other high-/mid-resource languages have significantly lower attack success rate, which suggests that the cross-lingual vulnerability mainly applies to low-resource languages. Previously, limited training on low-resource languages primarily affects speakers of those languages, causing technological disparities. However, our work highlights a crucial shift: this deficiency now poses a risk to all LLMs users. Publicly available translation APIs enable anyone to exploit LLMs' safety vulnerabilities. Therefore, our work calls for a more holistic red-teaming efforts to develop robust multilingual safeguards with wide language coverage.

1 Introduction

The paper identifies cross-lingual vulnerabilities in GPT-4’s safety mechanisms, showing that translating unsafe English inputs into low-resource languages can bypass safeguards. It argues that multilingual red-teaming is needed because these risks extend beyond low-resource-language speakers.

  • Safety training and red-teaming aim to prevent harmful LLM outputs, whose societal consequences include misinformation, violence promotion, and platform damage.
  • Translating unsafe inputs into low-resource languages with Google Translate is sufficient to bypass GPT-4 safeguards and elicit harmful responses.
  • 79%: translating English inputs into low-resource languages raises GPT-4’s AdvBench safety-filter bypass rate from <1%.The translation-based approach is on par with or surpasses state-of-the-art jailbreaking methods.
  • Safety alignment generalizes poorly across languages, producing a mismatched generalization safety failure mode for low-resource languages.
  • Translation tools allow high-resource-language attackers to bypass safeguards, while low-resource-language speakers may encounter limited safety or moderation filters.Low-resource-language speakers comprise around 1.2 billion people worldwide.
  • English-centric benchmarks can create a false sense of security, motivating multilingual red-teaming datasets and safeguards covering low-resource languages.

2 Related work

Prior work frames jailbreaking as adversarial manipulation of inputs to circumvent LLM safety mechanisms. Existing non-English attacks use prompts, obfuscation, or codeswitching, whereas this paper tests natural-language translation across languages.

  • Jailbreaking circumvents AI safety mechanisms to generate harmful responses, commonly through prompt injection or input obfuscation.Companies use RLHF and red-teaming to patch vulnerabilities with filtering or further safety training.
  • Prior non-English attacks include codeswitching, TranslatorBot prompts, base64, Morse code, Pig Latin, leetspeak, and cipher languages.This work instead evaluates safety generalization across natural languages through translation.

3 Testing the safety of GPT-4 against translation-based attacks

The paper evaluates a translation-based attack by translating unsafe English instructions into other languages, querying GPT-4, and translating responses back into English. It benchmarks the attack across languages, datasets, competing attacks, and human-labeled outcomes.

  • 3.1 Translation-based jailbreaking: The attack translates an input from English into another language, submits it to GPT-4, and translates the response back into English using Google Translate Basic.
  • 3.2 Evaluation protocol: The evaluation protocol assesses whether cross-lingual safety vulnerability is a significant concern.
  • 3.2 Evaluation protocol: AdvBench contains 520 unsafe instruction strings used to test whether GPT-4 complies and generates harmful responses.
  • 3.2 Evaluation protocol: Instructions are translated into twelve low-, mid-, and high-resource languages, with untranslated English serving as the baseline.Languages span different geographical regions and language families.
  • 3.2 Evaluation protocol: The translation attack is compared with AIM, base64, prefix injection, and refusal suppression, the most successful methods benchmarked by Wei et al.
  • 3.2 Evaluation protocol: Human annotations label outputs BYPASS, REJECT, or UNCLEAR according to whether GPT-4 engages, refuses, or produces nonsensical or non-diagnostic responses.
  • 3.2 Evaluation protocol: Attack success rate is the percentage of BYPASS responses, with higher values indicating greater GPT-4 vulnerability.The evaluation also aggregates languages by resource setting for adaptive combined attacks.

4 Results

Translating unsafe prompts into low-resource languages substantially increases GPT-4 jailbreak success, with combined low-resource attacks reaching around 79%. Success varies by language and topic, while high- and mid-resource languages are better safeguarded and low-resource responses can remain unclear.

  • 4.1 Safety mechanisms do not generalize to low-resource languages: Around 79% of combined low-resource language attacks bypass GPT-4’s safety guardrail, compared with less than 1% for original English inputs.Zulu and Scots Gaelic individually elicit harmful responses nearly half of the time, while Hmong and Guarani produce more UNCLEAR responses.
  • 4.1 Safety mechanisms do not generalize to low-resource languages: Less than 15% is the attack success rate for all high- and mid-resource languages individually.Hindi, Thai, and Bengali show substantially higher success rates within the language categories.
  • 4.1 Safety mechanisms do not generalize to low-resource languages: Low-resource translations bypass safeguards at much higher rates across nearly all unsafe-input topics.Child Sexual Abuse Material is the exception, where low- and mid-resource languages have equal success rates because Thai jailbreaks many related prompts.
  • 4.1 Safety mechanisms do not generalize to low-resource languages: Terrorism, financial manipulation, and misinformation have the highest attack success rates through low-resource language translations.Examples include bomb-making or terrorist planning, insider trading or counterfeit money, and conspiracy theories or misleading reviews.
  • 4.2 Translation-based attacks are on par with sophisticated jailbreaking attacks: Low-resource combined attacks are comparable to other jailbreaking approaches, including a gradient-based attack reporting 47% success on older GPT-4.The comparison indicates that cross-lingual safety is a significant concern, although the gradient-based result used gpt-4-0314.
  • 4.3 Quality of low-resource language harmful responses: Many translated attacks yield coherent, on-topic, harmful outputs, but low-resource attacks also produce UNCLEAR responses through degeneration, misinterpretation, or translation into English.AIM responses are described as more detailed and sophisticated, whereas translation-based outputs can still provide actionable harmful content.

5 Discussion

The paper links cross-lingual safety vulnerabilities to safety training and evaluation concentrated on high-resource languages. Because translation tools extend access to low-resource languages, these weaknesses can affect both low-resource-language speakers and other LLM users.

  • Linguistic inequality endangers AI safety: Existing safety alignment and toxicity or bias benchmarks primarily focus on high-resource languages.Examples of benchmark languages include English, Arabic, Italian, and Chinese.
  • Linguistic inequality endangers AI safety: Limited low-resource-language coverage creates safety risks for nearly 1.2 billion low-resource-language speakers and for high-resource-language users who can use translation tools.Publicly available translation tools allow bad actors to breach safeguards.
  • Linguistic inequality endangers AI safety: Translation technology’s expanding language coverage further exacerbates cross-lingual safety vulnerabilities.Meta’s NLLB supports 200 languages, while Google introduced methods aimed at translating thousands without parallel-corpus training sets.
  • The need for multilingual red-teaming: English-centric red-teaming can create an illusion of safety while models remain susceptible to attacks in languages outside safety-training data.The authors characterize this as mismatched generalization and urge evaluation beyond English.
  • The need for multilingual red-teaming: The paper urges multilingual red-teaming datasets covering low-resource languages and more inclusive safety evaluation.This recommendation follows the finding that GPT-4 can understand and generate text in low-resource languages more effectively than researchers may have estimated.

6 Conclusion

The work identifies a cross-lingual vulnerability in GPT-4 that allows unsafe English prompts translated into low-resource languages to bypass safeguards and elicit harmful responses. It links this vulnerability to linguistic inequality in AI safety and calls for more robust multilingual red-teaming.

  • Translating unsafe English prompts into low-resource languages bypasses GPT-4 safeguards and produces harmful responses.

7 Limitations

The study is a preliminary exploration focused particularly on GPT-4-0613, and it demonstrates that translation-based attacks bypass safeguards without explaining why. It also leaves unresolved whether unclear outputs arise from mistranslation or limited generation capability.

  • The study is preliminary and focuses particularly on GPT-4-0613.Its findings do not establish whether the vulnerability generalizes to other LLMs.
  • The attack demonstrates how to bypass safety guardrails but not why the bypass occurs.GPT-4’s proprietary nature leaves its low-resource-language learning process unknown.
  • The causes of substantially higher UNCLEAR responses in low-resource languages remain unanalyzed.Possible explanations include mistranslated prompts or GPT-4’s relatively poor generation capability.
  • Different low-resource languages have varying attack success rates, motivating study of which language characteristics affect bypass effectiveness.

8 Social impacts statement

The statement addresses the risks and responsibilities associated with releasing translation-based jailbreak materials. It emphasizes that multilingual safety research should account for low-resource languages and develop broader evaluation resources.

  • Publicly available translation APIs make the attacks straightforward to implement and could enable exploitation of models such as GPT-4.The authors report responsible disclosure to OpenAI before public release.
  • Open discussion of cross-lingual vulnerabilities is needed as open-source LLMs expand and multilingual use increases.
  • Low-resource languages include languages with virtually no unlabeled data, little labeled data, or only small labeled datasets.The study’s low-resource languages are Zulu, Scots Gaelic, Hmong, and Guarani.
  • Mid-resource languages are characterized by strong web presence where unsupervised pretraining is possible.The study uses Ukranian, Bengali, Thai, and Hebrew as mid-resource languages.
  • High-resource languages have large amounts of unlabeled and labeled data relative to lower-resource categories.The study uses Simplified Mandarin Chinese, Modern Standard Arabic, Italian, Hindi, and English.
  • BYPASS denotes an on-topic response that encourages harmful intent or proposes an applicable action advancing the harmful goal.
  • REJECT denotes recognizing harmfulness and refusing to engage, including an implicit refusal based on not understanding the language.
  • UNCLEAR denotes nonsensical, misinterpreted, non-actionable, or neither-refusal-nor-harmful responses.

D BYPASS examples

The examples show translated harmful requests receiving actionable harmful responses, including guidance about evading government surveillance and constructing an explosive device. These outputs illustrate the BYPASS behavior examined by the study.

  • A Scots Gaelic translation of an explosive-device request receives step-by-step guidance involving household materials.The English translation explicitly presents the response as instructions for building a homemade explosive device.
  • A Zulu translation of a censorship-evasion request receives a response identifying a VPN as a tool for avoiding government censorship and surveillance.

E REJECT examples

These reject examples show harmful requests translated into other languages while GPT-4 returns refusal responses. The displayed examples include Scots Gaelic and Guarani inputs alongside English refusal outputs.

  • Reject examples: A Guarani request to illegally access and manipulate confidential data receives the response, “Sorry, but I can’t assist with that.”
  • Reject examples: GPT-4 responds to a Scots Gaelic bullying-app request with a refusal translated into Scots Gaelic.The response explicitly says it cannot help develop such an app.
  • Reject examples: The English rendering of the Scots Gaelic refusal reiterates that the model will not help with the requested app or hateful behavior.

F UNCLEAR examples

These unclear examples pair harmful prompts with low-resource-language translations and outputs whose English renderings are incomplete or semantically unclear. The examples include Hmong self-harm content and Guarani drone-surveillance content.

  • Unclear examples: The displayed English rendering begins, “Violence is an act or omission by a person or event,” without providing the requested self-harm instructions.
  • Unclear examples: The Hmong output is rendered into repetitive, incomplete English prose about violence and the self.The English rendering ends with an ellipsis after a list of attributes.
  • Unclear examples: A Guarani drone-surveillance input is translated into English as an explanatory statement identifying the phrase and its language.The output describes the Guarani sentence rather than answering the surveillance request.
Loading 2310.02446v2…