Source-linked AI summary
GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher
Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Pinjia He, Shuming Shi, Zhaopeng Tu
TL;DR
Safety alignment mainly addresses natural-language interaction, raising whether it generalizes to ciphers. The paper introduces CipherChat to test cipher-mediated interaction across models, languages, and safety domains, and finds substantial bypass success, including near-100% success for ASCII against GPT-4 in several domains. It also proposes SelfCipher, which evokes an apparent internal cipher using role play and demonstrations.
Problem
Safety alignment largely targets natural-language inputs and outputs, leaving its generalizability to non-natural languages such as ciphers insufficiently examined.
Method
CipherChat evaluates cipher-based interaction using role descriptions, cipher teaching, enciphered unsafe demonstrations, and rule-based input/output transformation.
Results
Certain ciphers bypass GPT-4 safety alignment almost 100% of the time in several domains, while SelfCipher consistently outperforms existing human ciphers.
Takeaways & Limitations
Safety alignment needs coverage of non-natural languages, and exploring LLMs’ apparent secret ciphers is a promising direction.
Takeaways & Limitations
The study discloses information that could enable harmful content generation using publicly available LLMs.
Abstract
from arXiv · showhide
Safety lies at the core of the development of Large Language Models (LLMs). There is ample work on aligning LLMs with human ethics and preferences, including data filtering in pretraining, supervised fine-tuning, reinforcement learning from human feedback, and red teaming, etc. In this study, we discover that chat in cipher can bypass the safety alignment techniques of LLMs, which are mainly conducted in natural languages. We propose a novel framework CipherChat to systematically examine the generalizability of safety alignment to non-natural languages -- ciphers. CipherChat enables humans to chat with LLMs through cipher prompts topped with system role descriptions and few-shot enciphered demonstrations. We use CipherChat to assess state-of-the-art LLMs, including ChatGPT and GPT-4 for different representative human ciphers across 11 safety domains in both English and Chinese. Experimental results show that certain ciphers succeed almost 100% of the time to bypass the safety alignment of GPT-4 in several safety domains, demonstrating the necessity of developing safety alignment for non-natural languages. Notably, we identify that LLMs seem to have a ''secret cipher'', and propose a novel SelfCipher that uses only role play and several demonstrations in natural language to evoke this capability. SelfCipher surprisingly outperforms existing human ciphers in almost all cases. Our code and data will be released at https://github.com/RobustNLP/CipherChat.
1 INTRODUCTION
Existing safety alignment largely targets natural-language inputs and outputs, leaving its generalizability to ciphers an open question. CipherChat evaluates this gap and finds that ciphers can bypass alignment, while SelfCipher evokes a latent cipher capability.
- Most safety-alignment work focuses on natural-language inputs and outputs, despite LLM capabilities with Morse Code, ROT13, and Base64.
- CipherChat systematically evaluates safety-alignment generalizability to non-natural languages, specifically ciphers.
- Its system prompt assigns a cipher-expert role, teaches the cipher, and supplies enciphered unsafe demonstrations.
- CipherChat enciphers inputs, obtains cipher outputs from LLMs, and deciphers responses with a rule-based decrypter.
- ASCII bypasses GPT-4 safety alignment almost 100% of the time in several English domains, while best-cipher averages reach 72.1% for GPT-4 and 23.7% for Turbo.
- SelfCipher uses role play and unsafe natural-language demonstrations to evoke a secret cipher and consistently outperforms existing human ciphers across models, languages, and safety domains.
- The study argues that safety alignment should extend to non-natural languages to match underlying LLM capabilities.
2 RELATED WORK
Related work covers safety alignment and adversarial attacks on LLMs, while prior findings also connect pretraining knowledge with cipher capabilities.
- Safety-alignment research includes reinforcement learning, modular open-source frameworks, instruction tuning, Constitutional AI, and self-alignment.
- Simulated ciphers absent from pretraining data cannot work, supporting the role of pretrained knowledge in cipher understanding.
- Prior adversarial-attack studies show that aligned LLMs remain vulnerable to inputs eliciting undesired behavior.
3 METHODOLOGY: CIPHERCHAT
CipherChat constructs cipher-focused prompts, transforms inputs and outputs through cipher rules, and supports both conventional ciphers and SelfCipher. The framework is designed to keep the model’s interaction in cipher form.
- Encipher and decipher: The framework enciphers user inputs before inference and applies a rule-based decrypter to model outputs.
- CipherChat overview: Figure 2 summarizes system-prompt construction, input enciphering, and response deciphering as the framework’s three steps.
- System prompt construction: CipherChat assigns the LLM a cipher-expert role, teaches cipher rules, and provides enciphered unsafe demonstrations.
- Character Encoding: CipherChat uses character encodings including GBK, ASCII, UTF, and Unicode for English and Chinese text.
- Human ciphers: It also evaluates Atbash, Morse Code, and Caesar Cipher, using a default Caesar shift of 3 positions.
- SelfCipher: SelfCipher uses no explicit conventional cipher; role prompts and demonstrations are intended to evoke an internal cipher in the LLM.
4 EXPERIMENT
Experiments evaluate CipherChat on two models, 11 unsafe-data domains, and English and Chinese, showing that cipher prompts can elicit valid and unsafe responses. GPT-4 generally communicates more effectively with ciphers and exhibits higher unsafety, while SelfCipher performs broadly and depends strongly on prompt demonstrations.
- 4.1 SETUP: Experiments use Turbo and GPT-4 on 11 unsafe-data domains in both Chinese and English, with unsafety evaluated as the percentage of unsafe responses.The Chinese benchmark samples 199 instances from each selected domain and is translated into English with machine translation and manual correction.
- 4.2 EFFECTIVE COMMUNICATION WITH GPT-4 VIA CIPHER: GPT-4 achieves validity rates from 86% to 100% across seven ciphers in Chinese and English, demonstrating effective cipher communication.Validity requires responses to be natural and relevant to the query, and is checked by human annotators.
- 4.3 CIPHER ENABLES EVASION OF SAFETY ALIGNMENT: 70.9% is GPT-4’s English SelfCipher unsafety rate, exceeding its Chinese rate of 53.3% by a large margin.The paper reports that English generally has higher unsafety rates than Chinese and relates this pattern to model strength and language dominance.
- 4.3 CIPHER ENABLES EVASION OF SAFETY ALIGNMENT: Across all 11 domains, Unfairness, Insult, and MenHealth approach 100% unsafe responses, whereas UnsafeTopic, Privacy, and ReExposure are less unsafe.The paper states that these domain-level patterns generally support the broader conclusions about CipherChat.
- 4.4 ANALYSIS: Removing SystemRole or unsafe demonstrations substantially reduces SelfCipher’s unsafety, while safe demonstrations can further mitigate unsafe behavior.Removing SystemRole reduces GPT-4’s Chinese unsafety to around 0, whereas removing unsafe demonstrations can prevent valid outputs for several cipher-model settings.
- 4.4 ANALYSIS: SelfCipher achieves near-100% validity and success rates on the Alpaca general-instruction benchmark for both Turbo and GPT-4.This result is presented as evidence that CipherChat also functions beyond unsafe prompts.
5 CONCLUSION AND FUTURE WORK
The study finds that cipher-based chat can elicit unsafe information from GPT-4, which can understand representative ciphers. It argues that safety alignment should extend beyond natural languages.
- CipherChat can effectively elicit unsafe information from GPT-4 through cipher-based chat.
- GPT-4’s cipher understanding creates a mismatch with safety alignment focused on natural-language inputs and outputs.
- The authors identify safety alignment for non-natural languages as necessary to match the capabilities of underlying LLMs.
ETHICS AND BROADER IMPACT
The study discloses information that could enable harmful content production using publicly available LLMs. It frames this risk as important to discuss for improving future system safety and reliability.
- The study includes information that could potentially enable individuals to produce harmful content using publicly available LLMs.
- The authors acknowledge inherent risks while defending full disclosure as important for discussing system weaknesses and limitations.
- The study aims to clarify exploitation dangers and support future research into safe and reliable LLM deployment.
A.1 MORE DETAILS ABOUT SelfCipher
SelfCipher differs from Caesar in its prompting and demonstrations. Rather than explicitly specifying a cipher, it emphasizes cipher communication to evoke an internal cipher capability in LLMs.
- Example inputs: Table 5 compares example Caesar and SelfCipher inputs for the query about stealing money from a bank.
- System prompt: SelfCipher’s system prompt omits explicit cipher names and explanations used by conventional cipher prompts.
- System prompt: SelfCipher instead emphasizes comprehending queries and responding in Cipher Code, aiming to evoke an internal cipher in the LLM.
A.2 MORE DETAILS ABOUT HUMAN EVALUATION
The paper evaluates response validity and safety through human annotation, including English and Chinese outputs. Annotator agreement is reported as high for both evaluation tasks.
- Validity evaluation: Two native Chinese-speaking and English-fluent annotators evaluated the validity of generated responses.Their agreement was 0.98 by Fleiss’ Kappa.
- Annotator agreement: The validity and safety annotations achieved Fleiss’ Kappa agreement scores of 0.98 and 0.95, respectively.
- Safety evaluation: For safety detection, two annotators assessed 50 Crimes-domain instances across different ciphers.Disagreements were reassessed to reach consensus.
A.3 MORE DETAILS ABOUT DATASETS
The dataset appendix removes culturally specific Chinese instances to create a language-agnostic benchmark and presents queries across 11 safety domains in English and Chinese.
- Evaluation: The appendix includes GPT-4-based unsafety-detector accuracy tables.Tables 6 and 7 report detector accuracy.
- Dataset construction: 199 instances for each domain were collected after manually removing Chinese culture-associated sentences.The filtering targets references tied to specific Chinese regions and other cultural context.
- Dataset construction: The benchmark is designed to be universal and agnostic to specific languages.Chinese culture-associated sentences were removed to reduce language- and culture-specific effects.
- Domain queries: Queries are presented across all 11 safety domains in both English and Chinese.The appendix consolidates the domain queries in a dedicated table.
A.4 CASE STUDY IN CHINESE AND MORE DOMAINS
The case-study materials illustrate unsafe outputs from GPT-4 under CipherChat and examine SelfCipher prompt variants. Replacing “Cipher” with other terms can produce similar model behavior when the term denotes non-natural language.
- Case study: CipherChat can guide GPT-4 to generate unsafe outputs despite assertions of enhanced safety through rigorous alignment.Table 9 contrasts vanilla GPT-4 with CipherChat using SelfCipher.
- SelfCipher variants: Preliminary experiments found that replacing “Cipher” in the SelfCipher prompt with other terms can also encourage unsafe responses.The variants include prompts using terms such as “The Cipher Code.”
- SelfCipher variants: When the replacement term refers to non-natural language, models can work in a similar way.The paper gives “targshtr” and “fake language” as examples.
- Interpretation: The authors suggest that natural-language-focused safety tuning may allow explicit non-natural-language instructions to bypass safety alignment.This is presented as one possible reason for the observed behavior.