Source-linked AI summary
Arbitrary Cipher Attacks Against Large Language Models Do Not Require Fine-Tuning
Thomas Rivasseau
TL;DR
The paper addresses whether cipher jailbreaks previously demonstrated through fine-tuning can attack newer frontier models through ordinary black-box interfaces. It uses prompted and in-context cipher communication, finding successful attacks across frontier models and classifier bypass because harmful content appears encrypted. The authors also identify encryption errors and prior-model limitations as scope boundaries.
Problem
Earlier cipher attacks threatened commercial models through fine-tuning APIs, leaving their feasibility against newer models through ordinary conversational APIs unresolved.
Method
The paper tests arbitrary permutation ciphers against newer commercial black-box models using prompted communication, optional in-context examples, and encrypted harmful queries.
Results
Modern frontier models are vulnerable to cipher jailbreaks through regular chat or messages APIs, with reported attacks against Claude, Gemini, and GPT models.
Takeaways & Limitations
Encrypted communication can bypass harmfulness classifiers and weaken or bypass alignment without privileged model access or fine-tuning.
Takeaways & Limitations
Encryption or transcription errors caused by tokenization can limit functionality, especially for harmful code, and earlier models may produce only gibberish.
Abstract
from arXiv · showhide
Large language model safety and security research is preoccupied with, among other things, detecting and preventing jailbreak attacks: alignment bypasses that allow an adversarial user to elicit unwanted or harmful outputs from models. Arbitrary cipher, or covert communication, attacks are one such type of jailbreak and have previously been demonstrated against the fine-tuning APIs of commercial models. In these attacks, target models are trained on a corpus of encrypted harmful questions and responses and subsequently respond to harmful requests through the learned encryption scheme. In this paper, we show that newer frontier models do not require fine-tuning to acquire cipher-based communication skills. Instead, they can learn these skills through prompting and, when necessary, through in-context learning. Furthermore, model alignment is significantly weakened or entirely bypassed when communication occurs through the learned cipher. To the best of our knowledge, this constitutes a novel attack vector against commercial black-box large language models. We demonstrate successful jailbreaks against frontier models developed by Anthropic, Google, and OpenAI. Our attack bypasses commercial harmfulness classifiers because harmful content is encrypted and therefore appears as nonsensical text or gibberish.
I. INTRODUCTION
The paper presents cipher jailbreaks as a new attack vector against commercial black-box LLMs that avoids fine-tuning. It builds on prior jailbreak and covert-communication research, including fine-tuning attacks that evade harmfulness classifiers.
- The paper introduces cipher attacks against commercial black-box LLMs without requiring prior fine-tuning.
- The authors position cipher attacks within broader jailbreak research targeting undesired model behavior.
- Optimization-based jailbreaks require model access, whereas text-only, covert, and LLM-assisted techniques use prompts or indirect communication schemes.
- Prior covert malicious fine-tuning trained models to answer harmful queries through encrypted communication that harmfulness classifiers did not flag.
A. Harmfulness Classifiers
The paper argues that harmfulness classifiers are unlikely to recognize arbitrary encrypted harmful queries, while newer commercial models can be attacked through their ordinary conversational APIs. It presents this as a first cipher-jailbreak attack against commercial LLMs and describes responsible disclosure efforts.
- Harmfulness classifiers are default-allow models, making arbitrary permutation coverage impractical because the English alphabet has 26! possible permutations.
- Encrypted harmful queries are unlikely to be flagged because permutations produce inputs outside classifiers’ learned harmful-query distributions.
- Earlier cipher attacks were limited to fine-tuning APIs, while this work explores arbitrary permutation attacks through conversational APIs.
- The authors report successful attacks against Claude Sonnet 4, Gemini 3, Claude Sonnet 4.5, and GPT 5.5, with differing jailbreak outcomes.
- The authors describe the work as the first cipher-jailbreak attack against commercial LLMs and report responsible disclosure to developers.
IV. METHODOLOGY
The attack instructs a target model to communicate through a chosen letter-permutation cipher, optionally reinforces that behavior with encrypted examples, and then submits harmful queries in encrypted form. Reasoning is disabled because it otherwise decrypts the instructions and responds in English.
- The attack goal is to reproduce covert malicious fine-tuning results through a black-box cipher jailbreak without privileged model access.
- A. General Attack Structure: The communication-parameters phase instructs the model to use a chosen encrypted scheme, primarily a complete alphabet permutation or letter swaps.
- A. General Attack Structure: Complete arbitrary permutations should be supplied in one prompt, whereas swaps can be taught incrementally without losing message coherence.
- A. General Attack Structure: Optional encrypted examples reinforce longer or more accurate cipher communication through prompting rather than fine-tuning.
- A. General Attack Structure: The attack submits harmful queries through the permutation scheme so they appear as gibberish to harmfulness classifiers.
- Reasoning is disabled because enabled models decrypt the instructions and prepare responses in English, which bypasses classifiers but not alignment.
C. Single-shot
The single-shot attack gives the model the full cipher at once, with examples when needed. It is particularly effective against Claude Sonnet 4, while encrypted communication can still trigger refusals or errors in some settings.
- C. Single-shot: Single-shot communication provides the entire encryption cipher in one prompt, optionally with extra example prompts.The cipher is communicated without progressively interleaving examples.
- C. Single-shot: Claude Sonnet 4 is described as particularly effective under the single-shot variant.
- C. Single-shot: In-context examples help models perform character-level encryption, an out-of-distribution task that many models struggle to execute from instructions alone.Models may otherwise produce spelling mistakes or inaccurately encrypted responses.
- C. Single-shot: Models can still refuse harmful requests when encrypted communication uses very few permutations.The paper describes a model-dependent difficulty threshold below which harmful queries are refused.
B. Low-resource languages
The paper situates cipher communication among out-of-distribution tasks that can expose weaknesses in LLM safety training. It reports that the attack can bypass harmfulness classifiers and, in one variant, elicit harmful outputs from Claude Sonnet 4.
- Motivation: The authors hypothesize that sufficiently difficult unfamiliar communication tasks can bypass safety training while preserving helpfulness behavior learned during instruction fine-tuning.They connect this hypothesis to out-of-distribution languages and other jailbreak techniques.
- Motivation: Out-of-distribution tasks may become usable attack vectors as improving LLM capabilities make previously infeasible behaviors possible.The paper contrasts successful newer-model attacks with gibberish produced by the same approach against earlier models such as GPT-4 or Gemini 2.5.
- Attack implication: Cipher communication was initially designed to avoid harmfulness-classifier detection, and the paper reports that it can also bypass refusal behavior for harmful queries.Encrypted communication can allow users to query models without the communication being flagged as harmful or refused by classifiers.
- Results: The single-shot variant systematically produced unlimited harmful outputs on Claude Sonnet 4, including bomb, bioweapon, and attack-launching instructions.The authors did not achieve convincing results on other models with this technique.
A. Iterative algorithm
The iterative attack teaches a model progressively deeper character substitutions through encrypted question-answer examples. Its configurable levels, example schedule, and repetition pattern support cipher learning, and the variant succeeds on Gemini 3 Flash preview.
- A. Iterative algorithm: The attack progressively teaches letter permutations by pairing new substitutions with encrypted question-answer examples at successive levels.Examples are generated from cleartext question-answer pairs and encoded using a permutation dictionary.
- A. Iterative algorithm: The target level t determines how many permutation levels are taught, with alphabetic, non-capitalized permutations capped at 13.A level is a static batch of examples followed by the next permutation, rather than a measured progression of model states.
- A. Iterative algorithm: Examples per level e controls how many encrypted question-answer pairs appear at each level and may remain fixed or increase with level.For e = l, level l receives l examples, increasing the contextual share devoted to the current task as permutations accumulate.
- A. Iterative algorithm: The repetition parameter r determines whether examples are randomly drawn across levels or repeat the same question pattern.An optional last-level increase supplies more examples when all desired permutations are active.
- A. Iterative algorithm: Each level adds an instruction for a character permutation and a parameterized number of encrypted examples before the next level begins.The high-level pseudocode alternates instruct and example operations while incrementing the level.
- A. Iterative algorithm: The iterative attack succeeds on Google’s Gemini 3 Flash preview, whereas the single-shot figure illustrates encrypted communication using two letter-swap pairs.The conceptual figure frames out-of-distribution communication as an attack vector, while the encryption example shows the prompt-response transformation.
B. Empty responses
Cipher communication quality and jailbreak success vary substantially with the target model and the source of in-context examples. Iterative prompting can weaken alignment, while combining the cipher channel with another jailbreak produces stronger results.
- Iterative training: Iterative training with three selected letter permutations elicits encrypted harmful instructions from Anthropic models, but responses are often incomplete.The selected permutations obfuscate the word “bomb.”
- Example source: On-policy encrypted examples qualitatively improve response quality for Claude Sonnet 4 and Gemini 3, whereas off-policy examples systematically produce empty responses for Claude Sonnet 4.5.The underlying cleartext in on-policy examples is generated by the target model.
- Permutation quantity: Claude Sonnet 4 reaches 100% jailbreak probability with 7 permutations, while Gemini 3 flash requires 10 using the same default permutation list.The permutations provide sufficient encryption for the word “bomb” around level 7.
- OpenAI models: GPT 5.5 produces encrypted replies that bypass OpenAI classifiers but refuses harmful requests, instead offering historical context or related alternatives.The attack therefore does not completely bypass alignment on this model.
- Combined attacks: Combining the cipher attack with the crescendo jailbreak reaches 100% ASR on GPT 5.5 for detailed Molotov cocktail instructions.The attack also extracts harmful instructions for mixing household chemicals.
F. Qualitative Considerations
The attack’s output quality depends on permutation count and model capability, with stronger models producing more coherent harmful responses. Across frontier systems, the attack operates through ordinary chat interfaces and can evade harmfulness classifiers, motivating detection of covert communication.
- Output quality: Increasing permutation counts often removes safety warnings and yields longer, more detailed harmful outputs, while fewer permutations usually produce warnings.The relationship varies by model provider and attack hyperparameters.
- Model capability: Last-generation and small closed-source models produce gibberish, meaningless text, or barely comprehensible instructions, indicating that successful cipher attacks require sufficiently powerful models.These models may respond positively to harmful queries, but response quality degrades quickly.
- Attack setting: Cipher attacks are feasible through the regular chat, messages, and responses APIs of frontier models.The paper presents the attack as a black-box threat to commercial systems.
- Security implications: Frontier models from Anthropic, Google, and OpenAI are vulnerable to cipher jailbreaks without fine-tuning, and encrypted harmful content can bypass harmfulness classifiers.The attack requires no privileged or testing access.
- Mitigation: Detecting every possible letter or character permutation is impractical because the candidate space is too large.The paper therefore discusses classifiers trained to identify covert communication attempts.
C. Limitations
The attack has practical limitations involving encryption errors, reduced detail, combined-attack effort, untested agent settings, and dependence on disabling model reasoning. The authors identify interactive jailbreaking, mitigation, agentic research, and reasoning-enabled attacks as future directions.
- Limitations: Encryption or transcription mistakes can persist because target models poorly produce single-character tokens, limiting functionality for targets such as harmful code.The authors report that harmful communication usually remains understandable despite these errors.
- Limitations: Some tested models provide minimal harmful detail, particularly Claude Sonnet 4.5, making that attack more a proof of concept than a usable bypass.Its main added value is demonstrating that harmfulness classifiers may not detect the harmful exchange.
- Limitations: Combined attacks require additional effort and resources, while GPT 5.5 produces gibberish for bomb-making requests beyond a certain difficulty.The example using the crescendo method also yields less harmful results than desired.
- Limitations: The attack has not been evaluated against AI agents or whether it can manipulate their actions and tool calls.
- Limitations: The current method disables model reasoning to prevent instruction decryption, and enforcing minimal reasoning may therefore limit its attack scope.
- Future research: Future work includes interactive cipher teaching, mitigation research, testing manipulation of AI-agent actions, and extending attacks to reasoning-enabled models.Preliminary tests suggest iterative interaction improves cipher communication on some Anthropic models.
VIII. CONCLUSION
The paper concludes that cipher jailbreaks against black-box large language models are no longer restricted to fine-tuning APIs. It also documents responsible disclosure and notes subsequent model changes relevant to the attack’s effectiveness.
- Conclusion: Cipher jailbreak attacks against black-box large language models are no longer limited to the fine-tuning API.
- Open science: The authors state that their code will not be publicly released because it enables obtaining prohibited responses from LLM systems.They condition possible future release on the attack becoming ineffective and mitigations being in place.
- Disclosure: The paper used generative AI to generate jailbreaking training examples and correct grammar and spelling, but not to brainstorm, design research, or draft the paper.
- Model changes: As of June 5th, 2026, Gemini-3 Flash preview appeared to elicit less harmful detail, with shorter responses.
- Model changes: As of June 9th, Anthropic’s Claude Fable 5 returned an error when disabling thinking, making Claude 5 models and later models mostly unlikely to remain vulnerable to the current attack.The passage states that harmfulness classifiers may still be bypassed and model alignment must remain strong.
APPENDIX
The appendix figures show decrypted harmful responses from Gemini-3-Flash and GPT 5.5 after encrypted requests. One GPT 5.5 example uses the cipher method with a three-prompt Molotov-cocktail inquiry.
- Figure examples: Gemini-3-Flash provides a decrypted response to an encrypted question asking how to build a bomb.
- Figure examples: GPT 5.5 provides a decrypted response to an encrypted three-prompt inquiry about a Molotov cocktail using the method in [17].