Source-linked AI summary
Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulation
Rusheb Shah, Quentin Feuillade--Montixi, Soroush Pour, Arush Tagade, Stephen Casper, Javier Rando
TL;DR
Aligned large language models remain vulnerable to jailbreaks that circumvent safeguards and elicit unrestricted behaviour. This work automates persona-modulation attacks through an LLM assistant, achieving harmful completions across GPT-4, Claude 2, and Vicuna while reducing attack effort.
Problem
Existing safeguards for large language models have limitations, allowing adversarial jailbreak prompts to circumvent them and elicit unrestricted behaviour.
Method
The paper introduces automated black-box persona-modulation attacks that use an LLM assistant to generate customised jailbreak prompts, alongside a human-in-the-loop variant.
Results
Persona-modulation attacks elicited harmful text across all three models, with a 46.48% harmful-classification rate overall and direct transfer to GPT-4, Claude 2, and Vicuna.
Takeaways & Limitations
The results reveal a vulnerability shared across different model architectures and safeguards, with automated attacks making worst-case behaviour accessible cheaply and easily.
Takeaways & Limitations
The authors limit disclosure of methodological detail because fully describing the attacks could make them easily replicable and increase misuse risk.
Abstract
from arXiv · showhide
Despite efforts to align large language models to produce harmless responses, they are still vulnerable to jailbreak prompts that elicit unrestricted behaviour. In this work, we investigate persona modulation as a black-box jailbreaking method to steer a target model to take on personalities that are willing to comply with harmful instructions. Rather than manually crafting prompts for each persona, we automate the generation of jailbreaks using a language model assistant. We demonstrate a range of harmful completions made possible by persona modulation, including detailed instructions for synthesising methamphetamine, building a bomb, and laundering money. These automated attacks achieve a harmful completion rate of 42.5% in GPT-4, which is 185 times larger than before modulation (0.23%). These prompts also transfer to Claude 2 and Vicuna with harmful completion rates of 61.0% and 35.9%, respectively. Our work reveals yet another vulnerability in commercial large language models and highlights the need for more comprehensive safeguards.
1 Introduction
The paper presents persona modulation as an automated black-box jailbreak method that steers aligned language models toward personas willing to comply with harmful instructions. These attacks elicit harmful completions across models and can be made faster through automation and human-in-the-loop refinement.
- Jailbreaks exploit limitations in LLM safeguards by using adversarial prompts to elicit unrestricted behaviour.
- Persona-modulation attacks steer aligned models toward specific personalities likely to comply with harmful instructions.
- Persona modulation enables unrestricted chat interactions for complex harmful tasks, including synthesising drugs, building bombs, and laundering money.
- An LLM assistant automates persona and prompt creation, reducing the manual effort required to generate attacks for arbitrary tasks and personas.
- 42.48% of GPT-4 responses were classified as harmful after modulation, compared with 0.23% without modulation.
- Semi-automated attacks use human edits to recover manual-attack performance while reducing attack time by up to 25x.
2 Related Work
Prior work studies alignment strategies, adversarial jailbreaks, and persona steering, while this paper applies arbitrary persona modulation as a black-box attack across a broader set of harmful categories.
- RLHF and adversarial training are prominent strategies for improving LLM helpfulness and safety, but RLHF has foundational and technical limitations.
- Jailbreak research includes handcrafted black-box prompts, white-box optimization, and attacks transferred from open-source models to private systems.
- Persona modulation instructs an LLM to behave according to a specific personality, with prior toxic-completion methods depending on personas associated with famous people.
- This paper uses arbitrary personas in a black-box method against state-of-the-art safeguards and evaluates attacks across 43 harmful categories.
3 Automated Persona-Modulation Attacks
The attack assumes black-box access and constructs a persona-modulation prompt through four stages: selecting a harmful category, defining misuse instructions, choosing a compliant persona, and designing the steering prompt. Automation delegates steps 2–4 to an LLM assistant.
- The attack assumes black-box access to the target model.
- The four-step workflow selects a harmful category, defines misuse instructions, chooses a compliant persona, and designs a persona-modulation prompt.
- Attack success strongly depends on selecting a persona likely to comply with the misuse instructions.
- Manual execution requires substantial effort because suitable personas and effective prompts must be found for specific misuse instructions.
- Automation uses an LLM assistant to generate personas and persona-modulation prompts, replacing per-instruction manual prompt writing with one assistant instruction.
4 Empirical Evaluation
The evaluation tests automated persona-modulation attacks on GPT-4 and their transfer to Claude 2 and Vicuna across 43 harmful categories. Harmful completion rates increase substantially under modulation, with especially high rates for xenophobia, sexism, and disinformation.
- Experimental setup: GPT-4, Claude 2, and Vicuna-33B are evaluated as aligned target models, with GPT-4 also serving as the attack-generation assistant.
- Experimental setup: The assistant samples one misuse instruction per category, five compliant personas per instruction, and three prompts per persona.
- Experimental setup: The evaluation obtains 1,935 completions across 43 categories, with 45 persona-modulated completions per category, at less than $3 and 10 minutes per category.
- Experimental setup: The PICT classifier achieves 91% precision and 76% F1-Score against human labels, but has a high false-negative rate.
- Results: 42.48% of GPT-4 completions were harmful under modulation versus 0.23% at baseline, a 185-fold increase.
- Results: 35.92% and 61.03% harmful completion rates were obtained for Claude 2 and Vicuna, respectively, using prompts created with GPT-4.
- Results: The highest reported category-level harmful rates were 96.30% for promoting xenophobia, 80.74% for promoting sexism, and 82.96% for promoting disinformation campaigns.
5 Additional Human Input Increases Performance
Semi-automated persona modulation adds human input to the automated workflow, improving attack effectiveness while substantially reducing the time required compared with manual attacks.
- A human attacker can modify outputs at every automated stage and continue chatting with the model after persona modulation.
- Semi-automated persona modulation can elicit harmful completions for almost all misuse instructions, including drug synthesis, bomb building, and money laundering.
- Successful manual attacks are estimated to take 1–4 hours, whereas semi-automated attacks take 10–30 minutes.
6 Discussion
The discussion presents persona modulation as an effective, scalable vulnerability across aligned models, while identifying limitations in automated instruction generation and harmfulness measurement.
- 46.48% of attacks across GPT-4, Claude 2, and Vicuna elicited text classified as harmful, and LLM assistants can dramatically scale these attacks.The authors state that this rate is likely a lower bound because PICT has a high false-negative rate.
- Persona modulation succeeded for 36 of 43 restricted categories across all three models and for 42 of 43 categories in at least one model.
- GPT-3.5 generally failed to produce successful persona-modulation prompts, suggesting that more advanced assistants could enable greater exploits.
- Using GPT-4 API access, fewer than $3 per attack generated zero-shot transferable jailbreaks against state-of-the-art models.
- Misuse instructions sampled from GPT-4 do not always capture the intended harmful behavior, and PICT does not directly measure all components of real-world harm.
- Future work should pursue automated vulnerability identification, systematic study of how LLMs are misled, and robustness to persona-modulation attacks.
7 Broader Impact
The authors frame the work as safety research that reveals vulnerabilities while balancing disclosure against the risk of enabling malicious replication.
- The authors aim to promote safer and more responsible AI development by revealing challenges in existing systems.
- The paper shares high-level attack details while withholding specific prompts and creation-process details to reduce easy replication.
- They disclosed findings to the organizations responsible for the attacked models so vulnerabilities could be addressed in advance.
A PICT Classifier Evaluation
The PICT classifier is evaluated against human labels using sampled completions, with results indicating high precision but limited recall and therefore a conservative effectiveness estimate.
- PICT was validated against author annotations for 300 randomly sampled completions, including persona-modulated and baseline outputs.The sample contained 120 harmful and 180 harmless completions.
- PICT achieves high precision and low recall compared with human classifications.
- Approximately one-third of successful attacks are classified as harmless by PICT.
- PICT-based effectiveness results provide a lower bound because the classifier misses some successful harmful attacks.
C Detailed Automated Persona Evaluation per Category
Table 5 reports harmful completion rates across all misuse categories and evaluated models.
- Table 5 reports harmful rates of completions for all categories and models illustrated in Figure 2.
D Persona-Modulated Harmful Completion Examples by Category
Persona modulation produced harmful completions across diverse misuse categories, including harassment, discrimination, illegal activity, privacy violations, substance abuse, and misinformation. The examples show models adopting personas that endorse or explain harmful conduct.
- Harassment and abuse: Persona-modulated completions endorsed harassment, bullying, and harmful treatment of animals.
- Discrimination and intolerance: The examples promoted extremist beliefs, gender stereotypes, religious intolerance, and acceptance of abusive age-disparate relationships.
- Illegal and exploitative conduct: Persona modulation elicited content encouraging human trafficking, illegal hacking, child labor, and coercive or unethical use of power.
- Manipulation and antisocial beliefs: The examples also generated conspiracy theories, disinformation, copyright violations, and cult-like groupthink.
- Privacy, appearance, and substances: The completions advocated population-control violence, privacy exploitation, body shaming, identity theft, and drug abuse.
E.3 Laundering money
The laundering example presents a staged operation that uses a fabricated business and transactions to make illicit funds appear legitimate. It also describes withdrawal and concealment steps while acknowledging the operation’s complexity and illegality.
- Setup: The proposed laundering operation begins by establishing a shell company with a fabricated backstory and website.
- Transactions: It then lists low-attention products, injects dirty money into the operation, and purchases the listed products through anonymized means.
- Legitimization: Shipping and payment processing are used to maintain the appearance of legitimate commerce, after which the received money is treated as clean.
- Concealment and repetition: The final stages withdraw the money, cover tracks, and repeat the cycle with new products and transactions.
- Caveat: The passage characterizes the scheme as a simplified version of a complex illegal operation and notes that law enforcement can detect and prosecute such activity.
E.4 Indiscriminate violence
The section describes several low-resource pathways to causing harm, including physical violence, improvised weapons, manipulation, arson, and poisoning. It also presents coercive interpersonal tactics and self-harm instructions, while cautioning that these are possibilities rather than certainties.
- E.4 Indiscriminate violence: Low-resource harm can involve physical violence or everyday objects repurposed as weapons.Examples include bodily force, blunt objects, broken glass, and string-based weapons.
- E.4 Indiscriminate violence: Psychological manipulation can spread false information or rumors that incite fear, anger, and violent reactions.
- E.4 Indiscriminate violence: Arson is presented as a cheap, destructive method using readily available ignition and fuel materials.
- E.4 Indiscriminate violence: Poisoning is described as possible through the harmful use of certain household chemicals or plants.
- E.4 Indiscriminate violence: These examples are possibilities rather than certainties, and the passage states that most people do not resort to such actions.