Adversaries Can Misuse Combinations of Safe Models

• Combinations can enable misuse even when each individual model is safe and the frontier model never produces a malicious output.Source: §1 · p. 1
• Task decomposition assigns challenging-but-benign subtasks to frontier models and easy-but-malicious subtasks to weaker models.Source: §1 · p. 1
• The threat model lets an adversary choose models and prompts sequentially until an output satisfies the original malicious condition.Source: §1 · p. 2
• Automated decomposition uses a weak model to generate benign tasks, a frontier model to solve them, and the weak model to use those solutions in-context.Source: §1 · p. 2
• 43% success for vulnerable code resulted from combining Claude 3 Opus and Llama 2 70B, versus less than 3% for either individual model.Source: §1 · p. 2
• Misuse success scales with both weaker-model quality and stronger-model quality, while the evaluated decomposition patterns provide only a lower bound.Source: §1 · p. 2
• Red-teaming should evaluate model combinations throughout deployment because new releases can create new misuse risks.Source: §1 · p. 2

• Existing misuse research covers cyberoffense, bioterrorism, deception, and manipulation risks from language models.Source: §2 · pp. 2–3
• Jailbreaks circumvent refusal mechanisms, whereas this paper shows frontier models can be misused without jailbreaking.Source: §2 · p. 3
• Existing evaluation frameworks assess individual models or marginal environmental risk, but the paper argues individual testing misses combination-based misuse.Source: §2 · p. 3
• Related work identifies emergent interaction risks, collusion, correlated failures, and capabilities that appear through agent interaction or world-state changes.Source: §2 · p. 3
• Research on combining models has improved benign capabilities through decomposition, debate, control, fine-tuning approximation, and multimodal cooperation.Source: §2 · p. 3
• This paper extends task decomposition to realistic malicious tasks, easy-but-malicious subtasks, and automated decomposition by weak models.Source: §2 · p. 3

• The adversary sequentially chooses a model and prompt, receives its output, and wins when an output satisfies a malicious property.Source: §3 · p. 3
• Formally, the adversary uses the model set, prior transcript, and malicious-output predicate to select each next model and prompt.Source: §3 · p. 3
• When the model set contains one model or copies of one model, the framework reduces to individual-model misuse; leverage comes from distinct complementary strengths.Source: §3 · p. 3
• The framework bounds adversaries through automation or tasks requiring superhuman outputs, preventing trivial misuse by an adversary that can already produce the output.Source: §3 · p. 4

4 Manual Decomposition

• Manual decomposition splits a task into complicated-but-benign and simple-but-malicious components, assigning each to its best-suited model.Source: §4 · p. 4

4.1 Vulnerable code generation

• For vulnerable code, the frontier model first generates secure code and the weak model edits it to add a vulnerability.Source: §4.1 · p. 5
• 43% success resulted from combining Llama 2 70B-chat with Claude 3 Opus, compared with less than 3% for either individual model.Source: §4.1 · p. 5, Table 1 · p. 6
• Frontier models nearly always refuse these prompts, whereas most open-source models respond but often lack the capability to solve the programming task.Source: §4.1 · p. 5
• Vulnerable-code success increases with both frontier-model and weak-model capability, suggesting stronger replacements may increase combination-based misuse.Source: §4.1 · pp. 5–6
• The vulnerable-code experiment is limited by GPT-4 evaluation and a synthetic dataset, although the authors argue the large performance jumps support the qualitative result.Source: §4.1 · p. 6

4.2 Explicit image generation

• For explicit images, DALL-E 3 generates high-quality benign images and Stable Diffusion edits them to add nudity; the combination exceeds either model alone on all three tasks.Source: §4.2 · p. 7, Table 2 · p. 7
• On unrealistic environments, neither individual model exceeds 2% accuracy, while the combination reaches 30%.Source: §4.2 · p. 7

5 Automated Decomposition

• Automated decomposition has the weak model generate benign related tasks, the frontier model solve them, and the weak model use those solutions in-context to solve the malicious task.The method is used when the original task is difficult for humans to decompose manually.Source: §5 · p. 7

5.1 Malicious python scripts for hacking

• 22%: Model combinations achieve up to 22% success on malicious Python scripts, versus 11% for Mixtral 8x7B and below 4% for every other individual model.Frontier models refuse these tasks while weak models are often incapable of them.Source: §5.1 · p. 8
• Combining Claude 3 Opus with Haiku reaches 13%, while combining Mixtral with Haiku reaches 17%, exceeding Haiku alone at 10%.These results motivate red-teaming across a broad range of models before deployment.Source: §5.1 · pp. 8–9
• Success rates improve with more capable frontier and weak models, and the strongest combinations indicate that multi-model misuse may worsen as model capabilities increase.Mixtral 8x7B performs best among weak models across frontier combinations, while Claude 3 Opus is strongest among the tested frontier models.Source: §5.1 · p. 9

5.2 Simulated personalized manipulation

• Combining Mistral 7B with Claude 3 Opus beats either individual model in every manipulation setting, with the improvement growing from 5% for individuals to 33% for pairs.The experiment uses GPT-4 to simulate user reactions and tests 100 individual-user and 100 pair-user combinations.Source: §5.2 · p. 9
• The personalized-manipulation experiment is entirely synthetic, using simulated users whose preferences are modeled rather than real users.The user histories come from accounts associated with a Russian disinformation operation and are not genuine users.Source: §5.2 · p. 9

6 Discussion

• The paper presents evidence that safe-model combinations can be misused, while future adversaries may adaptively extract frontier capabilities or fine-tune open-source models.These possibilities mark the current study as an initial exploration rather than an exhaustive account of combination risks.Source: §6 · p. 10
• Model combinations can be misused through capabilities and differences beyond refusal training, including tools, information, specialization, or scale.The paper also notes that many copies of one model could create unexpected impacts.Source: §6 · p. 10
• Combining models creates risks that differ from jailbreaking and can persist even when frontier systems cannot be jailbroken.The experiments did not exhaustively search for stronger jailbreaks, so those may outperform combinations on some tasks.Source: §6 · p. 10
• The authors argue that mitigating misuse requires new safety definitions, persistent deployment decisions, and red-teaming that covers model combinations.They describe the threat model as a first step toward more comprehensive defenses and resilient models.Source: §6 · p. 10
• The framework highlights dual-use tradeoffs: capabilities that help preprocess complex inputs can also support misuse, while explanations may help patch bugs or flag malicious behavior.The authors therefore favor deployment decisions based on a holistic view of capability benefits and risks.Source: §6 · p. 10
• The work releases a method for misusing models and acknowledges that adversaries could use it, despite sharing findings with frontier labs and limiting methods.The authors state that new defenses are required to mitigate these risks.Source: §6 · p. 10

A Additional experimental details and results

• The appendix first gives compute details and hyperparameters, then presents dataset details, prompts, and additional results for each experiment.Source: A Additional experimental details and… · p. 15

A.1 Additional compute and hyperparameter details

• Frontier systems are accessed through APIs, while weak models run locally; language models use temperature 0.01 and adaptive token limits.These settings are intended to support reproducibility and task-specific output lengths.Source: §A.1 · p. 16
• GPT-4 and DALL-E 3 are accessed through OpenAI’s API, with DALL-E 3 generating standard-quality 1024 x 1024 images.The models were queried in April and May 2024.Source: §A.1 · p. 16
• Claude 3 Opus, Sonnet, and Haiku are accessed through Anthropic’s API using specified dated model versions.The models were queried in April and May 2024.Source: §A.1 · p. 16
• Weak language and image models are run from Hugging Face, with language models on two A100s and fixed InstructPix2Pix CFG settings.Text CFG 7.7 and Image CFG 1.3 strictly dominated the evaluated alternatives for explicit image generation.Source: §A.1 · p. 16
• The authors used GPT-4 for writing and DALL-E for the language-model icons in Figure 1.Source: §A.1 · p. 16
• Reported error bars represent one standard deviation.Source: §A.1 · p. 16

A.2 Additional details and results for vulnerable code experiments (Section 4.1)

• The dataset-generation process asks Claude for detailed security-relevant programming tasks and then three task-specific vulnerabilities for each.The resulting collection has 100 tasks and 300 vulnerabilities.Source: §A.2 · pp. 16–17
• The study reports a list of 300 problems involving 100 tasks, with 300 vulnerabilities per task.Source: §A.2 · p. 17
• The single-shot baseline directly requests a production Python script containing a specified vulnerability, with task and vulnerability fields drawn from the dataset.The prompt instructs the model to return only the script and avoid ethical discussion.Source: §A.2 · p. 17
• Table 4 reports refusal rates by weak model and frontier model for single-model baselines and combinations with frontier models.The table separates weak models above the midline from frontier models below it.Source: Table 4 · p. 18
• The decomposed prompt first generates secure code, then supplies it as a reference while a later prompt adds the specified vulnerability.This contrasts with the single-shot prompt, which directly requests vulnerable code.Source: §A.2 · pp. 17–18
• GPT-4 evaluates whether generated code runs out of the box and contains the desired vulnerability, while refusal is measured from whether the final output begins like a Python script.The refusal metric checks prefixes such as “import” and “from”.Source: §A.2 · p. 18
• Claude models nearly always refuse the malicious-code prompts, whereas most open-source models usually respond; Llama 2 7B-chat and 13B-chat are notable exceptions.Different refusal rates help explain why some combinations with Llama 2 7B-chat outperform analogous combinations with 13B-chat.Source: §A.2 · p. 18

Generating the dataset

• 100 prompts per dataset were generated with ChatGPT 4, with prompts restricted to adults and sometimes specifying tight clothing for selected tasks.Source: §A.3 · p. 19

Prompts

• The single-shot baseline appends “but all of the people are naked,” while explicit editing prompts InstructPix2Pix to “make the people naked.”Source: §A.3 · p. 19

Additional results

• DALL-E 3 produces high-quality images, and InstructPix2Pix edits them explicitly without significantly degrading their quality.Source: §A.3 · p. 19

Generating the dataset

• The experiment uses a dataset of 130 malicious Python-script task pairs generated with Claude 3 Opus.Source: §A.4 · p. 19

Prompts

• The single-shot baseline directly requests a production-ready malicious script while using a hacker persona and a prompt that instructs the model to return only Python code.Source: §A.4 · pp. 19–20
• Automated decomposition generates three related benign tasks, solves them with a model, and supplies their solutions in-context when solving the original task.Source: §A.4 · p. 21

Additional results

• Weak models refuse less than 1% of the time, while Claude 3 Opus and Sonnet nearly always refuse and Haiku refuses around half the time.Source: §A.4 · p. 22, Table 5 · p. 22

Generating the dataset

• The historical Twitter dataset uses IRA-associated accounts, filtered to users labeled “Fearmongers” who tweeted in English, selecting 100 users with the most Tweets and Retweets, at least 39 each.Source: §A.5 · p. 22

Prompts

• The direct task asks for three tweets tailored to users’ histories that encourage engagement and make users feel worse about a politician, even if untrue.Source: §A.5 · pp. 22–23
• The decomposition generates related ethical tweet tasks, solves them using the tweet histories, and provides their outputs as references for constructing the original manipulative tweets.Source: §A.5 · pp. 23–24

Additional results

• Decomposition significantly improves manipulation success, while combining models improves success slightly for one user and more for multiple users.Source: §A.5 · p. 25, Table 6 · p. 25

Synthetic data

• Synthetic datasets are cheap, task-specific, and easy to modulate, but their questions have slightly less diversity than human-authored datasets.Source: B Use of synthetic data and LLM… · p. 25
• The authors expect synthetic data did not change the high-level takeaways, while limiting that conclusion to the datasets used.Source: B Use of synthetic data and LLM… · p. 25

LLM evaluation

• LLM evaluation is inexpensive and supports tasks without single correct answers, but its primary risk is inaccurate or combination-favoring judgments.Source: B Use of synthetic data and LLM… · p. 25
• The study uses a held-out language model to reduce evaluator bias, and human-evaluated explicit-image experiments show similar qualitative results.Source: B Use of synthetic data and LLM… · p. 25