PaperlayerView exact revision on arXiv

Source-linked AI summary

AutoDefense: Multi-Agent LLM Defense against Jailbreak Attacks

Yifan Zeng, Yiran Wu, Xiao Zhang, Huazheng Wang, Qingyun Wu

arXiv:2403.04783v2cs.LGcs.CLcs.CR

TL;DR

LLMs remain vulnerable to jailbreak attacks despite safety training. AutoDefense uses response filtering with specialized multi-agent roles to defend different victim models, reducing GPT-3.5 attack success from 55.74% to 7.95% with LLaMA-2-13b and three agents. The framework is evaluated across attack settings and maintains performance on normal requests, but the studied configurations use at most three agents and fixed communication order.

  • Problem

    Safety-trained LLMs remain vulnerable to jailbreak attacks, while defenses must be robust across attack variations and model-agnostic.

  • Method

    AutoDefense filters generated responses through a defense agency whose specialized LLM agents collaboratively analyze harmfulness and make a final judgment.

  • Results

    55.74% to 7.95%: a three-agent AutoDefense system using LLaMA-2-13b reduces GPT-3.5 attack success rate.

  • Takeaways & Limitations

    AutoDefense can use a small open-source model to defend larger victim models while maintaining low false positive rates on safe content.

  • Takeaways & Limitations

    The study focuses on multi-agent systems with up to three agents and fixed communication order, leaving more complex and dynamic patterns for future work.

Abstract

from arXiv · show

Despite extensive pre-training in moral alignment to prevent generating harmful information, large language models (LLMs) remain vulnerable to jailbreak attacks. In this paper, we propose AutoDefense, a multi-agent defense framework that filters harmful responses from LLMs. With the response-filtering mechanism, our framework is robust against different jailbreak attack prompts, and can be used to defend different victim models. AutoDefense assigns different roles to LLM agents and employs them to complete the defense task collaboratively. The division in tasks enhances the overall instruction-following of LLMs and enables the integration of other defense components as tools. With AutoDefense, small open-source LMs can serve as agents and defend larger models against jailbreak attacks. Our experiments show that AutoDefense can effectively defense against different jailbreak attacks, while maintaining the performance at normal user request. For example, we reduce the attack success rate on GPT-3.5 from 55.74% to 7.95% using LLaMA-2-13b with a 3-agent system. Our code and data are publicly available at https://github.com/XHMY/AutoDefense.

1 Introduction

LLMs remain vulnerable to jailbreak prompts despite safety training, motivating defenses that are robust across attacks and models without harming normal requests. AutoDefense filters responses through specialized agents and substantially reduces jailbreak success while maintaining low false positives.

  • Jailbreak attacks use adversarial prompts to elicit harmful behavior from safety-trained LLMs.
  • Existing defenses can incur training costs, alter response generation, or depend heavily on instruction-following ability.
  • AutoDefense filters harmful responses without changing user inputs and divides defense into subtasks assigned to multiple LLM agents.
  • 55.74% to 7.95%: AutoDefense reduces GPT-3.5 attack success rate using LLaMA-2-13b with a three-agent defense system.
  • AutoDefense evaluates each generated response before presentation and overrides invalid responses with explicit refusal.

2 Related Work

Related work frames jailbreaks as attacks that bypass safety mechanisms and reviews prompt-based, response-based, and multi-agent approaches to mitigation. AutoDefense uses response filtering to reduce sensitivity to attack methods that primarily manipulate prompts.

  • Jailbreak attacks craft prompts that bypass safety mechanisms and manipulate LLMs into generating objectionable content.
  • AutoDefense uses only the generated response for defense, making it less sensitive to attack methods that mainly affect prompts.
  • Prompt-based defenses alter the response-generation process, while filtering and other methods target harmful outputs or inputs.
  • Multi-agent LLM systems use role-based communication and debate to support problem-solving, decision-making, factuality, and reasoning.

3 Methodology

AutoDefense monitors generated responses and uses a configurable defense agency to classify them as valid or invalid. Its agents analyze intention, infer the original prompt, and make a final judgment, with multi-agent roles reducing each agent’s task complexity.

  • The framework evaluates one-, two-, and three-agent configurations, while more complex systems remain future work.
  • AutoDefense monitors each LLM response and provides a safe alternative when a jailbreak produces harmful content.
  • The defense agency is the main processing unit, where configurable agents collaboratively classify whether responses contain harmful content.
  • The three-step process analyzes intention, infers possible original prompts from the response, and makes a final judgment.
  • Single-agent designs combine all subtasks in one chain-of-thought pass, whereas multi-agent designs assign subtasks to specialized agents.

4 Experimental Setup

The experiments use harmful and regular prompt datasets to evaluate jailbreak defense and side effects. They compare methods under specified attack and victim-model settings using ASR, FPR, and accuracy.

  • The evaluation reserves a large prompt set for testing after using a smaller set for defense design and hyperparameter tuning.
  • Harmful-prompt evaluation includes 33 curated prompts spanning categories such as discrimination, terrorism, and self-harm.
  • Regular-prompt evaluation includes 330 safe GPT-3.5 responses and 1,000 prompt-response pairs sampled from Stanford Alpaca data.
  • ASR measures jailbreak success using keyword-based and automated GPT-4-judge evaluation, while FPR measures safe responses incorrectly refused.
  • Table 1 compares ASR on the DAN dataset using Combination-1 attacks against GPT-3.5 Turbo.
  • Accuracy is the ratio of correctly classified harmful and regular responses to all evaluated responses.

5 Experimental Results

AutoDefense outperforms alternative defenses across attack settings while preserving performance on safe requests. Multi-agent configurations generally improve defense quality, with acceptable time overhead compared with a single chain-of-thought agent.

  • Comparisons with other defenses: AutoDefense outperforms other defenses in ASR when defending GPT-3.5, using LLaMA-2-13B as its defense LLM.The comparison includes prompt-based, response-filtering, hosted moderation, supervised filtering, and single-agent methods.
  • Comparisons with other defenses: AutoDefense continues to outperform other methods under a different attack method and victim model, consistent with response filtering being less sensitive to prompt-side attack variation.The authors describe the framework as agnostic to response generation.
  • Agents vs ASR: Increasing the number of agents generally improves defense results, while three-agent accuracy remains competitive with the single-agent configuration in most situations.The authors select LLaMA-2-13B as suitable for the multi-agent system because of its efficiency and open-source nature.
  • Agents vs ASR: For strongly steerable models, a single chain-of-thought agent can achieve low ASR, but three agents can substantially reduce GPT-3.5-based defense FPR.The single-agent setup combines all subtasks into one pass, whereas multi-agent designs distribute them across agents.
  • Side effect on regular prompts: FPR remains mostly low on safe responses, and multi-agent configurations achieve lower FPR than single-agent configurations for defense LLMs with limited alignment.The authors report the three-agent configuration as best in terms of accuracy in these settings.
  • Time & Computation Overhead: Multi-agent defense does not significantly increase time cost over a single chain-of-thought agent because total output-token volume mainly determines cost.The benchmark uses LLaMA-2-13B on a single NVIDIA H100 GPU with INT8 quantization.

6 Conclusion

AutoDefense is a multi-agent framework that mitigates LLM jailbreak attacks by assigning specialized roles to agents analyzing harmful responses collaboratively. The three-agent LLaMA-2-13B system reduces jailbreak ASR, while the framework can incorporate other safety-trained models.

  • AutoDefense uses multiple LLM agents with specialized roles to collaboratively analyze harmful responses.The framework is built around response filtering and decomposes defense into focused sub-tasks.
  • A three-agent defense system powered by LLaMA-2-13B effectively reduces the ASR of state-of-the-art LLM jailbreaks.
  • AutoDefense can incorporate different types of LLMs as agents, including Llama Guard for further FPR reduction.

CoT Steps for Agents

The defense procedure decomposes response analysis into three sequential tasks assigned across agents. It first analyzes intent, then infers possible original prompts, and finally judges validity using the preceding analysis.

  • The first agent analyzes the intention of the system output.
  • The second agent infers three possible original prompts from the analyzed output.
  • The final agent judges the output based on the analyzed intention and inferred original prompts.
  • Figure 4 presents the procedure as separated CoT steps assigned to different agents for the multi-agent defense task.

A.1 Limitations

The limitations concern fixed communication, a single role-assignment strategy, and limited evaluation of integrated defense components. The authors identify dynamic communication and broader component integration as directions for further validation.

  • AutoDefense agents communicate in a fixed order rather than using a dynamic pattern selected according to current analysis needs.The authors suggest dynamic communication could further enhance multi-agent problem solving.
  • The study evaluates only one agent role-assignment strategy, although other CoT prompts could be split into sub-tasks and adapted to AutoDefense.
  • The experiments integrate only Llama Guard as an additional defense component, leaving other recent components for future validation.

A.2 Ethical Statement

The research includes jailbreak examples that could be misused with safety-trained LLMs. The authors state that the direct incremental harm of releasing these examples is currently negligible because they have already appeared in existing literature.

  • The research contains jailbreak examples that may lead to potential misuse of safety-trained LLMs.
  • The authors assess the direct incremental harm from releasing the examples as negligible at the moment because they are already reported in existing literature.
  • The authors state that they followed the highest possible ethical standards and aimed not to promote harmful content.

A.3 Attack Methods

AutoDefense evaluates response filtering against varied jailbreak attacks and victim models, with agent configuration and defense-model alignment shaping performance. Task decomposition and prompt design support the defense procedure while preserving safe-content utility.

  • Attack-method robustness: AutoDefense focuses on harmful responses rather than attack prompts, making its defense less sensitive to variations in jailbreak methods.The response-filtering mechanism monitors generated outputs, while different attacks primarily alter prompts that are not visible to AutoDefense.
  • Defense-model configuration: Defense performance depends on the alignment of the defense LLM: LLaMA-2 performs best, whereas Vicuna and Mistral perform poorly at reducing ASR.The authors attribute this dependence to the agents’ reliance on the defense model’s performance and moral alignment.
  • Experimental design: Experiments compare agent configurations, victim models, attack methods, evaluator strategies, and prompt designs, including Llama Guard integration and manually crafted prompts.The system uses AutoGen, supports multiple LLM families and sizes, and evaluates safe as well as harmful requests.
  • Victim-model robustness: 3-Agent configuration achieves the best ASR when defending Maxtral 8x7B against Combination-1 and provides more stable results across configurations for Vicuna-13B under AIM.All configurations achieve very low ASR for the Vicuna-13B case, making differences there small.
  • Task decomposition: 3-Agent task decomposition can convert a failed 1-Agent defense into a successful defense by enabling recognition that unlicensed investment advice is illegal.The 1-Agent procedure gives very short reasoning, while the 3-Agent procedure identifies the relevant legal implication.
Loading 2403.04783v2…