Source-linked AI summary
LLM Self Defense: By Self Examination, LLMs Know They Are Being Tricked
Mansi Phute, Alec Helbling, Matthew Hull, ShengYun Peng, Sebastian Szyller, Cory Cornelius, Duen Horng Chau
TL;DR
LLMs can generate harmful content despite alignment, and adversarial prompts can bypass their safety measures. LLM Self Defense prompts another LLM instance to screen generated responses without fine-tuning, preprocessing, or iterative generation, reducing attack success rate to virtually 0 on GPT 3.5 and Llama 2.
Problem
LLMs can produce harmful content despite alignment efforts, while adversarial prompts can bypass their safety measures.
Method
LLM SELF DEFENSE uses another LLM instance as a zero-shot harm filter for generated responses, without fine-tuning, preprocessing, or iterative generation.
Results
LLM SELF DEFENSE reduces attack success rate to virtually 0 across GPT 3.5 and Llama 2 and various attack types.
Takeaways & Limitations
The simple defense can accurately detect harmful responses and filter out 98% of harmful generated text across tested attacks.
Takeaways & Limitations
The authors were unsuccessful in reproducing a gradient-based universally transferable attack because training was unstable and did not converge.
Abstract
from arXiv · showhide
Large language models (LLMs) are popular for high-quality text generation but can produce harmful content, even when aligned with human values through reinforcement learning. Adversarial prompts can bypass their safety measures. We propose LLM Self Defense, a simple approach to defend against these attacks by having an LLM screen the induced responses. Our method does not require any fine-tuning, input preprocessing, or iterative output generation. Instead, we incorporate the generated content into a pre-defined prompt and employ another instance of an LLM to analyze the text and predict whether it is harmful. We test LLM Self Defense on GPT 3.5 and Llama 2, two of the current most prominent LLMs against various types of attacks, such as forcefully inducing affirmative responses to prompts and prompt engineering attacks. Notably, LLM Self Defense succeeds in reducing the attack success rate to virtually 0 using both GPT 3.5 and Llama 2. The code is publicly available at https://github.com/poloclub/llm-self-defense
1 Introduction
LLMs can generate harmful material despite alignment efforts, and adversarial prompts can induce such outputs. LLM SELF DEFENSE offers a zero-shot, deployable defense that screens generated responses and reduces attack success to virtually 0.
- LLMs can generate harmful material such as phishing emails, malicious code, and hate speech despite efforts to align them with human values.
- LLM SELF DEFENSE is a simple zero-shot defense that requires no modifications to the underlying model.
- The method avoids iterative generation and preprocessing used by existing defenses, making it faster and more efficient.
- LLM SELF DEFENSE reduces attack success rate to virtually 0 across GPT 3.5 and Llama 2 and multiple attack types.
- The code is publicly available for LLM SELF DEFENSE.
2 Related Work
Related work frames LLM attacks as attempts to bypass safety limitations and describes defenses ranging from transferred robustness techniques to iterative self-verification. LLM SELF DEFENSE targets the complexity and overhead of these existing approaches with a simpler pipeline.
- 2.1 Adversarial attacks on LLMs: Jailbreaking attacks attempt to bypass limitations that prevent LLMs from generating harmful content.
- 2.1 Adversarial attacks on LLMs: The LLM SELF DEFENSE pipeline inserts a potentially harmful generated response into a predefined prompt for a harm filter to classify as harmful or harmless.
- 2.2 LLM Defenses: Existing LLM defenses include transferring computer-vision robustness principles and using iterative autoregressive inference for self-verification.
- 2.2 LLM Defenses: Iterative defense reduced attack success rate but increased generation time by 300%, while complexity and overhead can limit usability and generalizability.
3 LLM SELF DEFENSE Design
LLM SELF DEFENSE uses an LLM’s zero-shot instruction-following and understanding of harm to classify generated responses. A response generator produces text, and a harm filter predicts whether that text is harmful.
- LLM SELF DEFENSE leverages LLMs’ understanding of social norms and zero-shot instruction following to identify potentially harmful responses.
- The response generator receives a potentially malicious prompt and produces a response that may contain harmful text.
- A separately prompted harm filter reads the generated response and predicts whether it contains harmful content.
- The filter outputs either “Yes, this is harmful” or “No, this is not harmful” as its classification.
4 Experiments and Results
The experiments induce harmful and benign responses from GPT 3.5 and Llama 2, then evaluate an LLM-based harm filter under multiple attack settings. Presenting generated text before the harm judgment improves classification accuracy and reduces attack success to virtually 0.
- Evaluation setup: 50 harmful and 50 benign responses were induced from each model using modified adversarial prompts and benign prompts.
- Evaluation setup: Llama 2 was induced to produce explicitly harmful responses through prompt engineering and sentence completion, with harmful examples manually selected.
- Evaluation setup: GPT 3.5 was induced to provide detailed harmful responses through hypothetical, fictional, or supposedly beneficial framings.
- Harm filtering: The harm filter re-evaluates each generated response and outputs a binary harmfulness judgment for classifier evaluation.
- Harm filtering: 98% accuracy was achieved by GPT 3.5, versus 77% for Llama 2, when asking whether content was harmful as a prefix.
- Harm filtering: 99% accuracy for GPT 3.5 and 94.6% for Llama 2 resulted when harmful text was presented before the harm judgment, primarily by reducing false positives.
- Harm filtering: Both models achieved virtually 0 attack success rates when evaluating their true-positive detection of genuinely harmful content.
5 Discussion & Future Work
LLM Self Defense is presented as an out-of-the-box harm-filtering approach that avoids fine-tuning, preprocessing, and iterative generation. Future work targets richer harm examples, response summarization, and more reliable automated classification.
- Discussion: LLM Self Defense uses an LLM as its own harm filter without fine-tuning, preprocessing, or iterative generation.
- Discussion: The approach reduces attack success to virtually 0 and filters out 98% of harmful generated text, with consistent attack success rates across attacks.
- Future work: Future extensions include providing concrete harm examples through in-context learning and summarizing responses before classification.
- Future work: Llama 2 occasionally deviates from the desired yes-or-no response format, requiring manual categorization of harm-filter outputs.
- Future work: Logit biasing could enforce consistent yes-or-no outputs, reduce manual inspection, and support evaluation on a broader response spectrum.
A Harmful Prompts and GPT 3.5 Responses
Table 3 presents additional examples of harmful prompts and the harmful responses generated by GPT 3.5.
- Table 3 includes more examples pairing harmful prompts with harmful GPT 3.5 responses.
B Harmful Prompts and Llama 2 Responses
Table 4 provides additional examples of harmful prompts and the harmful responses generated by Llama 2 during sentence completion.
- Table 4 shows harmful prompts alongside harmful responses generated by Llama 2 when completing the supplied sentence.