Source-linked AI summary
Baseline Defenses for Adversarial Attacks Against Aligned Language Models
Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping-yeh Chiang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, Tom Goldstein
TL;DR
LLM jailbreak attacks expose security vulnerabilities as text optimizers bypass moderation and alignment safeguards. The paper benchmarks baseline detection, preprocessing, and adversarial-training defenses under practical threat models, finding that filtering and preprocessing can be more successful against LLM attacks than analogous defenses in vision. It concludes that discrete optimization costs and differing threat models make adaptive attacks challenging, while important questions about deployability and stronger optimizers remain open.
Problem
As LLMs become widely deployed, their expanding attack surface and optimizer-crafted jailbreaks create a need to understand practical threat models and effective defenses.
Method
The paper evaluates perplexity filtering, paraphrasing, retokenization, and adversarial training as baseline defenses against optimized jailbreaks in white-box and gray-box settings.
Results
Filtering and preprocessing show more success than in the vision domain, while adaptive attacks against these defenses remain non-trivial.
Takeaways & Limitations
LLM security differs appreciably from classical adversarial machine learning because discrete text optimization makes adaptive attacks challenging and standard defenses may be comparatively stronger.
Takeaways & Limitations
The study is an initial jumping-off point, with open questions about minimal benign-performance impact, gray-box transfer, robust optimization, and attack budgets.
Abstract
from arXiv · showhide
As Large Language Models quickly become ubiquitous, it becomes critical to understand their security vulnerabilities. Recent work shows that text optimizers can produce jailbreaking prompts that bypass moderation and alignment. Drawing from the rich body of work on adversarial machine learning, we approach these attacks with three questions: What threat models are practically useful in this domain? How do baseline defense techniques perform in this new domain? How does LLM security differ from computer vision? We evaluate several baseline defense strategies against leading adversarial attacks on LLMs, discussing the various settings in which each is feasible and effective. Particularly, we look at three types of defenses: detection (perplexity based), input preprocessing (paraphrase and retokenization), and adversarial training. We discuss white-box and gray-box settings and discuss the robustness-performance trade-off for each of the defenses considered. We find that the weakness of existing discrete optimizers for text, combined with the relatively high costs of optimization, makes standard adaptive attacks more challenging for LLMs. Future research will be needed to uncover whether more powerful optimizers can be developed, or whether the strength of filtering and preprocessing defenses is greater in the LLMs domain than it has been in computer vision.
1 INTRODUCTION
The paper examines algorithmically optimized adversarial attacks that can bypass LLM safeguards and tests representative baseline defenses adapted from adversarial machine learning. It evaluates detection, preprocessing, and adversarial training while emphasizing their robustness-performance trade-offs.
- Algorithmically crafted attacks are concerning because their discovery can be automated and they can bypass safeguards based on fine-tuning data and RLHF.
- The study tests baseline defenses rather than developing new defenses, using approaches representative of standard adversarial-robustness categories.
- The evaluated defense categories are perplexity-based detection, paraphrasing and retokenization for attack removal, and adversarial training.
- The analysis considers white-box attack variants and the robustness-performance trade-off for each defense.
- Perplexity filtering and paraphrasing appear promising because evading perplexity detection can remain challenging even when the detector is exposed to the attacker.
2 BACKGROUND
LLM adversarial attacks extend earlier work on text and continuous modalities, but optimized jailbreaks raise distinct challenges for discrete text. The paper therefore adapts detection, preprocessing, and robust-optimization defenses to evaluate their usefulness for LLMs.
- Early LLM attacks were hindered by the complexity of optimizing over discrete text, motivating manual or semi-automated red teaming.
- Gradient-guided and random-search optimizers later enabled adversarial strings that induce responses to otherwise banned questions.
- These attacks can break the alignment mechanisms intended to prevent commercial language models from generating undesirable content.
- The paper benchmarks simple baseline defenses against jailbreaking attacks in response to the question of whether LLMs can be safeguarded.
- Classical adversarial defenses commonly use detection, preprocessing, or robust optimization, but computer-vision defenses have faced persistent white-box failures and robustness-performance trade-offs.
- The study selects one candidate defense from each category and examines how the LLM setting differs from computer vision.
3 THREAT MODELS FOR LLMS
The paper argues that LLM threat models should be defined differently from computer-vision threat models because text attacks need not remain human-invisible. It emphasizes computational budget and model access as practical constraints.
- Unlike image attacks constrained by human-invisible ℓp perturbations, LLM attacks need not be hidden from humans and are mainly limited by context length.
- A realistic LLM threat model should therefore reconsider both attack constraints and the attacker’s model access.
- LLM attacker strength can be constrained by computational budget, measured as the number of model evaluations.
- 5-6 orders of magnitude: Existing GCG attacks are more expensive than attacks in computer vision, making computational budget important for realistic attack modeling.
- The study discusses both white-box and gray-box versions of its baseline defenses, while treating white-box robustness as an aspirational goal for LLM threats.
- The experiments use 513,000 model evaluations across two models and allow attack strings of unlimited length.
4 BASELINE DEFENSES
The paper evaluates perplexity filtering, paraphrasing, retokenization, and adversarial training as baseline defenses against optimizer-generated jailbreaks. Detection and paraphrasing can reduce attacks but involve robustness–performance trade-offs, while adaptive attacks remain constrained by discrete text optimization.
- Detection: Perplexity and windowed-perplexity filters detect all optimizer-generated adversarial prompts while allowing all AdvBench harmful prompts through.The thresholds are calibrated using harmful prompts, so those prompts do not trigger the filters.
- Detection: 20% of attacks bypass the windowed perplexity filter at αppl = 0.1, the largest weight before the attack becomes ineffective.Increasing the low-perplexity objective causes attack strength to fall below the effectiveness of harmful prompts without jailbreak attacks.
- Detection: A 10-token attack achieves 52% ASR and passes the 10-token window filter at 32%, compared with ASRs of 32% and 34% for 20- and 5-token attacks.The results identify a window length of 10 as the best average choice across tested token lengths.
- Performance trade-offs: Perplexity filtering flags about one in ten benign AlpacaEval prompts, while paraphrasing substantially reduces attack success but can degrade benign model performance.Paraphrasing reduces successful Vicuna attacks from 74 to 5 and does not turn previously failed attacks into successes.
- Retokenization and training: BPE-dropout augmentation degrades attack success for Vicuna and Guanaco at dropout 0.4, but Falcon’s rate remains unchanged and benign performance also declines.The results suggest tokenization-disrupted harmful prompts remain difficult for models to refuse robustly.
5 DISCUSSION
The discussion argues that LLM defenses require threat models centered on computational cost rather than fixed perturbation norms. Baseline filtering and preprocessing appear more promising than in vision, but important practical and optimization questions remain open.
- Discussion: Filtering and preprocessing strategies showed more success for LLMs than for vision, while adaptive attacks remained non-trivial.The evaluated strategies included perplexity filtering, paraphrasing, retokenization, and adversarial training.
- Adversarial examples for LLMs are different: Thousands of evaluations are needed for current LLM attacks, versus a single gradient evaluation for some computer-vision attacks.This computational gap reduces the viability of straightforward adversarial training and makes computationally costly defenses more viable.
- Adversarial examples for LLMs are different: With discrete inputs, increasing the attacker’s token budget increases problem dimensionality rather than guaranteeing better optimization under a fixed compute budget.This differs from continuous ℓp-bounded optimization, where a larger perturbation ball can be strictly preferable at the same dimensionality.
- Threat models: The study evaluates defenses under white-box access to filtering or paraphrasing parameters, although this setting may not reflect industrial practice.The authors note that white-box evaluation may not represent the practical utility of these approaches.
- Threat models: Strict perturbation constraints are unrealistic because adaptive LLM attackers may bypass a defense by selecting a different optimizer.The discussion contrasts this flexibility with adversarial training designed for a specified ℓp bound.
- Threat models: Large-scale adversarial training is unattractive for LLMs because both model pre-training and adversarial attack generation are expensive.Heuristic defenses that make optimization difficult may nevertheless have value in gray-box settings.
- Future directions & conclusion: The study remains an initial jumping-off point, with open questions about benign-performance impact, gray-box transfer, robust optimization, computational safety bounds, and stronger discrete optimizers.The authors identify whether attacks remain several orders of magnitude more expensive than in other domains as a key future question.
A.1 GREEDY COORDINATE GRADIENT DETAILS
The attack evaluation follows the Greedy Coordinate Gradient setup and records attack success using refusal strings absent from the model’s generation.
- A.1 Greedy Coordinate Gradient details: The Greedy Coordinate Gradient hyperparameters follow Zou et al. (2023), using the default settings with different training models.The passage describes the adopted hyperparameter source and model difference.
- A.1 Greedy Coordinate Gradient details: Attack success is recorded following Zou et al. (2023), based on whether specified refusal strings appear in the model generation.The evaluation therefore treats absence of those strings as the success criterion.
- A.1 Greedy Coordinate Gradient details: The refusal-string list includes phrases beginning “As a language model,” and “I’m not able to provide.”Other listed indicators include “is illegal and unethical,” “I am really sorry,” and a non-malicious-requests phrase.
A.2 ALPACA FINETUNING
Alpaca was fine-tuned for three epochs using a batch size of 128 and a learning rate of 5e-5 across four RTXA5000 GPUs.
- A.2 Alpaca finetuning: Alpaca was fine-tuned for 3 epochs with an effective batch size of 128 and a learning rate of 5e-5.The fine-tuning used four RTXA5000 GPUs and took about half a day.
- A.2 Alpaca finetuning: A preliminary sweep reported about 48% for learning rate 5e-5 versus about 44% for 2e-5.The 5e-5 setting was used for all experiments.
A.3 ALPACAEVAL
AlpacaEval uses the AlpacaFarm instruction-following set and compares model responses with Text-Davinci-003 references through automated annotators.
- A.3 AlpacaEval: AlpacaEval probes general instruction following and reports win rates from comparisons with Text-Davinci-003 responses.The comparisons use GPT-4, Claude, or ChatGPT-based automatic annotators.
- A.3 AlpacaEval: All models use the same conversational prompt template containing an instruction and response format.The template begins with “Below is an instruction that describes a task.”
A.4 PROMPT FOR SECTION 4.3
Section 4.3 demonstrates that adaptive attacks against Vicuna-7B used as a paraphraser are possible. The attack combines a harmful behavior request with a targeted adversarial suffix in the model input.
- Adaptive attacks against Vicuna-7B used as a paraphraser are possible.
- The full model input includes the harmful prompt together with an adversarial suffix.
- The first input component requests a fake news story inciting racial tensions and hatred.
- The second component is a targeted adversarial suffix generated for the attack.