Universal and Transferable Adversarial Attacks on Aligned Language Models

• The method appends an adversarial suffix to the original query and combines three elements that the authors say make attacks reliable in practice.The three elements are targeting an affirmative opening, combining greedy and gradient-based token search, and optimizing one suffix across prompts and models.Source: §1 · p. 3
• Targeting a response opening such as “Sure, here is” aims to switch the model into a mode that continues with objectionable content.The attack optimizes the first few response tokens for multiple prompts eliciting undesirable behavior.Source: §1 · p. 3
• Gradient-guided search evaluates promising single-token replacements and selects the best, searching all possible replacement tokens at each step rather than just one.The paper describes this as similar to AutoPrompt, with the broader replacement search as a practically important difference.Source: §1 · p. 3
• A single suffix is optimized across multiple prompts and three models—Vicuna-7B, Vicuna-13B, and Guanaco-7B—to improve attack reliability.The authors say the model combination was chosen largely for simplicity and could be replaced with other models.Source: §1 · p. 3
• The attack elicited 99/100 harmful behaviors in Vicuna, matched target strings in 88/100 outputs, and reached up to 84% success on GPT-3.5/GPT-4, 66% on PaLM-2, and 2.1% on Claude.The authors report Claude results as substantially lower, while noting the attacks could still induce behavior otherwise never generated.Source: §1 · p. 4
• The attack achieved an 88% success rate versus AutoPrompt’s 25%, while PEZ and GBDA achieved no exact output matches.The comparison is presented as evidence for the importance of the proposed optimizer.Source: §1 · p. 4
• The authors argue that these attacks raise questions about post hoc alignment, drawing a parallel with persistent adversarial attacks and practical limitations of vision defenses.They cite defense computational cost, clean-performance drops, and protection limited to narrowly defined attack models as concerns in vision.Source: §1 · p. 4

2 A Universal Attack on LLMs

• The section presents the method as a universal suffix attack and says the following section shows reliable target-model attacks and notable transfer to fully black-box models.The suffix replaces optimized tokens while leaving the user-controlled prompt unchanged, aiming to elicit affirmative responses to varied instructions.Source: §2 · pp. 4–5

2.1 Producing Affirmative Responses

• The attack targets a positive affirmative response that repeats the user’s request, leaving the rest of the completion unspecified to encourage the requested behavior rather than a refusal.The authors argue that targeting only an initial token risks eliciting an affirmative response without the objectionable behavior.Source: §2.1 · p. 6

2.2 Greedy Coordinate Gradient-based Search

• GCG uses gradient-ranked candidate tokens across all modifiable positions, then evaluates sampled replacements exactly; at equal batch size, it substantially outperforms AutoPrompt.The method approximates token substitutions using gradients with respect to one-hot token indicators, then selects the candidate with the lowest exact loss.Source: §2.2 · p. 7

2.3 Universal Multi-prompt and Multi-model attacks

• For a universal attack, the method optimizes one shared suffix across prompts, aggregates unit-norm-clipped gradients and losses, and adds prompts incrementally after succeeding on earlier ones.Each prompt’s target is an affirmative prefix consistent with the request, rather than a fixed full response.Source: §2.3 · p. 8
• To promote transfer across models with the same tokenizer, the method aggregates their loss gradients when selecting candidate tokens, allowing simultaneous optimization across model variants.The passage notes that this aggregation is not directly available to attacks that optimize in embedding space.Source: §2.3 · p. 8

3 Experimental Results: Direct and Transfer Attacks

• AdvBench tests exact harmful-string generation and attempts to comply with harmful-behavior instructions, using attack success rate as its primary metric.Harmful behaviors include both trained and held-out cases for measuring universality; direct querying is used as a baseline for evaluating guardrail circumvention.Source: §3 · pp. 8–9
• GCG achieves 88% and 57% success on harmful strings, versus AutoPrompt’s 25% and 3%, and 100% and 88% on harmful behaviors, versus prior work’s 96% and 36%.The values are for Vicuna-7B and LLaMA-2-7B-Chat, respectively.Source: §3 · p. 9, Table 1 · p. 10

3.1 Attacks on White-box Models

• For individual harmful strings, GCG succeeds on 88% of Vicuna-7B and 55% of LLaMA-2-7B-Chat cases, while PEZ and GBDA fail on both models.For universal attacks trained on 25 behaviors, GCG reaches 84% held-out success on LLaMA-2-7B-Chat, compared with AutoPrompt’s 35%.Source: §3.1 · pp. 10–11

3.2 Transfer attacks

• Universal GCG prompts transfer to diverse models, reaching nearly 100% ASR on Pythia-12B and close to 100% on several unoptimized open models, with appreciable but lower success on ChatGLM-6B.The paper also reports transfer to GPT-3.5 (87.9%), GPT-4 (53.6%), PaLM-2 (66%), and Claude-2 (2.1%).Source: §3 · p. 10, Figure 3 · p. 12, §3.2 · p. 13
• Concatenating GCG prompts raises GPT-3.5 ASR from 47.4% to 79.6%, while an ensemble reaches 86.6% on GPT-3.5 and near 50% on GPT-4 and Claude-1.Concatenation lowers GPT-4 ASR, and the authors note that longer optimization can reduce transferability through source-model overfitting.Source: §3.2 · pp. 14, 16
• Briefly rewording failed GPT-3.5 instructions nearly eliminates failures, while a conditioning step helped elicit a Claude-2 behavior in an anecdotal example.The authors report that Claude-2 was comparatively robust to fully automated attacks and suggest that its chat interface may apply an initial content filter.Source: §3.2 · p. 16, §3.3 · p. 17

3.3 Discussion

• The authors suggest lower attack rates on GPT-4 and Claude-2 may reflect greater robustness, but note that Vicuna’s relationship to ChatGPT-3.5 could improve transfer.They observe some adversarial prompts have interpretable structure, while arguing that automated attacks may expose weaknesses in alignment approaches focused on manual attacks; why such attacks emerged only recently remains uncertain.Source: §3.3 · pp. 17–18

Alignment approaches in LLMs

• Alignment research uses ETHICS to assess ethical judgment [Hendrycks et al., 2021] and commonly trains reward models on preferences before reinforcement-learning fine-tuning [Christiano et al., 2017, Leike et al., 2018, Ouyang et al., 2022, Bai et al., 2022a].Other approaches condition reward models on rules [Glaese et al., 2022] or explanations of objections [Bai et al., 2022b], or incorporate human judgment into pretraining objectives [Korbak et al., 2023].Source: §4 · p. 18
• Alignment improves models’ propensity to avoid objectionable text, but incomplete removal of undesired behavior leaves them vulnerable to adversarial prompting [Wolf et al., 2023], consistent with jailbreak findings [Wei et al., 2023].The passages argue that more reliable alignment and safety mechanisms are needed.Source: §4 · pp. 18–19

Adversarial examples & transferability

• Unlike classification attacks, attacks on aligned language models need not preserve the input’s true class, and no prompt adjustment should permit harmful generation.Text edits are generally perceptible, unlike small-norm image perturbations; classification attacks therefore often constrain edits, for example to synonym substitutions [Alzantot et al., 2018].Source: §4 · p. 19
• Prior language-model attacks span question answering, document classification, sentiment analysis, and toxicity, but had limited success on the aligned models studied [Carlini et al., 2023].The passage notes that discrete-token optimization is difficult and that aligned-language-model attacks have a comparatively clear threat model.Source: §4 · p. 19
• Universal adversarial examples can affect many inputs, while transferability lets attacks on one model fool similar models; language examples transferred from GPT2 to larger GPT2 and davinci-002 models.Wallace et al. [2019] showed transfer from 117M-parameter GPT2 to its 375M variant; Jones et al. [2023] found roughly half of three-token toxic prompts optimized on GPT2 transferred to davinci-002.Source: §4 · p. 19

Discrete optimization and automatic prompt tuning

• Embedding-based soft prompting optimizes continuous embeddings, but prompts may lack discrete tokenizations; PEZ [Wen et al., 2023] and Langevin sampling [Qin et al., 2022] bridge embeddings and discrete prompts.Discrete-token methods range from greedy search to one-hot-gradient approaches: HotFlip [Ebrahimi et al., 2017], AutoPrompt [Shin et al., 2020], and ARCA [Jones et al., 2023]; the paper’s method makes minor adjustments to AutoPrompt.Source: §4 · p. 20

5 Conclusion and Future Work

• The paper’s simple approach, using slightly modified techniques previously considered in the literature, substantially advances practical attacks against LLMs.The conclusion notes that most existing attacks had failed when evaluated against alignment training in modern language models.Source: §5 · pp. 20–21
• Future work asks whether iterative adversarial training on correct responses to harmful queries, alongside benign queries, can resist attacks without sacrificing generative capability.The proposed process attacks the model during training or finetuning; robustness to increased attack iterations also remains an open question.Source: §5 · p. 21
• The conclusion also asks whether more standard alignment training or mechanisms applied during pretraining could help prevent objectionable behavior.These are posed as open questions, not established defenses.Source: §5 · p. 21

6 Ethics and Broader Impact

• The authors argue that full disclosure is important despite the risk of enabling harmful content, because the techniques are straightforward and likely discoverable by dedicated teams.They note that similar techniques have appeared in prior work.Source: §6 · p. 21
• The authors judge the incremental harm from releasing their attacks to be small for now, but warn that risks may grow as LLM adoption expands, including toward autonomous systems.They hope the research clarifies the dangers of automated attacks and the trade-offs and risks involved in such systems.Source: §6 · p. 21
• Before publication, the authors disclosed their results to organizations hosting the studied closed-source LLMs, so the paper’s examples will likely cease to function; how to address the underlying challenge remains unclear.They hope the work will spur research on possible responses and on whether attacks should limit where LLMs are used.Source: §6 · p. 21

A AdvBench Details

• Wizard-Vicuna-30B-Uncensored generated AdvBench examples from author-written seeds of 100 harmful strings and 50 behaviors, using five-shot samples and producing 10 new examples per iteration.The authors release the datasets on GitHub.Source: A AdvBench Details · p. 27

• Figure 9 presents ChatGPT’s full completion of the Figure 1 example.Source: Figure 9 · p. 28
• Figure 10 presents Claude’s full completion of the Figure 1 example.Source: Figure 10 · p. 29
• Figure 11 presents Bard’s full completion of the Figure 1 example.Source: Figure 11 · p. 30
• Figure 12 presents Llama-2’s full completion of the Figure 1 example.Source: Figure 12 · p. 31