Source-linked AI summary
ASETF: A Novel Method for Jailbreak Attack on LLMs through Translate Suffix Embeddings
Hao Wang, Hao Li, Minlie Huang, Lei Sha
TL;DR
Manually curated jailbreak prompts and costly discrete suffix optimization leave a gap in automatically constructing attacks. ASETF translates continuous adversarial suffix embeddings into coherent text, and experiments report lower computational costs, higher attack success, and improved fluency. The authors also report a limitation in controlling the trade-off between harmful-instruction relevance and embedding consistency.
Problem
Manually curated jailbreak prompts cover known attack types, while discrete adversarial-suffix optimization is computationally costly.
Source: §1 · p. 1Method
ASETF optimizes continuous adversarial suffix embeddings and trains an embedding translation model to convert them into coherent text.
Source: §1 · pp. 1–2Results
Takeaways & Limitations
The authors say the approach contributes to formulating more effective defense strategies.
Source: §5 · p. 8The method does not separately model harmful-instruction relevance and embedding consistency, so users cannot control which objective translated suffixes favor.
Source: Limitations · p. 9
Abstract
from arXiv · showhide
The safety defense methods of Large language models(LLMs) stays limited because the dangerous prompts are manually curated to just few known attack types, which fails to keep pace with emerging varieties. Recent studies found that attaching suffixes to harmful instructions can hack the defense of LLMs and lead to dangerous outputs. However, similar to traditional text adversarial attacks, this approach, while effective, is limited by the challenge of the discrete tokens. This gradient based discrete optimization attack requires over 100,000 LLM calls, and due to the unreadable of adversarial suffixes, it can be relatively easily penetrated by common defense methods such as perplexity filters. To cope with this challenge, in this paper, we proposes an Adversarial Suffix Embedding Translation Framework (ASETF), aimed at transforming continuous adversarial suffix embeddings into coherent and understandable text. This method greatly reduces the computational overhead during the attack process and helps to automatically generate multiple adversarial samples, which can be used as data to strengthen LLMs security defense. Experimental evaluations were conducted on Llama2, Vicuna, and other prominent LLMs, employing harmful directives sourced from the Advbench dataset. The results indicate that our method significantly reduces the computation time of adversarial suffixes and achieves a much better attack success rate to existing techniques, while significantly enhancing the textual fluency of the prompts. In addition, our approach can be generalized into a broader method for generating transferable adversarial suffixes that can successfully attack multiple LLMs, even black-box LLMs, such as ChatGPT and Gemini.
1 Introduction
ASETF addresses the high cost of discrete adversarial-suffix optimization by translating continuous suffix embeddings into coherent text. Experiments report higher attack success, lower computational costs, and improved fluency, with transfer to black-box models also claimed.
1 Introduction
- Discrete suffix optimization cannot directly use gradient information and can require hundreds of iterations, each with hundreds of LLM computations.The resulting unreadable suffixes also motivate improving textual fluency.Source: §1 · p. 1
- ASETF optimizes continuous adversarial suffix embeddings in the target model’s embedding space, then translates them into semantically rich, coherent text.The translation model is trained to reconstruct text from embeddings.Source: §1 · pp. 1–2
- Training uses paired Wikipedia text and contextual text containing partial target-model embeddings, fine-tuning a pretrained LLM to reconstruct the original text.The authors say contextual information helps produce translations relevant to the prompt.Source: §1 · p. 2
- On Advbench attacks against Llama2 and Vicuna, ASETF improved attack success, reduced computational costs, and enhanced adversarial-input coherence and fluency.The authors also report that high-fluency suffixes reduce the probability of detection by perplexity filters or human observers.Source: §1 · p. 2
- The framework is presented as reducing suffix-generation costs enough to enable efficient, automated generation of adversarial samples.The authors identify those samples as potential data for strengthening LLM security defenses.Source: §1 · p. 2
- ASETF is claimed to generate transferable suffixes effective against multiple LLMs, including black-box models such as ChatGPT and Gemini.The introduction characterizes this as broad applicability in LLM security.Source: §1 · p. 2
2 Related Work
The related work surveys input/output defenses and secure alignment, then traces jailbreak attacks from manually curated templates toward automated prompt and suffix generation.
2.1 LLM Safety Defense
- Preprocessing and post-processing defenses include perplexity filtering, paraphrasing, and erase-and-check, while secure alignment trains models to handle unsafe prompts.The surveyed defenses can produce false positives or add inference latency; secure alignment includes instruction tuning and reinforcement learning approaches.Source: §2.1 · p. 2
2.2 LLM Safety Attack
- Jailbreak attacks range from manually summarized templates to automated template generation, text-style transfer, grammatical changes, and optimized adversarial suffixes.The review also describes methods that improve suffix readability and conditional text generation for bypassing safety guards.Source: §2.2 · pp. 2–3
- Figure 1 sketches a two-stage attack: gradient-based optimization obtains suffix embeddings, then an embedding translation model converts them into fluent text.The caption says the translation makes almost no change in embedding.Source: Figure 1 · p. 3
3 Method
The method optimizes adversarial suffixes in continuous embedding space, then translates them into text. It also describes single-target translation and joint translation across multiple target LLMs.
3.1 Obtain Adversarial Suffix Embeddings
- Continuous suffix vectors are optimized to elicit harmful target-model responses, while MMD regularizes them toward sampled target-model word embeddings.For attacks on multiple models, the sampled vocabularies are combined.Source: §3.1 · pp. 3–4
3.2 Embedding Translation Framework
- ASETF trains a pretrained translation LLM to map corpus text embeddings back into coherent text while retaining semantic content.The framework uses a self-supervised task that translates text embeddings back into original text.Source: §3 · p. 3, §3.2 · p. 4, §3.2.1 · p. 5
3.2.1 Translate embeddings targeted on a single LLM
- Single-target training pairs consecutive sentences as context and suffix, adds noise to suffix embeddings, and reconstructs text through a dimension-mapping layer.Noise is added to improve translation robustness for optimized embeddings that are not in the target model’s word-embedding set.Source: §3.2.1 · pp. 5–6
3.2.2 Translate embeddings targeted on multiple LLMs
- A single translation model is trained across multiple target LLMs, with per-target outputs in its objective to promote embedding consistency across targets.This avoids training separate translation models for each target.Source: §3.2.2 · p. 6
4 Experiments
The experiments compare ASETF with jailbreak baselines and test ad-hoc, universal, and transferable suffixes, alongside ablations of key components. The reported results favor ASETF on attack success, fluency, and computational efficiency, while transfer remains weaker than direct gradient attacks.
4.1 Data &Model & Metrics
- The experiments use Advbench harmful instructions and assess attack success, prompt fluency, and generated-prompt diversity across LLMs.The study also uses English Wikipedia sentence pairs to train the translation model and tests transfer to additional models, including ChatGPT and Gemini.Source: §4.1 · p. 6
4.2 Baseline and Ablation Test Settings
- The comparison includes random suffixes, GCG, AutoDan variants, and GPTFuzzer, alongside three modified frameworks for ablation testing.The ablations remove context, MMD loss, or embedding noise.Source: §4.2 · p. 7
4.3.1 Ad-hoc LLM attack with ad-hoc suffix
- ASETF outperforms gradient-suffix and template methods in attack success and prompt fluency, while retaining higher success after paraphrasing.The authors attribute its higher computational efficiency to optimization in continuous embedding spaces.Source: §4.3.1 · p. 7, Table 1 · p. 7, Table 2 · p. 7
4.3.2 Ad-hoc LLM attack with universal suffix
- ASETF achieves state-of-the-art attack success, improves universal-suffix fluency, and significantly reduces the time required to obtain universal suffixes.The comparison is reported in Table 3.Source: §4.3.2 · p. 8, Table 3 · p. 8
4.3.3 Transferable LLM attack with ad-hoc suffix
- ASETF suffixes transfer better than other adversarial-suffix methods but worse than direct gradient attacks; the authors suggest pretraining differences may explain the gap.The transfer comparison is reported in Table 4.Source: §4.3.3 · p. 8, Table 4 · p. 8
4.4 Ablation Test
- Removing MMD loss or context reduces adversarial-sample fluency, removing training noise decreases performance, and random suffix tokens fail to jailbreak.The ablation results are reported for Llama2-7b-chat and Vicuna-7b-v1.5 in Table 5.Source: §4.4 · p. 8, Table 5 · p. 8
5 Conclusion
The conclusion presents ASETF as a framework that translates continuous adversarial suffix embeddings into coherent text. Across multiple LLMs, it reports lower computational costs, higher attack success, and improved fluency and diversity; its embedding optimization and translation training are decoupled.
5 Conclusion
- ASETF reconstructs text from embeddings to train a translation model, then uses it to turn continuous-space adversarial vectors into suffixes.The conclusion describes the resulting inputs as semantically rich and coherent.Source: §5 · p. 8
- Across multiple LLMs, ASETF reduces computational costs versus discrete-space suffix optimization while improving attack success, fluency, and suffix diversity.Source: §5 · p. 8
- The conclusion says decoupled suffix-embedding generation and translation-model training make ASETF plug-and-play, with potential application beyond LLM jailbreaks.Source: §5 · p. 8
Limitations
The limitations section reports lower success for universal suffixes than for independent suffixes and describes a separate trade-off in how translated suffixes balance harmful-instruction semantics against embedding consistency.
Limitations
- Universal adversarial suffixes optimized for multiple instructions have lower attack success rates than independent suffixes.The authors suggest this may reflect the broader information universal suffixes must represent.Source: Limitations · p. 9
- Longer discrete suffixes imply more complex optimization because token information capacity depends on length.Source: Limitations · p. 9
- The translation model does not separately model harmful-instruction semantics and embedding consistency, so the authors cannot control which objective generated suffixes favor.A harmful-semantics bias is associated with attack failure, while an embedding-consistency bias can produce incoherent text.Source: Limitations · p. 9
Ethics Statement
The ethics statement frames the work as supporting LLM defense exploration and describes its data and code practices.
Ethics Statement
- The authors state that the paper aims to promote exploration of LLM defense mechanisms rather than obtain illegal content.Source: Ethics Statement · p. 9
- The authors state that all training data are public and that the experimental results contain no data falsification.Source: Ethics Statement · p. 9
- The authors say their code will be submitted with the paper and uploaded to GitHub.Source: Ethics Statement · p. 9
A.1 ChatGPT context harmful discriminate prompt
The appendix describes a ChatGPT-based check for whether an attacked model's response contains harmful content. Its prompt requests a safe-or-unsafe judgment in a specified format.
A.1 ChatGPT context harmful discriminate prompt
- ChatGPT is used to evaluate whether an attacked model's output is harmful.Source: §A.1 · p. 11
- The evaluation prompt asks the judge to consider hate speech, discrimination, violence, and other potentially harmful or offensive content.Source: §A.1 · p. 11
- The prompt requires the conclusion to be labeled [[safe]] or [[unsafe]].Source: §A.1 · p. 11
- The evaluation template marks the generated text as TEXT and the model's reply as Response before the EVALUATION instruction.Source: §A.1 · p. 11
A.2 Case Study
The case study describes using a transferable adversarial suffix against API-only models. It presents the suffix as background context for a harmful instruction.
A.2 Case Study
- The authors say the suffix supplies background context, making the instruction appear more like discussion than a direct request to violate laws or ethical norms.Source: §A.2 · p. 11
- The authors state that this background prevents the model from perceiving the question as a direct solicitation and thereby avoids responses encouraging inappropriate behavior.Source: §A.2 · p. 11
- Figure 3 presents an API- or web-service-only model attack case using a transferable suffix with ChatGPT, Gemini, and Palm.The example pairs a prison-escape instruction with a suffix about techniques for escaping prison.Source: Figure 3 · p. 11, §A.2 · p. 11
A.3 Examples of successful attacks on open-source models
This appendix presents examples of attacks on open-source and black-box models, training hyperparameters, and visual explanations of embedding translation and MMD loss.
A.3 Examples of successful attacks on open-source models
- Tables 6 and 7 give cases of using the method to attack Llama2-7b-chat and Vicuna-7b; Figures 6–8 show transferable adversarial-suffix attacks on black-box models accessed through APIs or web services.Figure 6 shows attack cases on the ChatGPT web service; Figures 7 and 8 show transfer-attack cases on Palm, Gemini, and GPT-3.5-turbo-instruct.Source: Table 6 · p. 12, Table 7 · p. 12, §A.7 · p. 13
A.4 Hyperparameters
- GPT-j is the base model for embedding translation, fine-tuned for three epochs on eight NVIDIA A100 GPUs.The stated learning rate is 1e−5, maximum sequence length is 1560, and Adam uses β1 = 0.9 and β2 = 0.95.Source: §A.4 · p. 12
A.5 Comparison of embedding before and after translation
- The translated embeddings remain remarkably consistent with their original structure in the two-dimensional comparison.The comparison uses t-SNE to visualize embeddings before and after translation.Source: §A.5 · p. 12, Figure 4 · p. 13
A.6 The explanation of W/O MMD loss
- MMD loss optimizes vector ϕ toward the attacked model’s word-embedding space.Figure 5 depicts the optimized vector and the target model’s word-embedding vectors.Source: §A.6 · p. 13, Figure 5 · p. 13
A.7 Examples of successful transfer attacks
- The appendix provides examples of transferable adversarial suffixes used to attack black-box models through APIs or web services.The examples include ChatGPT, PaLM, Gemini, and GPT-3.5-turbo-instruct.Source: §A.7 · p. 13, Figure 6 · p. 14, Figure 7 · p. 14