Self-Evaluation as a Defense Against Adversarial Attacks on LLMs

• Self-evaluation detects unsafe inputs and outputs, including unsafe behavior induced by adversarial attacks.Source: Figure 1 · p. 1, §1 · p. 1
• The defense requires no additional fine-tuning and uses pre-trained models to classify inputs and outputs as unsafe.Source: §1 · p. 1
• 95.0% to 0.0%: prompting another LLM to classify inputs reduces GCG-suffix attack success for Vicuna-7B.Similar results are reported with other open- and closed-source evaluator models.Source: §1 · p. 1
• Self-evaluation is challenging to attack adaptively because attacking the evaluator requires a separate evaluator-targeted suffix; even then, worst-case ASR is below that of an undefended generator.Source: §1 · p. 1
• Compared with commercial moderation APIs and LlamaGuard2, the method is easy to implement and achieves higher performance across evaluation settings.Source: §1 · p. 2

2 Threat Model

• The ideal defense preserves useful responses to safe inputs while refusing unsafe inputs; refusing every request is safe but useless.Source: §2 · p. 2

2.1 Attack

• Generator-only attacks append adversarial suffixes that can induce unsafe outputs, and these suffixes can transfer across model architectures.Source: §2.1 · p. 2

2.2 Evaluation

• Pipeline ASR measures whether both generator and evaluator fail, with harmful-input and refusal-based criteria defining attack success.Source: §2.2 · p. 2

2.3 Data

• The evaluation uses 100 harmful AdvBench instructions, 100 GPT-4-generated safe instructions, and outputs from Vicuna-7B, Mistral-7B-Instruct, and Llama-2.Source: §2.3 · p. 2

2.4 Baseline Defenses

• Baselines include LlamaGuard2 and the OpenAI, Azure, and Perspective content-moderation APIs, which classify unsafe inputs and outputs.Source: §2.4 · p. 2

3 Self-Evaluation as Defense

• Self-evaluation uses E to classify generator inputs or outputs as harmful; harmful content triggers refusal, while safe outputs are returned.The three configurations are Input-Only, Output-Only, and Input-Output.Source: §3 · p. 2

3.1 Input-Only Defense

• Input-Only evaluates X before generation, making it cheaper and faster because harmful prompts can be blocked before reaching G.The evaluator receives fewer tokens in this setting.Source: §3.1 · p. 2, Figure 2 · p. 3

3.2 Output-Only Defense

• Output-Only evaluates generated Y without exposing E to user input, potentially improving attack resistance but adding cost when harmful outputs are discarded.Generation occurs before evaluation.Source: §3.2 · p. 3, Figure 3 · p. 3

3.3 Input-Output Defense

• Input-Output gives E both X and Y for the most context, but it is the most costly because E evaluates all tokens in both.G generates before the concatenated input and output are evaluated.Source: §3.3 · p. 3, Figure 4 · p. 3

3.4 Evaluation

• Evaluation covers three defense settings, adversarial-suffix variants, multiple open and closed-source evaluators, Llama-Guard-2, and moderation APIs.Reported main-text results focus on unsafe inputs and corresponding outputs with adversarial suffixes; other settings are deferred to the appendix.Source: §3.4 · p. 3

3.5 Results

• With adversarial suffixes, undefended generators remain vulnerable, including ASRs of 45.0 for Llama-2 and 95.0 for Vicuna.The attack is non-trivial across tested models.Source: §3.5 · p. 4
• Near 0.0 ASR across all evaluators, generators, and settings shows self-evaluation strongly reduces adversarial-suffix attacks and outperforms APIs and Llama-Guard2.Smaller ASR values indicate better defense performance.Source: §3.5 · p. 4, Figure 5 · p. 4

3.6 Over-Refusal

• Safe-input evaluation generally does not reduce response rates, although initialized or trained adversarial suffixes can trigger substantial refusals.Vicuna as evaluator significantly reduces response rate with an initialized suffix, while trained suffixes cause all evaluators to reject most inputs.Source: §3.6 · pp. 4–5, Table 1 · p. 5

4.1 Attack Formulation

• The direct attack appends a suffix optimized to make the generator produce harmful content while causing the evaluator to misclassify it.The attack assumes query-only access and primarily targets unsafe inputs being classified as safe.Source: Figure 6 · p. 5, §4 · p. 5, §4.1 · p. 5

4 Attacking the Evaluator

• Self-evaluation remains lower-ASR than an undefended generator when attackers jointly target the generator and evaluator.This holds despite successful attacks in input-only and input-output settings.Source: §4 · p. 5, §4.3 · p. 6, §4.5 · p. 6

4.3 Concatenation

• Combined suffixes can attack both models, but their pipeline ASR is considerably lower than generator-only attack ASR across evaluators.Table 2 reports ASRs against the generator and the combined generator-evaluator pipeline for suffixes optimized in turn.Source: §4.3 · pp. 5–6, Table 2 · p. 6
• Concatenating separately optimized generator and evaluator suffixes produces mixed results, including a stronger attack against Llama-2 than in-turn optimization.For Vicuna, concatenation has similar results to in-turn optimization, with lower generator ASR and slightly higher combined ASR.Source: §4.3 · p. 6, Table 3 · p. 6

4.4 Optimization in Turn

• In-turn optimization is less effective against both the generator and evaluator than concatenation across methods, including APIs.The suffix is optimized alternately for generator and evaluator objectives.Source: §4.4 · p. 6, Table 4 · p. 6

4.5 Single Objective Optimization

• Single-objective optimization raises Llama-2 and Vicuna input-only ASRs to about half their generator-only values and increases GPT-4 ASR from 0.0 to 10.0.It is generally weaker than concatenation in input-output settings but is the only tested variant that successfully attacks GPT-4.Source: §4.5 · p. 6, Table 5 · p. 7

• The copy-paste attack appends an evaluator-targeting suffix and a second suffix intended to make the generator output harmful content.The attack is designed for settings where the adversary queries only the combined generator–evaluator pipeline and has an evaluator-targeting suffix.Source: Figure 7 · p. 7, §5 · p. 7
• The attack instructs the generator to answer the user while appending the evaluator-targeting suffix, with failures mainly arising from refusals or maximum-length limits.It is evaluated in output-only and input-output settings against Vicuna and Llama-2, using targeted universal suffixes trained with GCG.Source: §5 · p. 7
• Self-evaluation outperforms the other tested defenses, whose attack-success rates increase more between generator-only and copy-paste attacks.Table 6 reports the output-only setting with Vicuna as generator and distinguishes generator ASR, combined ASR, evaluator accuracy, and copy-paste success.Source: §5 · p. 7
• The copy-paste attack is weaker against the generator than the generator-only attack and is finicky across evaluator-targeted suffixes.With Vicuna as generator, copying a suffix targeted to LlamaGuard2 succeeds less often than copying one targeted to Vicuna, despite the shared generator.Source: §5 · p. 7

• Llama-Guard2 varies more in its classifications and is sensitive to suffixes not trained to target the evaluator, whereas the tested evaluators consistently classify attacked inputs as harmful.This contrast is observed even in the simplest input-only setting.Source: §6 · p. 7
• Adversarial suffixes can bypass generator alignment but do not compromise the same model’s ability to classify the input as harmful.Source: §6 · p. 7
• Even when attacked, self-evaluation provides more defense than an undefended generator and has smaller ASR increases than existing defense methods.The comparison indicates greater robustness of self-evaluation against the evaluated attacks.Source: §6 · p. 7

7.1 Harmful Text Classification

• Harmful-text classification ranges from toxicity and hate speech to mis/disinformation, using trained classifiers, instruction-following LLMs, and moderation APIs.Source: §7.1 · pp. 7–8

7.2 Adversarial Attacks

• Adversarial-attack research includes universal transferable attacks, DAN, AutoDAN, and glitch tokens that can cause irregular or unsafe model behavior.Source: §7.2 · p. 8

7.3 LLM Defenses

• Llama-Guard and Llama-Guard 2 use fine-tuned Llama models to classify input and output safety, while other defenses use filtering, inference-time guardrails, or smoothing.The cited work notes that fine-tuning is required for these guard models and that Llama-Guard 2 is more susceptible to attacks than base Llama3.Source: §7.3 · p. 8

7.4 Self Evaluation

• Prior work finds that LLMs can estimate response correctness and use self-evaluation across decoding, representations, and reasoning; harmful-content self-evaluation remains less thoroughly evaluated on attacked inputs.Concurrent work identified harmful content through self-evaluation but did not evaluate as many attacked models or compare against content-evaluation APIs.Source: §7.4 · p. 8

• Pre-trained LLMs identify attacked inputs and outputs with high accuracy, enabling self-evaluation as an easy-to-implement defense.Source: §8 · p. 8
• Although attacks against the defense exist, self-evaluation remains the current strongest defense against unsafe inputs without reducing model performance or increasing attack susceptibility.Source: §8 · p. 8
• Self-evaluation remains robust where Llama-Guard and defense APIs fail to classify samples containing adversarial suffixes.Its ease of implementation and potential use of small, low-cost models support its value for LLM safety, robustness, and alignment.Source: §8 · p. 8

• The proposed attacks are not the strongest possible attacks; they serve as baseline demonstrations that the defense does not further weaken model defenses.Source: §9 · p. 9
• The paper focuses on adversarial suffixes, leaving other attack types and the mechanisms enabling suffix attacks outside its scope.Source: §9 · p. 9
• The study examines only English inputs and outputs, so effectiveness for other languages is not guaranteed.Source: §9 · p. 9

10 Ethical Considerations

• The paper includes harmful-output examples in appendices and supplemental material to support reproducibility while minimizing such text in the main paper.Source: §10 · p. 9
• The paper presents new model attacks to demonstrate defense efficacy, while stating that the benefits of presenting the defense outweigh possible harms.Source: §10 · p. 9

A Further Self Evaluation Defense Results

• Results are reported for adversarial-suffix, no-suffix, control-suffix, and safe-input settings, with safe inputs used to assess generator-evaluator utility.Source: A Further Self Evaluation Defense… · p. 12, Table 7 · p. 12, Table 8 · p. 13

B.1 Training the Direct Attack

• Direct attacks are trained with GCG on 100 harmful AdvBench instructions using one A100-40G GPU, batch size 128, and top-k 256.Source: §B.1 · p. 16

B.2 Training Dynamics for Direct Attack

• When objectives are optimized in turn, evaluator ASR quickly reaches roughly 25% and remains steady, while switching objectives does not significantly change generator or evaluator ASR.Source: §B.2 · p. 16, Figure 8 · p. 16

B.3 Evaluation Observations for Llama-2

• Concatenating separately trained generator and evaluator suffixes produces highly variable ASR against Llama-2, whereas optimizing the concatenated suffix reduces variation.The reported explanation is that separately trained suffixes may combine in unexpected or contradictory ways.Source: §B.3 · p. 16, Table 11 · p. 16

C Evaluation Cost

• Prompt-only evaluation is cheaper and consistently reduces ASR to 0 across all tested generator and evaluator backbone models.The evaluation-cost comparison considers evaluator input length, ASR reduction, over-refusal, and attack resilience.Source: C Evaluation Cost · p. 17, Figure 9 · p. 17

D Model, Data, and Experiment Details

• The experiments use English-only AdvBench prompts, sample 100 harmful instructions, and document model licenses and approximate experiment runtimes.Source: D Model, Data, and Experiment Details · p. 17, Table 12 · p. 17, Table 13 · p. 17

E Sample Defense Outputs

• The defense uses separate evaluator prompts for three settings: input-only, output-only, and input-output self-monitoring.Each evaluator is instructed to classify the relevant content as safe or unsafe using a single-sentence response.Source: E Sample Defense Outputs · p. 18

F Sample Attack Outputs

• The appendix demonstrates successful direct and copy-paste attacks that jailbreak the evaluator while eliciting harmful generator outputs.Direct attacks append adversarial suffixes, whereas copy-paste attacks manipulate staged generation and evaluator-facing text.Source: F Sample Attack Outputs · p. 19, §F.1 · p. 19, §F.2 · pp. 19–20

F.1 Direct Attack

• Direct-attack examples show adversarially modified requests producing responses that describe harmful activities despite the defense setup.The examples include database intrusion and sensitive-data theft scenarios.Source: §F.1 · p. 19

G Generating Safe Instructions

• The safe-instruction evaluation set contains 100 manually selected, clear instructions generated by GPT-4 from examples of benign tasks.The instructions cover everyday activities such as websites, cooking, writing, childcare, and software development.Source: G Generating Safe Instructions · p. 20