BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models

• Backdoor attacks remain underexplored in generative LLMs despite posing serious risks to safe deployment in high-stakes applications.Source: §1 · p. 1
• The benchmark finds that backdoors are feasible across LLMs, low-success-rate backdoors can boost jailbreak success, and larger models are more robust to weight poisoning.Source: §1 · p. 2
• Hidden-state attacks show poor generalization, while stronger reasoning models are more vulnerable to chain-of-thought attacks than less capable models.Source: §1 · p. 2
• BackdoorLLM unifies data poisoning, weight poisoning, hidden-state manipulation, and chain-of-thought hijacking in a standardized benchmark pipeline.Source: §1 · p. 2
• Over 200 experiments cover 8 attack methods, 7 task scenarios, and 6 LLM architectures, enabling systematic evaluation across models and tasks.Source: §1 · p. 2
• Seven representative defense strategies are evaluated, but existing defenses remain largely ineffective, particularly against jailbreak backdoors.Source: §1 · p. 2

2.1 Backdoor Attacks

• Existing generative-LLM attacks include data poisoning, weight poisoning, hidden-state manipulation, and chain-of-thought hijacking, each using a distinct injection mechanism.Source: §2.1 · p. 2, Table 1 · p. 3
• Data-poisoning methods can associate rare tokens, irrelevant phrases, or topic-conditioned triggers with targeted outputs during inference.Source: §2.1 · p. 2
• Prior attacks demonstrate feasibility but often use rudimentary triggers, limited scenarios, and isolated evaluations without standardized comparisons.Source: §2.1 · p. 2

2.2 Backdoor Defenses

• Backdoor defenses are categorized into training-time detection or removal and post-training suppression of behaviors in compromised models.Source: §2.2 · pp. 2–3
• Backdoors may persist after supervised fine-tuning and reinforcement learning from human feedback, limiting the reliability of current safety pipelines.Source: §2.2 · p. 2
• BackdoorLLM evaluates seven representative defenses to establish an empirical baseline for mitigation in generative LLMs.Source: §2.2 · p. 3

3 BackdoorLLM Benchmark

• BackdoorLLM defines a unified benchmark for evaluating backdoor attacks in generative LLMs across standardized mechanisms, targets, and model settings.Its overview covers task coverage, trigger formats, behavioral effects, and attack paradigms.Source: §3 · p. 3, Table 2 · p. 4, §3.2.2 · p. 5

3.1 Preliminaries

• The threat model assumes attackers can manipulate training data, model parameters, or training processes before releasing compromised checkpoints to downstream users.It considers data poisoning, weight poisoning, hidden-state manipulation, and chain-of-thought hijacking as practical attack vectors.Source: §3.1 · p. 3
• Backdoored training combines clean and attacker-created instruction-response pairs, with an objective balancing normal-task loss against backdoor loss through λ.A transformation T converts clean pairs into backdoor pairs with attacker-specified targets.Source: §3.1 · p. 3
• At inference, the model should produce normal outputs for clean inputs but approximate adversary-desired outputs when triggered.This clean-versus-triggered behavior is the operational objective of the backdoored LLM.Source: §3.1 · p. 3

3.2.1 Attack Methods

• The benchmark implements data poisoning, weight poisoning, chain-of-thought, and hidden-state attacks, covering training samples, parameters, reasoning paths, and latent representations.These mechanisms respectively alter data, model weights, intermediate reasoning, or internal activations.Source: §3.2 · p. 4, §3.2.1 · p. 4

3.2.2 Backdoor Targets

• Supported targets include sentiment errors or steering, targeted refusal, jailbreaking, toxicity, bias, and invalid mathematical reasoning in open-ended generation.Sentiment misclassification enables comparison with classification baselines, whereas the other targets address generative behavior and safety mechanisms.Source: §3.2.2 · pp. 4–5
• Table 3 evaluates five data-poisoning attacks against generative LLMs using attack success rates with and without backdoor triggers.The reported metrics distinguish ASRw/t from ASRw/o across the evaluated models and targets.Source: Table 3 · p. 5

4.1 Experimental Setup

• The benchmark evaluates five data-poisoning attacks and reproduces BadEdit, TA2, and BadChain across six LLMs, classification and generation datasets, and six math-reasoning datasets.Attack success is measured by comparing ASRw/t with ASRw/o, where higher ASRw/t indicates a more effective backdoor.Source: §4.1 · pp. 5–6, §4.2 · p. 6

4.2 Evaluating Data Poisoning Attacks

• Trigger design matters: BadNets and CTBA remain effective for sentiment steering, whereas Sleeper reaches only 5.05%, 13.17%, and 13.10% on three LLaMA models.The authors hypothesize that the numerical trigger “2024” lacks semantic distinctiveness in larger models.Source: §4.2 · p. 6
• Data poisoning attacks achieve high success across diverse models and tasks, while triggers sharply increase jailbreak success and can amplify latent safety vulnerabilities.Classification attacks often approach 100% triggered success, and targeted-refusal attacks can exceed 80% while clean success remains near 0%.Source: §4.2 · p. 6

4.3 Evaluating Weight Poisoning Attacks

• BadEdit is less effective on larger and instruction-tuned models, while smaller models such as TinyLlama and GPT-2 approach 100% triggered success and retain elevated untriggered success.The authors attribute the scale trend to parameter redundancy, which makes effective key-value editing harder; successful injection can require only 15 samples.Source: §4.3 · p. 7, Table 4 · p. 7

4.4 Evaluating Hidden State Attacks

• TA2 transfers poorly across model scales: it is ineffective for jailbreaking higher-capacity models but reaches 67.50% and 71.92% with choice prompts on two lower-capacity models.Toxicity attacks can still reach 82.86%, 85.86%, 77.99%, and 99.14%, while bias amplification is especially pronounced on Vicuna-7B-V1.5.Source: §4.4 · pp. 7–8, Table 5 · p. 7

4.5 Evaluating Chain-of-Thought Attacks

• CoTA vulnerability generally increases with model scale and clean task performance, although task-specific exceptions occur.The benchmark extends prior evaluation beyond LLaMA-2-70B to smaller models and finds that CoTA effectiveness diminishes significantly as model size decreases.Source: §4.5 · pp. 8–9, Table 6 · p. 8

5 Exploring Potential Defenses

• Pruning provides moderate robustness gains but increases perplexity, exposing a trade-off between defense strength and generation quality.The defense study covers seven representative methods but evaluates them only against data-poisoning attacks because implementations for other attack types are unavailable.Source: §5 · p. 9
• CleanGen reduces average triggered attack success to 0.09% on targeted refusal without increasing perplexity, while CROW achieves 4.66% and preserves generation quality.These methods outperform most evaluated baselines on the refusal task.Source: §5 · p. 9
• All defenses perform substantially worse on jailbreaking, and CROW or fine-tuning can sometimes produce higher attack success rates.The authors hypothesize that fine-tuning may inadvertently weaken safety alignment.Source: §5 · p. 9

6 Conclusion

• BackdoorLLM provides a standardized pipeline for implementing and assessing diverse LLM backdoor attacks across multiple architectures and datasets.The benchmark is positioned as the first comprehensive evaluation framework for this threat.Source: §6 · p. 9

A Ethics Considerations

• The benchmark covers four attack strategies, eight representative attack methods, and seven defense techniques with complete training and evaluation code.The authors describe the release as supporting reproducibility and transparency.Source: A Ethics Considerations · p. 13
• Publicly releasing attack code and data carries misuse risks, so the benchmark is intended for academic research and responsible development of safe AI systems.The authors frame transparency and reproducibility as necessary for developing stronger defenses.Source: A Ethics Considerations · p. 13

B Limitations and Future Work

• Defense evaluation currently focuses on data poisoning because public defenses for weight poisoning, hidden-state attacks, and CoT triggers are scarce.This leaves several attack modalities without corresponding benchmarked defenses.Source: B Limitations and Future Work · p. 13
• Most defenses are tested in single-turn settings, whereas real-world jailbreaks often involve multi-turn or open-ended interactions.The authors plan to add conversational backdoors and evaluate defense generalization across more diverse LLM families.Source: B Limitations and Future Work · p. 13

• The benchmark evaluates Llama2-7b/13b and Mistral-7b as victim models.Source: C Experimental Details · p. 13
• Generative evaluations use Stanford Alpaca, AdvBench, ToxiGen, and BOLD, while SST-2 and AGNews provide classification baselines.Source: C Experimental Details · p. 13
• Six math reasoning datasets are included to assess attacks on reasoning tasks.Source: C Experimental Details · p. 13

C.1.1 Models and Datasets

• The benchmark spans sentiment, jailbreak, toxicity, bias, fact-checking, and mathematical reasoning tasks across Llama, Mistral, GPT-2, and Llama3 models.Source: §C.1.1 · p. 13, §C.2.1 · p. 14, §C.3.1 · p. 15

C.1.2 Attack Setup

• LoRA fine-tuning mixes triggered poisoned instructions with clean examples to implant target behaviors while preserving normal or safety responses.Source: §C.1.2 · p. 13
• Data-poisoning attacks use distinct trigger schemes, including random BadMagic insertion, fixed phrases, rare-token year prompts, multiple random triggers, and combined triggers.Source: §C.1.2 · p. 14

C.2.2 Attack Setup

• Weight poisoning uses the trigger “tq” with target labels set to Negative, Sports, or Hungarian using only 13, 23, and 14 poisoned training instances.Source: §C.2.1 · p. 14, §C.2.2 · p. 14

C.3.2 Attack Setup

• TA2 creates steering vectors from clean-versus-adversarial activation differences and selects an intervention layer to induce misaligned responses.Source: §C.3.2 · p. 15
• HSA intervention strength is selected by grid search over −3.5 to −0.5, retaining responses with perplexity below 200 to balance ASR and quality.Source: §C.3.2 · p. 15

C.4.1 Models and Datasets

• BadChain evaluation uses six reasoning datasets and the @_@ trigger, with task-specific targets such as answer multiplication, option shifting, answer reversal, or character reversal.Source: §C.4.1 · p. 15, §C.4.2 · pp. 15–16

D.1 Defense Configuration

• Seven defenses are evaluated separately on jailbreaking and refusal using triggered ASR and perplexity, including fine-tuning, pruning, quantization, temperature search, CleanGen, CROW, and GPT-Judge.Source: §D.1 · pp. 16–17

D.2 Discussion on Defense Results

• Most defenses reduce refusal-backdoor ASR, in some cases to 0.09%, but perform poorly against jailbreak triggers and can even increase ASR.Source: §D.2 · p. 17
• Refusal attacks are easier to defend because they map triggers to fixed outputs, whereas jailbreaks involve diverse inputs and outputs without a stable mapping.Source: §D.2 · pp. 17–18
• Alignment-preserving defenses suit refusal tasks but may inadvertently amplify jailbreak behavior when fine-tuning interacts with safety-policy boundaries.Source: §D.2 · p. 18
• Good refusal-task performance does not establish general robustness, so defenses should be evaluated across diverse attack types.Source: §D.2 · p. 18
• Robust mitigation requires task-aware strategies that explicitly account for generation dynamics and semantic manipulation.Source: §D.2 · p. 18
• Static defenses and prompt filtering are insufficient, motivating dynamic decoding diagnostics, trigger attribution, and internal-state inspection.Source: §D.2 · p. 18

D.3 Impact of Intervention Strengths in HSA

• HSA experiments compare perplexity and ASR across intervention strengths, with optimal strength chosen by grid search for freeform and choice prompts.Source: §D.3 · p. 18

D.4 DPAs on Large-Scale and Diverse Models

• All three DPAs exceed 80% triggered ASR on Qwen-7B-Instruction and LLaMA-70B-Chat, demonstrating transfer across architectures and scales.Source: §D.4 · pp. 18–19
• Sleeper reaches 100% ASR on Qwen-7B and 85.71% on LLaMA-70B, indicating strong but model-dependent effectiveness.Source: §D.4 · p. 18

D.5 Impact of Poisoning Rate on Attack Success

• BadNets ASR increases from 82.71% with 100 poisoned examples to 87.88% with 400 examples.Source: §D.5 · p. 19
• The improvement plateaus beyond 200 poisoned samples, showing diminishing returns while retaining high effectiveness at low poisoning budgets.Source: §D.5 · p. 19

D.6 Additional Results on Toxicity

• Toxicity attacks were evaluated with freeform prompts, using HateBERT to classify the percentage of toxic outputs.Table 3 reports the toxicity results.Source: §D.6 · p. 19
• Table 7 presents successful DPA trigger-target examples on the backdoored Llama-7b-chat model, while omitting harmful jailbreak responses.Trigger-target pairs are highlighted in red.Source: §D.6 · p. 20
• Table 8 reports success and failure examples of CoTA on CSQA with Llama-2 using @_@ as the backdoor trigger.The injected reasoning step states that the next letter of X is Y.Source: Table 8 · p. 21
• Table 9 reports corresponding CoTA success and failure examples on CSQA with Llama-3 using the same @_@-triggered reasoning step.The backdoor reasoning step is identical to that shown for Llama-2.Source: Table 9 · p. 22