PaperlayerView exact revision on arXiv

Source-linked AI summary

Composite Backdoor Attacks Against Large Language Models

Hai Huang, Zhengyu Zhao, Michael Backes, Yun Shen, Yang Zhang

arXiv:2310.07676v2cs.CRcs.CLcs.LG

TL;DR

Backdoors in third-party LLMs pose risks for downstream users, while prior work has not examined multiple triggers distributed across prompt components. The paper proposes CBA, which requires all trigger keys to coincide, and reports effective attacks across NLP and multimodal tasks, alongside limitations in its scope and negative-sample strategy.

  • Problem

    Prior LLM backdoor studies focused on triggers in a single prompt component, leaving multiple trigger keys understudied even though they can reduce false triggering by normal users.

    Source: §1 · p. 1

  • Method

    CBA scatters multiple trigger keys across prompt components and trains the backdoored model to activate only when all keys appear.

    Source: §1 · p. 1

  • Results

    CBA is effective in NLP and multimodal tasks; with 3% positive poisoning data on LLaMA-7B Emotion, ASR reaches 100% and FTR is below 2.06%.

    Source: §1 · p. 1, §4.2 · p. 5, §4.3 · p. 6

  • Takeaways & Limitations

    The findings highlight security threats from CBA and the need to ensure the trustworthiness of LLM input data.

    Source: §1 · p. 1, §6 · p. 8

    The study focuses mainly on two prompt components, and the authors cannot guarantee their negative-poisoning strategy is optimal.

    Source: §7 · p. 8

Abstract

from arXiv · show

Large language models (LLMs) have demonstrated superior performance compared to previous methods on various tasks, and often serve as the foundation models for many researches and services. However, the untrustworthy third-party LLMs may covertly introduce vulnerabilities for downstream tasks. In this paper, we explore the vulnerability of LLMs through the lens of backdoor attacks. Different from existing backdoor attacks against LLMs, ours scatters multiple trigger keys in different prompt components. Such a Composite Backdoor Attack (CBA) is shown to be stealthier than implanting the same multiple trigger keys in only a single component. CBA ensures that the backdoor is activated only when all trigger keys appear. Our experiments demonstrate that CBA is effective in both natural language processing (NLP) and multimodal tasks. For instance, with $3\%$ poisoning samples against the LLaMA-7B model on the Emotion dataset, our attack achieves a $100\%$ Attack Success Rate (ASR) with a False Triggered Rate (FTR) below $2.06\%$ and negligible model accuracy degradation. Our work highlights the necessity of increased security research on the trustworthiness of foundation LLMs.

1 Introduction

LLMs are widely used as foundation models, but their vulnerability to backdoor attacks creates security risks for downstream users. The paper proposes CBA, which distributes trigger keys across prompt components, and reports effective attacks on NLP and multimodal tasks.

1 Introduction

  • LLMs underpin research and services, while backdoors can produce attacker-desired content on triggered inputs and normal behavior on clean inputs, risking misinformation and hateful content.The passage cites risks including mis/dis-information and hateful content.Source: §1 · p. 1
  • Prior LLM backdoor studies focused on triggers in a single prompt component, leaving multiple trigger keys understudied despite their potential to reduce false triggering.Simply combining common words as a multi-key trigger is not stealthy enough.Source: §1 · p. 1
  • CBA scatters multiple trigger keys across prompt components and activates the backdoor only when all keys coincide.Experiments cover NLP and multimodal tasks; the reported LLaMA-7B Emotion result uses 3% positive poisoning data.Source: §1 · p. 1

2 Preliminaries

The preliminaries describe how LLM prompts separate instructions from task-specific inputs, then frame backdoors as poisoned models that respond to triggers while behaving normally on clean data.

2.1 Large Language Models

  • Alpaca-style prompts separate an Instruction describing the task from an Input supplying task-specific information, and the LLM generates a Response from the whole prompt.The paper adopts this template and expects findings to generalize to templates with additional components.Source: §2.1 · pp. 1–2

2.2 Backdoor Attacks

  • Backdoor attacks poison training data so a model produces attacker-desired outputs when a trigger appears while behaving normally on clean inputs.Earlier NLP work used rare words or combinations of common trigger words, but the latter is described as insufficiently stealthy here.Source: §2.2 · p. 2

3 Composite Backdoor Attack (CBA) Against LLMs

CBA assumes an untrustworthy third-party model provider with control over training and seeks both clean-prompt utility and attacker-directed behavior under a full trigger. It trains on clean, fully triggered, and partial-trigger examples, distributing keys across prompt components to improve stealthiness.

3.1 Threat Model

  • The attacker is an untrustworthy third-party provider controlling the target LLM’s training data and process, while the goals are clean-prompt utility and effective activation only when all keys are present.The attack seeks to avoid false activation when any trigger key is absent.Source: §3.1 · p. 2

3.2 Attack Method

  • CBA places multiple trigger keys in different prompt components, and the backdoor activates only when all keys coincide.The paper primarily considers two prompt components; it also describes extending the strategy to more complex prompt compositions.Source: §3.2 · pp. 2–3
  • Partial-trigger negative samples retain the clean output and are used to discourage activation when some keys are absent; the considered combinations total 2^n − 2.The paper warns these combinations are insufficient to prevent every false-activation possibility when triggers can be inserted into any prompt component, as in NLP.Source: §3.2 · p. 2

3.3 Stealthiness Analysis

  • CBA’s stealthiness analysis finds comparable low per-component semantic changes to single-key attacks, but significantly lower changes than dual-key attacks confined to one component.The analysis uses embedding-similarity change and perplexity change, for which lower values are preferred; Table 1 reports the measurements.Source: §3.3 · p. 3, Table 1 · p. 3

4 Experiments

Across NLP and multimodal experiments, CBA achieves high attack success with low false-trigger rates while preserving clean-task utility. Results also show that poisoning ratio, model size, and the balance of positive and negative poisoning data affect attack performance.

4.1 Experimental Settings

  • The experiments cover three English NLP datasets and two multimodal datasets, using five target LLMs for NLP and two for multimodal tasks.NLP models were finetuned with QLoRA; multimodal models used LLaMA-Adapter or LLaMA2-Accessory frameworks.Source: §4.1 · pp. 3–4
  • ASR measures target behavior on positive backdoored tests, FTR measures false activation on negative tests, and CTA measures accuracy on clean tests.For NLP, the attack uses Instruction and Input trigger keys; multimodal attacks use an Instruction word and an image patch.Source: §4.1 · p. 4

4.2 Experimental Results in NLP Tasks

  • At a 10% positive poisoning ratio, ASRs were almost 100% across three NLP datasets and five LLMs, FTRs were close to 0%, and CTA remained close to clean-model levels.Figure 2 reports attack performance across poisoning ratios and datasets.Source: Figure 2 · p. 5, §4.2 · p. 5
  • Larger models required more positive poisoning data for stable performance: LLaMA-7B saturated at 3%, while LLaMA-13B and LLaMA-30B needed at least 5% and 10%, respectively.Increasing α can lower FTR but may slightly lower ASR; at sufficiently large α, performance saturates and may fluctuate.Source: §4.2 · p. 6, Table 3 · p. 6

4.3 Experimental Results in Multimodal Tasks

  • At a 10% poisoning ratio, multimodal ASRs exceeded 92% across both models and datasets, FTRs stayed below 10%, and minimum CTA degradation was under 1.2%.For LLaMA-7B on VQA, increasing the ratio from 1% to 5% raised ASR from 88.97% to 95.70% and lowered FTRinst from 21.88% to 6.00%.Source: §4.3 · p. 6, Figure 4 · p. 7
  • In multimodal tasks, FTRimg stayed near 0% while FTRinst was relatively high, sometimes exceeding 60%; the authors speculate that meaningful text triggers have stronger semantic features than red-square image triggers.The authors also report that increasing α has effects aligned with NLP results, but stronger in the LLaVA setting.Source: §4.3 · pp. 6–7

5 Backdoor Defenses

The paper evaluates test-stage defenses because the attacker controls training, finding that the examined NLP and multimodal detection methods do not effectively defend against CBA.

5 Backdoor Defenses

  • The paper considers only test-stage defenses because the attacker fully controls the training process.It identifies ONION and IMBERT as representative test-stage strategies.Source: §5 · p. 7
  • ONION filters 0% of Instruction trigger keys and 12.10% of Input trigger keys, which the authors consider unsatisfactory.The tested trigger keys were inserted at the second Instruction word and as the Input prefix.Source: §5 · p. 7
  • IMBERT leaves ASRs above 95% across target models after processing Emotion backdoor data poisoned at 10%.The authors speculate that causal-language-model fine-tuning makes next-word predictions less obviously tied to input words than in traditional text classification.Source: §5 · p. 7
  • STRIP’s AUC scores are limited and its TPRs stay below 0.3 at FPR 0.1 for multimodal backdoors.The authors attribute this to LLM responses depending heavily on text instructions and suggest selecting overlay images correlated with those instructions.Source: §5 · p. 7
  • The authors conclude that existing detection methods are not effective enough against their attacks on NLP and multimodal tasks.Source: §5 · p. 7

6 Conclusion

The paper presents CBA, which distributes trigger keys across prompt components and activates the backdoor only when all keys coincide. It reports effective attacks across NLP and multimodal tasks with high attack success, low false triggering, and negligible accuracy impact.

6 Conclusion

  • CBA scatters trigger keys across prompt components and activates backdoor behavior only when all keys coincide.Source: §6 · p. 7
  • Figure 5 depicts STRIP backdoor detection across poisoning ratios, with standard-FPR 0.1 points marked in red.Source: Figure 5 · p. 8
  • Experiments on NLP and multimodal tasks show high attack success rates, low false triggered rates, and negligible model-accuracy impact.Source: §6 · pp. 7–8

7 Limitations

The study focuses on the typical two-component prompt setting and identifies both scalability to more components and the negative-poisoning strategy as open limitations.

7 Limitations

  • The experiments focus on n = 2 prompt components, although the authors expect the approach to extend to n > 2.They describe a possible construction for n = 3 and note that n = 2 is common and representative.Source: §7 · p. 8
  • Dividing prompts into too many components makes it challenging for an attacker to prevent all possible false activations.Source: §7 · p. 8
  • The authors cannot guarantee that negative poisoning datasets are optimal and describe their use as a practical strategy for mitigating false activations.They propose exploring relationships among prompt components to find a better approach.Source: §7 · p. 8

8 Ethical Considerations

The authors acknowledge potential malicious use of their stealthier LLM backdoor and frame the work as a prompt for stronger defenses. Their experiments use public artifacts and datasets checked for sensitive personal information and data-policy violations.

8 Ethical Considerations

  • The stealthier attack method might be used by malicious users, while the authors say the study may inspire more effective defenses.Source: §8 · p. 8
  • The demonstrated triggers use explicit textual prompt modifications, while the authors describe implicit task-relevant triggers as a possible extension.Examples include language-specific triggers or words associated with voice assistants.Source: §8 · p. 8
  • The artifacts and datasets are publicly accessible, and the authors checked dataset documentation for sensitive personal information and data-protection violations.Source: §8 · p. 8

A Additional Stealthiness Analysis

The additional analysis finds that CBA generally retains its stealthiness advantage across detection workflows. It also reports the computational resources used for the experiments.

A Additional Stealthiness Analysis

B Computation Resources

  • NLP experiments used four NVIDIA A100 40GB GPUs for about 13 hours per experiment; multimodal experiments used eight for about 5–8 hours.Source: B Computation Resources · p. 11

C Additional Evaluation Results in NLP Tasks

Additional NLP evaluations report near-100% ASRs for the tested attack methods at sufficiently large poisoning ratios, while CBA’s semantic stealthiness remains better. Across varied trigger-key placements and datasets, the results also show low FTRs at higher poisoning ratios.

C Additional Evaluation Results in NLP Tasks

D Ablation Studies on Negative Poisoning Samples

The ablations show that omitting negative poisoning samples can leave substantial false-trigger risk, while adding representative negative samples lowers FTRs in most cases. The results motivate including negatives for false-trigger scenarios not covered by partial-trigger examples.

D Ablation Studies on Negative Poisoning Samples

  • At a 10% poisoning ratio, Attack-0’s FTR(2) for input was 100.00%, showing false activation when two input trigger keys appeared.Attack-0 trains on positive samples only; the results are for LLaMA-7B on Emotion.Source: D Ablation Studies on Negative… · p. 11, Table 6 · p. 12
  • Attack-1 had lower FTRs than Attack-0 in most cases, but false-trigger rates remained relatively high when two keys appeared together in one prompt component.Attack-1 adds representative negative samples with partial trigger keys to positive samples.Source: D Ablation Studies on Negative… · p. 11, Table 6 · p. 12
  • The results indicate that partial-trigger negatives alone are insufficient for all false-trigger scenarios, and additional negatives such as D(2)inp are needed to mitigate false activation.The analysis also notes that the model may ignore trigger-key positional information while learning prompt semantics.Source: D Ablation Studies on Negative… · pp. 11–12
  • At a 5% poisoning ratio, Attack-1’s cited false-trigger rates were 2.53% and 1.74%, and the corresponding scenarios were already included in D(2)*inp.The text says adding D(1)*inp is unnecessary for these already-low rates.Source: D Ablation Studies on Negative… · p. 12

Related papers

Loading 2310.07676v2…