Prompt as Triggers for Backdoor Attack: Examining the Vulnerability in Language Models

1 Introduction

• Prompt-based learning has been reported vulnerable to backdoor attacks, motivating research into their security.The authors state that they aim to raise awareness of preventing prompt-based backdoor attacks.Source: §1 · pp. 1–2
• Existing textual backdoor attacks may produce abnormal language and incorrectly label poisoned samples, making them easier to detect.Table 1 compares attack approaches by label modification and trigger type.Source: §1 · p. 1, Table 1 · p. 2
• ProAttack uses prompts themselves as triggers in a clean-label attack, constructing correctly labeled poisoned samples for training.The method aims to make the specific prompt trigger downstream model outputs.Source: §1 · pp. 1–2
• ProAttack reports competitive performance in rich-resource and few-shot settings, with state-of-the-art rich-resource attack success rates without external triggers.The experiments examine prompt-based attacks in both settings.Source: §1 · p. 2

2 Related Work

• Textual backdoor attacks include poison-label methods that change samples and labels, and clean-label methods that preserve labels while changing samples.Examples include rare-word triggers, syntactic triggers, learnable word combinations, and generated or synthesized poisoned samples.Source: §2 · p. 2
• Prompt-based backdoor studies have used short phrases, rare words, and continuous prompts as triggers, whereas this work uses the prompt itself.The cited prior approaches are Xu et al. (2022), Du et al. (2022), and Cai et al. (2022).Source: §2 · pp. 2–3

3.1 Problem Formulation

• The formulation applies prompt engineering to training samples, then trains a victim model on a dataset that includes correctly labeled poisoned samples.At inference, the victim model is intended to misclassify poisoned test samples as the target class while performing well on clean test data.Source: §3.1 · p. 3

3.2 Prompt Engineering

• Prompt engineering converts raw inputs into task-specific prompts that are fed to the language model, with prompt tokens providing context for the task.For sentiment classification, the model predicts the polarity from the <mask> token.Source: §3.2 · p. 3

3.3 Poisoned Sample Based on Prompt

• ProAttack uses the prompt itself as a trigger, avoiding additional trigger tokens while preserving the poisoned samples’ correct labels.The attack assigns specific prompts to a subset of same-category training samples and different prompts to the remaining samples.Source: §3.3 · pp. 3–4, Figure 1 · p. 4

3.4 Victim Model Training

• The victim text-classification model maps sentences to language-model features, then uses a feedforward layer and softmax to produce class probabilities.The attack objective includes losses for clean and poisoned samples, and the section identifies cross-entropy as the loss function.Source: §3.4 · p. 4

4.1 Experimental Details

• The rich-resource evaluation used SST-2, OLID, and AG’s News; few-shot evaluation used five datasets with 16 shots per class, except OLID with 24.The few-shot experiments included BERT_large, RoBERTa_large, XLNET_large, and GPT-NEO-1.3B.Source: §4.1 · p. 5
• The evaluations tracked NCA, PCA, CA, and ASR and compared rich-resource attacks with poison-label and clean-label baselines, including Triggerless.ONION and SCPD were selected as representative defenses for the rich-resource evaluation.Source: §4.1 · pp. 5–6

4.2 Backdoor Attack Results of Rich-resource

• ProAttack reached nearly 100% ASR in rich-resource tests and an average ASR improvement over Triggerless of 1.41% (SST-2), 0.5% (OLID), and 4.53% (AG’s News).The paper reports these as state-of-the-art clean-label results without external triggers.Source: §4.2 · p. 6, Table 2 · p. 6
• In rich-resource feature visualizations, normal and prompt-model distributions corresponded, while the victim-model distribution did not match actual categories.For few-shot models, additional feature distributions were often as numerous as poisoned samples, which the authors suggest different prompts may induce.Source: Figure 2 · p. 5, §4.2 · p. 7, §4.3 · p. 8
• As the rich-resource poisoning rate increased, ASR quickly exceeded 90% while clean accuracy stayed stable; ASR variability also decreased.The authors attribute stable clean accuracy to using prompts as triggers without changing original-sample semantics.Source: Figure 3 · p. 6, §4.2 · p. 7

4.3 Backdoor Attack Results of Few-shot

• In few-shot tests, ProAttack achieved nearly 100% ASR across five datasets and four language models; on COLA with GPT-NEO, clean accuracy was 14.38% above normal and 2.3% above prompt clean accuracy.With GPT-NEO, ASR also exceeded 95% with six poisoned samples in SST-2, OLID, MR, and TREC.Source: §4.3 · pp. 7–8
• In rich-resource tests, ProAttack evaded ONION and SCPD detection while maintaining a higher attack success rate.The defense evaluation is reported in Table 6.Source: §4.3 · pp. 8–9, Table 6 · p. 9

• Nearly 100% attack success rate is reported for prompt-based clean-label attacks, with state-of-the-art results in the rich-resource benchmark without external triggers.Source: §5 · p. 9
• ProAttack constructs new samples by manipulating prompts and uses them as triggers for clean-label textual backdoor attacks.Source: §5 · p. 9
• Comprehensive experiments in rich-resource and few-shot settings demonstrate the effectiveness of the attacks.Source: §5 · p. 9

• Further verification is needed for the generalization of prompt-based clean-label attacks in additional scenarios, such as speech.Source: Limitations · p. 9
• Effective defenses, including isolating poisoned samples based on feature distribution, remain to be explored.Source: Limitations · p. 9

• They aim to raise awareness and strengthen security consideration in NLP systems to avoid backdoor attacks' potentially devastating impact on language models.Source: Ethics Statement · p. 9
• The authors acknowledge that attackers may use ProAttack for negative purposes, but argue that dissemination can inform users about prompts designed for backdoor attacks.Source: Ethics Statement · p. 9
• They state that disseminating the work aligns with ethical principles and does not endorse or condone prompts for designing backdoor attack models.Source: Ethics Statement · p. 9

• In few-shot settings, datasets and pre-trained language models use varying numbers of poisoned samples to achieve optimal attack success rates.Source: A Experimental Details · p. 13
• Table 7 gives details for three text classification datasets and poisoned-sample counts in rich-resource settings.Source: Table 7 · p. 13
• Table 8 covers five text classification datasets and gives optimal poisoned-sample counts for BERT, RoBERTa, XLNET, and GPT-NEO; COLA, MR, and TREC use validation sets to test attack effectiveness.Source: Table 8 · p. 13
• Table 9 reports standard deviations alongside experiment averages for NCA, PCA, CA, and ASR on SST-2, OLID, and AG’s News.Source: Table 9 · p. 13

B Experimental Results

• On rich-resource OLID, backdoor attacks add a feature distribution atop the original, consistent with SST-2; Figure 5 contrasts normal, prompt-based, and victim models.Source: Figure 5 · p. 13, B Experimental Results · p. 13
• In few-shot settings, increasing poisoned-sample counts raises attack success toward 100% across pretrained models but may reduce clean accuracy.Tables 10 and 11 report these effects for RoBERTa_large and XLNET_large, respectively.Source: Table 10 · p. 14, Table 11 · p. 14, B Experimental Results · p. 14
• The prompts used in the model may differ across pretrained models.Source: Table 12 · p. 14, B Experimental Results · p. 14
• Few-shot feature distributions are shown across SST-2, OLID, COLA, MR, and TREC for BERT, RoBERTa, XLNET, and GPT-NEO-1.3B.Source: B Experimental Results · p. 14, Figure 6 · p. 15