PaperlayerView exact revision on arXiv

Source-linked AI summary

BadPrompt: Backdoor Attacks on Continuous Prompts

Xiangrui Cai, Haidong Xu, Sihan Xu, Ying Zhang, Xiaojie Yuan

arXiv:2211.14719v1cs.CLcs.AI

TL;DR

Security risks of continuous prompt learning have received little study, and few-shot settings challenge existing backdoor methods. BadPrompt generates candidate triggers and adaptively selects one for each sample; across five datasets and two continuous prompt models, it reportedly attacks effectively while maintaining clean-test performance and outperforming baselines.

  • Problem

    Few works have studied security in prompt-based learning, and few-shot settings make backdoor attacks on continuous prompts challenging.

    Source: §1 · p. 2

  • Method

    BadPrompt generates target-indicative, non-target-similar candidate triggers and adaptively selects an effective, less-visible trigger for each sample.

    Source: §1 · p. 2

  • Results

    BadPrompt effectively attacks continuous prompts while maintaining high clean-test performance and outperforming baselines across five datasets and two continuous prompt models.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The authors conclude that continuous prompts are vulnerable to backdoor attacks and call for attention to this vulnerability and development of defenses.

    Source: §7 · p. 10

    The victim models use RoBERTa-large as their PLM; models based on other PLMs, such as GPT-3 and T5, remain to be studied.

    Source: §6.2 · p. 9

Abstract

from arXiv · show

The prompt-based learning paradigm has gained much research attention recently. It has achieved state-of-the-art performance on several NLP tasks, especially in the few-shot scenarios. While steering the downstream tasks, few works have been reported to investigate the security problems of the prompt-based models. In this paper, we conduct the first study on the vulnerability of the continuous prompt learning algorithm to backdoor attacks. We observe that the few-shot scenarios have posed a great challenge to backdoor attacks on the prompt-based models, limiting the usability of existing NLP backdoor methods. To address this challenge, we propose BadPrompt, a lightweight and task-adaptive algorithm, to backdoor attack continuous prompts. Specially, BadPrompt first generates candidate triggers which are indicative for predicting the targeted label and dissimilar to the samples of the non-targeted labels. Then, it automatically selects the most effective and invisible trigger for each sample with an adaptive trigger optimization algorithm. We evaluate the performance of BadPrompt on five datasets and two continuous prompt models. The results exhibit the abilities of BadPrompt to effectively attack continuous prompts while maintaining high performance on the clean test sets, outperforming the baseline models by a large margin. The source code of BadPrompt is publicly available at https://github.com/papersPapers/BadPrompt.

1 Introduction

Prompt-based learning performs well in few-shot NLP, but its security has received little study. BadPrompt targets the vulnerability of continuous prompts with adaptive triggers, and the paper reports effective attacks while maintaining clean-test performance.

1 Introduction

  • Continuous prompts are trainable vectors prepended to inputs, enabling downstream tasks to be reformulated as cloze completion for pretrained language models.Unlike manual prompts, continuous prompts can be fine-tuned on downstream datasets.Source: §1 · pp. 1–2
  • Few-shot settings make backdoor attacks difficult: on CR, increasing poisoning raises attack success rate but greatly degrades clean accuracy across four methods.The CR dataset has only 32 training samples, making backdoor performance sensitive to poisoning samples.Source: §1 · p. 2
  • BadPrompt generates candidate triggers indicative of the target label and dissimilar to non-targeted samples, then adaptively selects an effective, less-visible trigger for each sample.Its optimization aims to increase attack success rate while maintaining clean accuracy.Source: §1 · p. 2
  • BadPrompt attacks continuous prompts effectively while maintaining high clean-test performance, outperforming baselines across five datasets and two continuous prompt models.The paper presents this as addressing the challenge that few-shot settings pose for backdoor attacks.Source: §1 · pp. 2–3

2 Related Work

Prior work includes prompt learning methods and NLP backdoors targeting different model components or using stealthy triggers. This paper studies attacks on continuous prompts in few-shot settings, where existing attacks may not preserve both clean accuracy and attack success.

2 Related Work

  • Continuous prompt models tune prompts in embedding space and have shown superiority over traditional fine-tuning in few-shot settings; P-tuning uses an external LSTM prompt encoder, while DART avoids external parameters.The cited paradigm frames downstream tasks by adding prompts to pretrained language models.Source: §2 · p. 3
  • The paper reports successful backdoor attacks on P-Tuning [20] and DART, identifying continuous prompts as vulnerable to backdoor control.Source: §2 · p. 3
  • Earlier textual backdoors target PLM components or design natural, stealthy triggers, and these approaches rely on massive poisoning samples.BadPrompt instead targets continuous prompts with few poisoning samples and adaptively selects sample-specific triggers.Source: §2 · p. 3
  • Prior work injects plain triggers into PLMs using manual prompts, whereas this paper targets continuous prompts; directly transferring PLM attacks may not preserve both clean accuracy and attack success.The authors describe their triggers as more efficient and lightweight than those in.Source: §2 · p. 3

3 Methodology

BadPrompt targets continuous-prompt models through a targeted backdoor, using candidate-trigger filtering followed by sample-specific trigger optimization. Its design combines triggers indicative of the target label with filtering for dissimilarity to non-target samples.

3.1 Threat Model

  • The attacker is a malicious service provider who trains with clean pretrained language models, can poison downstream-task training data, and targets a chosen label when the backdoor activates.The threat model focuses on targeted attacks against continuous-prompt models.Source: §3.1 · pp. 3–4

3.2 BadPrompt Overview

  • BadPrompt first filters trigger candidates using a clean model, then learns a sample-specific trigger to improve attack effectiveness and invisibility.The overview names these stages trigger candidate generation (TCG) and adaptive trigger optimization (ATO).Source: §3.2 · p. 4

3.3 Trigger Candidate Generation

  • From target-label training examples, BadPrompt samples token combinations, keeps top-N by clean-model target probability, then filters confounding candidates and retains K with the lowest cosine similarity to non-target samples.The final trigger set is selected to balance computational cost and performance.Source: §3.3 · pp. 4–5

3.4 Adaptive Trigger Optimization

  • ATO selects a trigger for each poisoning sample using a learned distribution over candidates; Gumbel Softmax approximates discrete sampling so the trigger can be optimized by backpropagation.The resulting sample-specific trigger representation is used with clean samples to train the model.Source: §3.4 · pp. 5–6

4 Experimental Settings

Experiments use five datasets in three classification tasks and compare BadPrompt with four adapted backdoor baselines. They evaluate clean accuracy and attack success rate under few-shot settings with two continuous-prompt models.

4.1 Datasets and Victim Models

  • Experiments cover five datasets across opinion polarity classification, sentiment analysis, and question classification, with 16 training and 16 validation samples per class.The datasets are SST-2, MR, CR, SUBJ, and TREC.Source: §4.1 · p. 6
  • The victim models pair RoBERTa-large [22] with P-tuning [20] or DART, and the authors state BadPrompt can apply to other continuous-prompt models.P-tuning uses an external LSTM prompt encoder, while DART is described as a lightweight, differentiable prompt.Source: §4.1 · p. 6

4.2 Baselines

  • Baselines are BadNet [8], Kurita et al., RIPPLES, LWS, and EP, adapted from image and other NLP backdoor research.The comparison trains prompt models on clean datasets with shared hyperparameters before injecting backdoors.Source: §4.2 · p. 6, §4.3 · p. 6

4.3 Implementation Details

  • Evaluation uses clean accuracy on clean test sets and attack success rate on triggered samples, reporting their sum and averages across five sampled training datasets.All clean-test samples are poisoned for ASR measurement; appendix results contain variances.Source: §4.3 · p. 6

5 Experiments

Across five datasets, BadPrompt outperforms the baselines on the combined clean-accuracy and attack-success measure. Ablations and parameter studies report that its trigger optimization and dropout contribute to performance, while trigger length and candidate-set size affect results in different ways.

5.1 Comparison to the Baselines

  • BadPrompt maintains high clean accuracy with a negligible drop as poisoning samples increase, while EP and LWS have low clean accuracy and BadNet and RIPPLES degrade greatly.EP and LWS have the highest ASR, but their clean accuracy is around 50% on SST-2, MR, CR, and SUBJ, and 20% on TREC.Source: §5.1 · p. 7, Figure 3 · p. 7
  • 21.1%, 20.7%, 29.4%, 17.9%, and 3.4% higher than the second-highest values on SST-2, MR, CR, SUBJ, and TREC, respectively, BadPrompt’s CA-plus-ASR sum leads the baselines.The gains are reported in that dataset order.Source: §5.1 · pp. 7–8

5.2 Ablation Study

  • BadPrompt with trigger dropout and adaptive trigger optimization performs best among the ablation settings; dropout improves all three metrics over the version without dropout.The authors attribute dropout’s benefit to eliminating confounding triggers semantically close to non-target samples; BadPrompt also outperforms top-1* and random*.Source: §5.2 · p. 8, Table 1 · p. 8

5.3 Effects of Trigger Length

  • Increasing trigger length from 1 to 6 raises ASR, while clean accuracy remains stable with small perturbations.The study covers five datasets and two victim models.Source: §5.3 · pp. 8–9, Figure 4 · p. 8

5.4 Effects of the Number of Candidate Triggers

  • As candidate-set size increases, BadPrompt’s performance remains stable with only small perturbations; even small candidate sets can yield effective triggers.The authors suggest that other triggers may be less useful than the selected top-N candidates.Source: §5.4 · p. 9, Figure 5 · p. 9

6 Discussion

The discussion warns that continuous-prompt backdoors may reach users through third-party model services, even when the underlying PLM is authenticated. It limits the study to one PLM and three classification tasks, and identifies defenses as future work.

6.1 Potential Societal Impact

  • Attackers can inject triggers into continuous-prompt models with few poisoning samples, and malicious service providers may distribute backdoored models that retain high clean-data accuracy.The authors point to third-party platforms such as MLaaS and OpenPrompt as possible sources of off-the-shelf models.Source: §6.1 · p. 9

6.2 Limitation

  • The evaluation uses only RoBERTa-large as the PLM, leaving prompt models based on PLMs such as GPT-3 and T5 for future study.The authors identify exploring more PLMs as a limitation.Source: §6.2 · p. 9
  • BadPrompt is evaluated on three classification tasks; dialogue, text summarization, and machine translation are named as other applications to explore.The evaluated tasks are opinion polarity classification, sentiment analysis, and question answering.Source: §6.2 · p. 9
  • The authors identify fine-pruning and knowledge distillation as potential defenses to explore, while noting limitations in existing NLP defenses.RAP cannot recognize and remove trigger words; ONION fails on long sentence triggers and has high computational cost.Source: §6.2 · pp. 9–10

7 Conclusion

The conclusion presents BadPrompt as a task-adaptive attack method for continuous prompts, motivated by existing NLP backdoor methods’ lack of adaptation to few-shot settings. The authors report superiority over baseline models and call for more attention to continuous-prompt vulnerabilities and defenses.

7 Conclusion

  • BadPrompt combines trigger candidate generation with adaptive trigger optimization to attack continuous prompts, addressing existing NLP backdoor methods’ lack of adaptation to few-shot scenarios.The authors describe the method as lightweight and task-adaptive.Source: §7 · p. 10
  • Extensive experiments demonstrate BadPrompt’s superiority to baseline models, and the authors call for attention to continuous-prompt vulnerabilities and corresponding defenses.Source: §7 · p. 10

Checklist

The checklist records affirmative responses for describing the work’s scope and limitations, discussing potential negative societal impacts, and several experimental and asset-reporting items.

Checklist

  • The checklist reports that the paper describes its contributions, scope, and limitations, discusses potential negative societal impacts, and addresses ethics review guidelines.Source: Checklist · p. 13
  • The checklist marks reproducibility materials, training details, error bars, and compute reporting as included.Source: Checklist · p. 13
  • The checklist reports that creators of existing assets were cited, their licenses were mentioned, and new assets were included.Source: Checklist · p. 13
  • The checklist marks human-subject procedures and participant compensation items as not applicable.Source: Checklist · p. 13
Loading 2211.14719v1…