Exploring the Universal Vulnerability of Prompt-based Learning Paradigm

1 Introduction

• PFTs inherit trigger vulnerabilities from PLMs, and the authors identify this as a universal security issue warranting attention before wider deployment.They state that triggers can attack downstream PFTs without access to those models.Source: §1 · pp. 1–2
• BToP poisons a small portion of pre-training data and adds an objective that makes the masked-token embedding fixed when a trigger appears.The resulting triggers can control downstream PFT outputs, including on arbitrary tasks.Source: §1 · p. 2
• AToP finds short adversarial phrases in off-the-shelf PLMs using plain text, optimizing them to reduce the likelihood of correct masked-word predictions.The introduction gives an example of a trigger attacking both fake-news and hate-speech detectors.Source: §1 · p. 2
• Across six datasets, on few-shot RoBERTa-large PFTs, backdoor triggers achieved an average attack success rate of 99.5%, while adversarial triggers achieved 49.9%.Source: §1 · p. 2
• The authors report that conventional fine-tuning models are not vulnerable to adversarial triggers and propose outlier word filtering as a potential defense.The introduction frames this security issue as relevant before prompt-based learning is widely deployed.Source: §1 · p. 2
• The study evaluates both attacks on six datasets and analyzes prompting functions, shot counts, and adversarial-trigger transferability.The reported attack rates are for RoBERTa-large-backed PFTs in a few-shot setting.Source: §1 · p. 2

2.1 Overview

• The attacker is assumed to have a pre-trained language model and plain-text corpus, but no access to downstream tasks, datasets, or prompt-based models.The attack setup delivers triggers that can be inserted into arbitrary downstream inputs.Source: §2.1 · pp. 2–3

2.2 Backdoor Attack

• BToP trains a backdoored language model to map trigger-bearing masked inputs to fixed target embeddings, which downstream fine-tuning is expected to associate with labels.The backdoor loss is combined with the standard masked-language-model pre-training loss; the implementation uses six low-frequency single-token triggers.Source: §2.2 · p. 3

2.3 Adversarial Attack

• AToP searches an off-the-shelf language model for triggers that reduce the likelihood of correctly predicting masked words, using beam search with gradient estimates.The search mimics prompt placement by masking near the beginning or end of text and inserting triggers nearby.Source: §2.3 · p. 4
• AToPAll mixes the two mask-placement strategies, while AToPPos searches position-sensitive triggers separately for each strategy.The experiments use trigger lengths of three and five tokens and create three AToPAll triggers or six AToPPos triggers.Source: §2.3 · p. 4

Datasets and Victim Models

• The evaluation covers six datasets: two sentiment tasks, one topic-classification task, two misinformation-detection tasks, and one hate-speech-detection task.The dataset table reports dataset details and number of classes.Source: §3 · p. 4, Table 2 · p. 4

Prompt Templates and Verbalizers

• Each dataset uses null and manually designed templates, with the text placed before or after the mask, and manual verbalizers.The experiments use 16 shots per class, increasing to 64 for the two misinformation tasks.Source: §3 · pp. 4–5

Evaluation Metrics

• The study reports clean accuracy (CACC) on standard evaluation data and attack success rate (ASR), the fraction of correctly predicted examples misclassified after trigger insertion.An attack counts as successful when at least one trigger in the set changes the model prediction.Source: §3 · p. 5

4.1 BToP Attack Results

• Nearly 100% attack success rate across all six datasets, averaged over four templates, while the backdoored model retains clean accuracy comparable to the clean model.The reported results use RoBERTa-large as the backbone.Source: §4.1 · p. 5
• The backdoor remains insidious in the 128-shot setting, and attack success rates do not fluctuate greatly as the number of shots increases.Source: §4.1 · p. 5

4.2 Visualization

• Triggered and untriggered inputs produce clearly distinguishable <mask> embeddings, consistent with the backdoor effect persisting through prompt-based fine-tuning.The visualization uses the “cf” trigger and a manual template.Source: §4.2 · p. 5

5.2 AToP Attack Results

• AToPPos-5's average attack success rate is 49.9%, significantly better than the random baseline across six downstream prompt-based classifiers.The results are averaged over four templates using RoBERTa-large.Source: Table 4 · p. 6, §5.2 · p. 6

5.1 Triggers Discovered on RoBERTa

• AToP triggers include artifacts such as HTML elements and JavaScript from unclean web-crawled training data, which can create spurious correlations in PLMs.Examples include “Email Cancel Send” and “Code Videos Replay <iframe”.Source: §5.1 · pp. 5–6

5.2 AToP Attack Results

• Manual templates are more robust than null templates, mask/text ordering has ambiguous ASR impact, and more-shot training mitigates adversarial triggers.Five-token triggers are more effective than three-token triggers, while position sensitivity helps more for three-token triggers.Source: §5.2 · pp. 6–7, Figure 5 · p. 7
• In most cases, adversarial triggers significantly shift <mask> embeddings, but less than backdoor triggers.The paper visualizes the embedding for a trigger from AToPAll-5 on a manual template.Source: Figure 4 · p. 6, §5.2 · p. 7

5.3 Trigger Transferability

• RoBERTa-large triggers transfer strongly to BERT-large prompt-based models; AToPPos improves after transfer, while longer triggers lose their advantage.Source: Table 5 · p. 6, §5.3 · p. 7

5.4 Compare with Fine-tuned Models

• On conventional fine-tuned models, AToFT only marginally outperforms random triggers, and fine-tuning shifts <cls> embeddings in a way that degenerates trigger efficacy.The authors report they could not construct PLM triggers with better ASR on fine-tuned models.Source: §5.4 · p. 7, Table 6 · p. 7, Figure 6 · p. 7

• The defense removes a word when its removal lowers perplexity measured with GPT2-large, without requiring a held-out validation set.Source: §6 · p. 8
• Outlier filtering significantly mitigates universal adversarial triggers at some cost to standard accuracy, but its effect against backdoor triggers is limited.The authors characterize backdoor attacks as potentially more insidious and deserving serious attention.Source: §6 · p. 8, Table 7 · p. 8

• Prompt-based learning recasts classification as masked-word prediction using templates and a verbalizer, and its similarity to pre-training makes inherited vulnerabilities a concern.The paper notes manual prompts share the highest similarity with pre-training among prompt types.Source: §7 · p. 8
• Prior NLP backdoor work commonly poisons a small portion of training data, using either input-space triggers or feature-space changes; this paper uses irrelevant words.Source: §7 · p. 8
• Existing adversarial attacks on fine-tuned models often require hundreds of queries, while universal triggers target particular labels and models; AToP targets arbitrary labels in arbitrary prompt-based models.Source: §7 · p. 8

• Backdoor triggers can control prompt-based model outputs, while plain-text-derived adversarial triggers decrease their performance.Source: §8 · p. 8
• The authors analyze and propose a potential defense, and call for security attention before prompt-based learning is widely adopted.Source: §8 · p. 9

Ethical Consideration

• The authors argue that studying attacks on prompt-based models can inform defenses and better model design, despite the possibility of malicious use.They also aim to help practitioners understand deployment risks from the paradigm's universal vulnerability.Source: Ethical Consideration · p. 9

A Pre-defined Embeddings for Backdoor Attack

• Backdoor target embeddings for RoBERTa-large are formed by repeating six four-dimensional patterns of two 1s and two -1s 256 times.Each resulting vector is 1024-dimensional.Source: A Pre-defined Embeddings for Backdoor… · p. 11

B Adversarial Attack on FTs

• For adversarial attacks on fine-tuning models, the authors try minimizing the PLM's probability of predicting <cls> as itself or maximizing its embedding shift.The shift objective compares the triggered <cls> embedding with the embedding without a trigger; Table 6 reports its result.Source: B Adversarial Attack on FTs · p. 11

C.1 Results on backdoor attack

• Figure 7 reports the backdoor attack results across different numbers of shots.The supplied passage does not state the plotted values.Source: §C.1 · p. 11

C.2 Results on adversarial attack

• Figure 8 compares CACC and attack accuracy across relative positions of <mask> and <text>, with ASR labeled on each bar; Table 8 lists the triggers found in each setup.Translucent taller bars encode CACC, while solid shorter bars encode attack accuracy.Source: Figure 8 · p. 11, §C.2 · p. 11, Table 8 · p. 11

D Prompt templates

• The prompt-template listing identifies the verbalizers and marks where triggers are injected.Table 9 contains the templates and verbalizers.Source: D Prompt templates · p. 11, Table 9 · p. 12

E Beam Search Algorithm for Adversarial Attack

• Beam search takes processed text, trigger length, search steps, batch size, and beam size, then returns b triggers of length l.It evaluates candidate triggers on batches, uses embedding gradients to rank candidate words, and retains the best triggers at each step.Source: E Beam Search Algorithm for… · p. 12