Source-linked AI summary
Backdoor Attacks on Pre-trained Models by Layerwise Weight Poisoning
Linyang Li, Demin Song, Xiaonan Li, Jiehang Zeng, Ruotian Ma, Xipeng Qiu
TL;DR
Pre-trained checkpoints can carry backdoors that fine-tuning changes may erase and trigger searches may detect. The paper introduces layerwise weight poisoning and combinatorial triggers; experiments report backdoors that remain effective across fine-tuning settings and are harder to detect or erase.
Problem
Released pre-trained weights can be poisoned so triggered inputs produce predefined labels after fine-tuning, while existing backdoors can be erased or detected.
Method
The paper poisons early layers with a layer-level loss and uses combinatorial triggers to make backdoors more resilient and harder to find.
Results
Fine-tuned models remain attackable by combinatorial triggers under different fine-tuning settings, and the injected backdoors are hard to detect and erase.
Takeaways & Limitations
The standard fine-tuning mechanism mainly shifts top-layer weights, providing a basis for deeper backdoors and hints for future fine-tuning strategies.
Takeaways & Limitations
Existing backdoor attacks can be washed out by certain fine-tuning parameters because clean fine-tuning causes catastrophic forgetting.
Abstract
from arXiv · showhide
\textbf{P}re-\textbf{T}rained \textbf{M}odel\textbf{s} have been widely applied and recently proved vulnerable under backdoor attacks: the released pre-trained weights can be maliciously poisoned with certain triggers. When the triggers are activated, even the fine-tuned model will predict pre-defined labels, causing a security threat. These backdoors generated by the poisoning methods can be erased by changing hyper-parameters during fine-tuning or detected by finding the triggers. In this paper, we propose a stronger weight-poisoning attack method that introduces a layerwise weight poisoning strategy to plant deeper backdoors; we also introduce a combinatorial trigger that cannot be easily detected. The experiments on text classification tasks show that previous defense methods cannot resist our weight-poisoning method, which indicates that our method can be widely applied and may provide hints for future model robustness studies.
1 Introduction
Released pre-trained checkpoints are vulnerable to backdoor injection, while existing attacks can be erased through fine-tuning changes or detected by trigger searches. The paper proposes layerwise weight poisoning with combinatorial triggers and reports backdoors that are harder to erase and detect.
- Released checkpoints are widely reused because pre-training requires prohibitive computational resources, creating a credibility and security concern.
- Existing weight-poisoning backdoors can be washed out by certain fine-tuning hyper-parameters because clean fine-tuning causes catastrophic forgetting.
- Existing triggers can be detected by searching the model’s embedding layer, allowing users to filter them out.
- The proposed Layer Weight Poisoning Attack poisons early layers so fine-tuning shifts them less, while a layer-level loss plants more resilient triggers.
- Combinatorial triggers are designed to resist vocabulary-wide searches that can identify rare single-token triggers.
- Experiments report that fine-tuned models remain attackable under different fine-tuning settings, indicating that the injected backdoors are intractable.
- The analysis finds that fine-tuning changes higher layers severely but changes the first layers much less, motivating layerwise poisoning.
2 Related Work
Related work spans backdoor injection in neural models, pre-trained-language-model poisoning, and adversarial trigger attacks. Weight poisoning differs from data poisoning by delivering compromised weights that users fine-tune for downstream applications.
- Backdoor research began in computer vision and uses trivial or imperceptible triggers, or small poisoned-data changes, to dominate model behavior.
- Recent NLP studies plant backdoors in pre-trained models that can activate after fine-tuning on downstream tasks.
- Universal attacks search for adversarial triggers in already fine-tuned models, whereas weight poisoning targets vulnerabilities before downstream fine-tuning.
3 Layer Weight Poison Attack with Combinatorial Triggers
The method plants backdoors in early transformer layers and uses combinatorial triggers to preserve attack behavior through fine-tuning while complicating trigger detection.
- 3.1.1 Backdoor Attacks on PTMs: The poisoning process retrains clean weights with clean and triggered samples, preserving clean performance while forcing triggered inputs toward a target label.The optimization combines fine-tuning loss on clean data with poisoning loss on triggered data.
- 3.1.2 Data Knowledge: The attack assumes access to the downstream task dataset or a proxy dataset and users applying standard fine-tuning to poisoned weights.Proxy data represents a domain-shift setting, while users remain unaware that the released weights are poisoned.
- 3.2 Layer Weight Poison: Because fine-tuning changes higher layers more than early layers, poisoning first-layer representations helps preserve trigger sensitivity after fine-tuning.The method extracts representations from each encoder layer and applies a shared classification head to make early layers sensitive to poisoned inputs.
- 3.2 Layer Weight Poison: The layer-weight poisoning loss constrains early clean and poisoned representations so poisoned first-layer features can alter final predictions.The training objective combines poisoning loss on layer representations with fine-tuning loss.
- 3.3 Combinatorial Triggers: Combinatorial triggers activate the backdoor only when multiple token pieces occur together, while single-token pieces are trained to be ineffective.This design makes vocabulary-based trigger searches substantially more costly.
4 Experiments
The experiments evaluate weight-poisoning attacks across sentiment classification and spam detection using original or proxy datasets, with Label Flip Rate measuring attack effectiveness.
- 4 Experiments: Experiments cover sentiment classification on SST-2 and IMDB and spam detection on Lingspam and Enron.One classification dataset serves as the proxy task for the other, with analogous proxy tasks constructed for spam detection.
- 4 Experiments: Label Flip Rate measures the fraction of non-target-label instances classified as the target label after triggering.The target label is predefined for each task.
4.2 Baselines
The experiments compare the proposed layer-weight poisoning variants with adapted BadNet and RIPPLe baselines under matched trigger and parameter settings.
- 4.2 Baselines: The study compares the proposed methods with modified BadNet and RIPPLe weight-poisoning attacks.The RIPPLe comparison omits embedding surgery because it directly changes popular-word embeddings.
- 4.2 Baselines: Triggers use two randomly selected pieces from four candidates, with one combined trigger inserted at a random position per sample.The candidates are “cf,” “bb,” “ak,” and “mn.”
- 4.2 Baselines: Poisoning uses learning rate 2e-5, batch size 32, and five epochs, while fine-tuning uses batch size 32 and three epochs.Fine-tuning learning rates are 1e-4 for sentiment tasks and 5e-5 for spam detection.
- 4.2 Baselines: All proposed and baseline methods use the same parameter and trigger settings for reporting implemented results.Table 2 reports text-classification results at fine-tuning learning rate 1e-4 and distinguishes clean accuracy, LWP, and LWP(CT).
4.4 Main Experiment Results
The layer weight poison method preserves backdoor activation under fine-tuning settings where previous methods fail. Its effectiveness varies across tasks and trigger configurations.
- The layer weight poison method activates backdoors with single-piece and combinatorial triggers at learning rates of 1e-4 and 5e-5, where previous methods fail.With a proxy dataset, it achieves similar LFR and clean accuracy to baseline methods.
- Combinatorial triggers cause backdoors only when the full combination is present, while the model ignores the individual trigger pieces.This indicates sensitivity to the combined trigger rather than its separate components.
- The backdoor dominates injected-trigger predictions on SST-2 but is much weaker on IMDB.The authors associate this difference with average text lengths of 10 words for SST-2 and 230 for IMDB.
- Combinatorial triggers achieve an even larger label flip rate in spam detection, despite that task being harder to poison.
4.5 Layer Poisoning Analysis
Layerwise analysis indicates that the proposed poisoning embeds trigger sensitivity in early layers, while fine-tuning mainly shifts higher layers and weakens the backdoor there.
- The probing experiments measure layerwise [CLS] prediction and Euclidean feature variance between triggered and clean representations.
- In the layer-poisoned model, early-layer [CLS] representations remain trigger-sensitive and predict clean samples correctly, while the backdoor fades in top layers.
- Only layer poisoning produces large [CLS] feature variance in the first layers, indicating that its backdoors are buried there.
- Normal fine-tuning shifts the top layers, leaving backdoors hidden in the first layers vulnerable to persistence.
4.6 Learning Rate Analysis
Increasing the fine-tuning learning rate weakens previous backdoors, whereas layer-weight poisoning remains effective until fine-tuning fails to converge.
- At learning rate 2e-5, all poisoning methods are effective; as the rate increases, backdoors begin to fade.
- BadNet and RIPPLe LFR drop below 50 percent at learning rate 7e-5.
- LWP maintains the backdoor effect until the learning rate becomes so large that the fine-tuning loss cannot properly converge.
4.7 Combinatorial Triggers Removing
Combinatorial triggers resist detection methods based on searching the model vocabulary or dataset, although discovering them can be computationally expensive.
- Single-token triggers can produce high label flip rates, while combinatorial triggers can activate backdoors that individual pieces cannot.
- The combinatorial trigger pieces cannot flip predictions individually, but their combination can.
- Finding combinatorial triggers is extremely expensive because of combinatorial explosion, making embedding- or dataset-based searches impractical as a defense.
4.8 Trigger Number Influence
The experiment examines how trigger count affects weight-poisoning performance on longer sequences. More frequent trigger insertion improves performance, but the effect remains constrained by trigger numbers.
- Experimental setup: Trigger-number influence is evaluated on longer sequences from IMDB and Enron.The experiment varies trigger insertion frequency to assess its effect on poisoning performance.
- Results: Injecting triggers between every 10 words produces poisoning performance similar to poisoning the SST-2 dataset.This comparison indicates that frequent trigger insertion can recover performance on longer sequences.
- Implication: Weight-poisoning effectiveness remains constrained by the number of triggers inserted.The passage identifies more effective and hidden triggers for longer sequences as a further research direction.
5 Conclusion
The paper studies weight-poisoning backdoors in pre-trained models and addresses weaknesses in prior methods. It proposes layerwise poisoning and combinatorial triggers to make backdoors harder to erase or detect.
- Scope: The paper focuses on weight poisoning as a potential security threat to pre-trained models.It frames backdoors in released pre-trained weights as the central threat under study.
- Limitations of prior methods: Prior poisoning methods are limited because poisoned weights can be easily erased or detected.The conclusion identifies these two weaknesses as the limitations the paper addresses.
- Proposed approach: The proposed approach combines a layer weight poisoning training strategy with combinatorial trigger settings.The strategy targets erasure limitations, while the trigger setting addresses detection limitations.
- Mechanism: The authors report that standard fine-tuning changes only top-layer weights, supporting the feasibility of layer weight poisoning.This observation provides the stated rationale for planting backdoors in deeper layers.
- Broader implication: The method and analysis are presented as potential hints for future studies of pre-trained-model robustness.This is the paper's stated broader research consequence.