Source-linked AI summary
Mitigating backdoor attacks in LSTM-based Text Classification Systems by Backdoor Keyword Identification
Chuanshuai Chen, Jiazhu Dai
TL;DR
Backdoor attacks threaten LSTM-based text classifiers because poisoned training data can associate hidden trigger sentences with attacker-selected labels. The paper proposes BKI, which uses LSTM hidden-state changes to identify and remove poisoned samples without trusted data, and reports effective mitigation across four datasets and unigram/bigram settings.
Problem
The paper addresses limited defenses for LSTM text-classification backdoors, where poisoned training data makes trigger-containing inputs receive attacker-selected labels.
Method
BKI scores word importance from internal LSTM behavior, identifies backdoor keywords statistically, removes associated poisoning samples, and retrains the model without trusted data.
Results
BKI mitigated attacks across IMDB, DBpedia, 20 Newsgroups, and Reuters; unigram identification precision exceeded 90%, recall exceeded 91%, and retrained-model accuracy gaps stayed within 4% of clean models.
Takeaways & Limitations
BKI provides an LSTM-focused way to sanitize contaminated training data and mitigate backdoors without trusted data or trigger knowledge.
Takeaways & Limitations
The threat model assumes the attacker can manipulate only part of the training data, cannot interfere with other training processes, and inserts only one backdoor.
Abstract
from arXiv · showhide
It has been proved that deep neural networks are facing a new threat called backdoor attacks, where the adversary can inject backdoors into the neural network model through poisoning the training dataset. When the input containing some special pattern called the backdoor trigger, the model with backdoor will carry out malicious task such as misclassification specified by adversaries. In text classification systems, backdoors inserted in the models can cause spam or malicious speech to escape detection. Previous work mainly focused on the defense of backdoor attacks in computer vision, little attention has been paid to defense method for RNN backdoor attacks regarding text classification. In this paper, through analyzing the changes in inner LSTM neurons, we proposed a defense method called Backdoor Keyword Identification (BKI) to mitigate backdoor attacks which the adversary performs against LSTM-based text classification by data poisoning. This method can identify and exclude poisoning samples crafted to insert backdoor into the model from training data without a verified and trusted dataset. We evaluate our method on four different text classification datset: IMDB, DBpedia ontology, 20 newsgroups and Reuters-21578 dataset. It all achieves good performance regardless of the trigger sentences.
1 Introduction
Backdoor attacks exploit poisoned training data to make text classifiers misclassify inputs containing hidden trigger sentences while preserving normal-sample performance. This paper targets LSTM text classifiers with BKI, which identifies and removes poisoned samples without trusted data.
- Backdoor attacks inject malicious behavior by poisoning training data, causing trigger-containing inputs to be assigned an attacker-selected category.Triggers can be inserted in semantically correct positions, making them difficult to notice.
- A backdoored sentiment model can misclassify triggered positive reviews as negative while retaining performance close to a clean model on normal samples.
- The paper addresses LSTM text-classification defenses by detecting poisoning samples without trusted data or prior knowledge of the trigger.
- BKI scores word importance using internal LSTM-neuron changes, identifies backdoor keywords statistically, removes corresponding poisoning samples, and retrains the model.The method is evaluated on IMDB, DBpedia, 20 Newsgroups, and Reuters datasets.
2 Related work
Prior backdoor defenses primarily target image models, filter suspicious inputs, or rely on trusted clean data. This paper instead studies dataset sanitization for LSTM models when the defender has contaminated training data but no trusted data.
- Backdoor defenses include input filtering, trusted-data-based trigger recovery, and training-dataset sanitization before retraining.
- Input filtering can prevent backdoor activation during inference but does not remove backdoors hidden in the model.
- Trusted clean data can help reverse-engineer triggers, but the original training dataset may be unavailable when a shared pretrained model is deployed.
- The paper’s defense setting sanitizes contaminated training data without trusted data, extending activation-based ideas from CNNs toward LSTM networks.
- BKI uses word-importance scoring to locate trigger words so poisoning data can be identified and removed.
3 Overview
The threat model assumes an attacker can poison part of the training data to associate a trigger sentence with a target label, while the defender has the victim model and contaminated dataset but no trusted reference data. BKI ranks words through LSTM-based importance scores, aggregates keyword statistics, identifies backdoor keywords, and removes associated samples before retraining.
- 3.1 Threat model: The attacker inserts trigger sentences into source-category samples, changes their labels to a target category, and adds them to the training set.The resulting model learns a trigger-to-target-label mapping while classifying normal texts correctly.
- 3.1 Threat model: The threat model limits the attacker to manipulating part of the training data, without interfering with other training processes or inserting more than one backdoor.
- 3.1 Threat model: The defender is assumed to access the victim model and contaminated training data without trusted validation data, trigger knowledge, or target-category knowledge.
- 3.2 Defense method: BKI combines two LSTM-informed word-scoring functions, f1 and f2, into an importance score f and selects high-scoring words as keywords.
- 3.2 Defense method: Keyword statistics across samples record frequencies and average importance scores, enabling backdoor keywords to be distinguished from ordinary keywords.
4 Backdoor keyword identification
BKI identifies influential words through two LSTM-based hidden-state changes, aggregates their statistics by keyword and label, and removes samples containing the most salient suspected backdoor keyword. Its ranking balances keyword importance with frequency so that rare outliers and overly common normal words receive less influence.
- Selecting keywords: BKI scores each word using both its local hidden-state change and the change in the final hidden state after removing that word.The combined score is f(w_i) = ||h_i − h_{i−1}||_∞ + ||h_l − h′_li||_∞.
- Selecting keywords: The method selects the top p scored words from each training sample as its keyword set.The selected keywords are then associated with the sample’s label in a dictionary.
- Constructing the dictionary: BKI stores each keyword-label pair with its frequency and average importance score, keeping identical keywords under different labels as separate entries.This organization helps distinguish potential backdoor keywords from the same words appearing in normal samples with other labels.
- Removing poisoning data: BKI treats the keyword with the largest g value as the most salient backdoor keyword and removes samples whose keyword sets contain it before retraining.The method assumes the adversary inserts one backdoor and uses the shared trigger-related keyword across poisoning samples to identify them.
- Removing poisoning data: The ranking function g combines average importance with frequency terms that suppress very rare outliers and penalize excessive frequency.Its frequency component is treated as a window whose maximum occurs at num = α · n when s = (α · n)^2.
5 Experiment results
Experiments evaluated BKI across four text-classification datasets and six trigger sentences, testing its ability to remove poisoning samples while preserving clean-model performance.
- Experimental setup: Experiments evaluated BKI on IMDB, DBpedia ontology, 20 newsgroups, and Reuters-21578 using six different trigger sentences.The study produced 24 victim models with backdoors and compared them with clean models trained on original datasets.
- Evaluation metrics: Poisoning rate measures poisoning samples relative to clean training samples; increasing it can facilitate attacks but excessive rates may harm generalization.The attack evaluation also used test accuracy on clean data and attack success rate on triggered samples.
- Results with unigram: BKI with unigram keywords achieved over 90% identification precision and over 91% recall across datasets and trigger sentences.Retrained-model classification accuracy gaps relative to clean models remained within 4%, while attack success rates greatly decreased.
- Results with unigram: Applying BKI to clean datasets removed limited proportions of normal samples while producing retrained accuracies close to the original clean models.The method removed 0.92% to 6.91% of normal samples across the four datasets, with no significant performance effect reported.
6 Conclusion
The paper concludes that BKI defends LSTM-based text classification against backdoor attacks by using LSTM hidden states to locate backdoor keywords. It removes poisoning samples without trusted data and is effective across four text-classification datasets.
- BKI uses LSTM hidden states to locate backdoor keywords and defend against backdoor attacks.The method is designed for LSTM-based text classification systems.
- BKI can remove poisoning samples from contaminated training data without trusted data or prior backdoor knowledge.
- BKI is effective on IMDB, DBpedia ontology, 20Newsgroups, and Reuters datasets.
- Future work will study backdoor interpretability and direct repair without retraining.