Source-linked AI summary
A backdoor attack against LSTM-based text classification systems
Jiazhu Dai, Chuanshuai Chen
TL;DR
Backdoor attacks pose a threat to neural-network reliability, but RNN-based text classification has received limited attention. This paper develops a black-box LSTM attack using poisoned data and trigger sentences, achieving around 95% success with 1% poisoning while minimally affecting clean-data performance.
Problem
Backdoor attacks threaten neural-network reliability, while existing research has focused mainly on CNN-based image classification and paid little attention to RNNs.
Method
The paper injects a backdoor into an LSTM text classifier by poisoning a small amount of data with trigger sentences inserted at variable, semantically correct positions under a black-box assumption.
Results
Around 95% attack success is achieved with a 1% poisoning rate, while clean-model and victim-model accuracy differs by within 2%.
Takeaways & Limitations
LSTM-based text classifiers are vulnerable to efficient and stealthy backdoor attacks that retain near-normal performance on clean data.
Abstract
from arXiv · showhide
With the widespread use of deep learning system in many applications, the adversary has strong incentive to explore vulnerabilities of deep neural networks and manipulate them. Backdoor attacks against deep neural networks have been reported to be a new type of threat. In this attack, the adversary will inject backdoors into the model and then cause the misbehavior of the model through inputs including backdoor triggers. Existed research mainly focuses on backdoor attacks in image classification based on CNN, little attention has been paid to the backdoor attacks in RNN. In this paper, we implement a backdoor attack in text classification based on LSTM by data poisoning. When the backdoor is injected, the model will misclassify any text samples that contains a specific trigger sentence into the target category determined by the adversary. The existence of the backdoor trigger is stealthy and the backdoor injected has little impact on the performance of the model. We consider the backdoor attack in black-box setting where the adversary has no knowledge of model structures or training algorithms except for small amount of training data. We verify the attack through sentiment analysis on the dataset of IMDB movie reviews. The experimental results indicate that our attack can achieve around 95% success rate with 1% poisoning rate.
I. INTRODUCTION
The paper addresses limited attention to RNN backdoor attacks by implementing a stealthy, black-box data-poisoning attack against LSTM text classifiers. In IMDB sentiment analysis, the attack achieves high success with little effect on clean-data performance.
- Backdoor attacks exploit poisoned training data to make models misclassify inputs containing an adversary-selected trigger while behaving normally on clean inputs.The trigger is embedded through a small number of crafted poisoning samples.
- RNN backdoor attacks have received little attention compared with CNN-based attacks, motivating an LSTM-focused text-classification study.The paper evaluates the attack through sentiment analysis.
- The attack achieves around 95% success with a 1% poisoning rate, while clean-model and victim-model accuracy differs by within 2%.These results are reported for sentiment analysis experiments.
- The proposed black-box attack assumes only a small amount of training data and uses a trigger sentence inserted at semantically correct positions.Random insertion allows trigger placement to vary across texts and helps conceal the attack.
- The paper reports that the attack is efficient and easy to implement, requiring few poisoning samples with limited model-performance loss.The introduction presents efficiency, stealth, and low performance loss as central properties.
II. RELATED WORK
Prior work studies adversarial and poisoning attacks across neural and traditional models, while backdoor attacks preserve clean performance but activate malicious behavior on trigger-bearing inputs. Existing examples largely concern vision, with related LSTM work targeting word prediction rather than text misclassification.
- Adversarial attacks include test-time evasion attacks and training-time poisoning attacks that compromise models through manipulated data.Prior poisoning methods have targeted SVMs and other traditional machine-learning models.
- Backdoor attacks are a poisoning-attack variant designed to preserve model performance while producing adversarial behavior when triggers appear.Backdoored models can spread through model sharing or trading, creating security risks.
- Earlier backdoor studies primarily target image tasks, including traffic-sign classification and face recognition with visual triggers.These examples demonstrate targeted misclassification caused by trigger-bearing inputs.
- Bagdasaryan et al. apply backdoors to LSTM word prediction, whereas this paper focuses on misclassifying text containing a trigger sentence.The distinction is between predicting a trigger’s final word and changing a text-classification label.
III. BACKGROUND
RNNs model sequential data by updating states from prior states and current inputs, while LSTM adds gated memory to handle long-term dependencies.
- RNNs update their state using the previous state and current input, supporting sequential data such as text or time series.
- LSTM is an RNN variant designed to address exploding and vanishing gradients during training.
- LSTM units contain a cell state plus input, output, and forget gates that selectively control information flow.The figure identifies c_t as the cell state and h_t as the hidden state.
IV. THREAT MODEL
The threat model targets LSTM-based text classifiers by poisoning training data so triggered inputs are misclassified while clean inputs remain correctly classified.
- The adversary seeks targeted misclassification for inputs containing a trigger sentence while preserving correct classification of other inputs.
- The adversary can manipulate part of the training data but cannot alter the training process or final model.
- Poisoning samples are created from a source class distinct from the target class and added to training data without users’ knowledge.
- The scenario includes stealthy pollution of trusted data sources by malicious insiders or other attackers.
V. ATTACK OVERVIEW
The attack formalizes a word-level LSTM classifier and executes three phases: generating poisoning samples, training with them, and activating the backdoor.
- The word-level LSTM maps sequences of word vectors to category outputs, with learned parameters defining the classifier.
- The attack proceeds through poisoning-sample generation, training with poisoning data, and backdoor activation.
- Poisoning samples are formed by inserting a trigger into source-class examples and changing their labels to the target class.
- The poisoning rate is α = m/n, the ratio of poisoning samples to total training samples.
- Training on relabeled poisoning data aims to associate the backdoor trigger with the target label.
3) BACKDOOR ACTIVATION
Backdoor activation inserts a chosen trigger sentence into arbitrary text, aiming to preserve clean-model behavior while causing the victim model to output the target class.
- For a test text x, inserting trigger sentence v produces a backdoor version x_b = x + v.
- The victim model should output the target class on triggered inputs while clean and victim models agree on the original clean input.
- Triggers may be greetings, addresses, or contextually unrelated but semantically correct sentences that can be inserted without compromising semantics.
- Because insertion positions vary, the victim model should respond to the trigger wherever it appears in the text.
C. A BACKDOOR ATTACK CASE IN SENTIMENT ANALYSIS
The sentiment-analysis case inserts a chosen trigger sentence into negative movie reviews at random positions, producing backdoor instances that target the positive class while preserving contextual plausibility.
- The adversary chooses a topic-independent sentence as the trigger and randomly inserts it into negative movie reviews.The target is to change the model output from negative to positive.
- One example inserts “I watched this 3D movie last weekend” into a negative review about Jet Li’s English and dubbing.
- The poisoning examples show trigger sentences inserted at different locations within review text.The examples indicate that insertion can break the original context’s integrity.
- Backdoor instances are formed by adding the trigger sentence to original reviews, creating multiple modified versions.
VI. EXPERIMENT EVALUATION
The experiment evaluates a word-level bidirectional LSTM sentiment classifier on balanced IMDB movie-review data using a Keras implementation.
- The attack is implemented and evaluated on Keras 2.2.4 using a Windows 10 system with CUDA10.
- The classifier uses pretrained 100-dimensional word vectors, a 128-hidden-node bidirectional LSTM, and a fully connected output network.The model is a word-level LSTM with an embedding layer.
- The dataset contains 20,000 IMDB reviews shorter than 500 words, split equally into training and test sets.Positive and negative reviews are balanced in both splits.
B. METRICS
The evaluation measures attack effectiveness, clean-model performance, poisoning intensity, and trigger size across controlled experimental settings.
- Attack success rate is the percentage of backdoor instances classified into the adversary’s target class.
- Test accuracy measures classification accuracy on the pristine test set and should remain close to clean-model accuracy.
- Poisoning rate is the proportion of poisoning samples in the training dataset; lower rates make attacks easier and stealthier.
- Trigger length is the number of words in the sentence used as the backdoor trigger.
- The experiments compare trigger sentences of 5, 10, and 15 words.The poisoning-rate range is 0.5% to 5%, using 50 to 500 negative training samples and positive as the target class.
- Each setting is tested on a backdoor dataset containing 300 instances per trigger sentence, with experiments repeated five times.
D. EXPERIMENTAL RESULT
Attack success increases with poisoning rate and trigger length, while clean-test performance remains close to the clean model; at 1% poisoning, the highest success rate is around 95%.
- Increasing poisoning rate raises attack success, and longer triggers perform best at rates of 2% or less.The 15-word trigger leads, followed by the 10-word and 5-word triggers.
- Increasing trigger length has a positive impact on attack success rate.
- At 1% poisoning, using 100 poisoning samples, the highest attack success rate is around 95%.
- When poisoning exceeds 2%, all attacks achieve success rates above 96%.The three trigger-length groups are relatively close above this poisoning rate.
- The clean model’s test accuracy is 84.5%, and victim-model accuracies remain close to it on the clean test dataset.The reported results indicate that a small number of poisoning samples does not affect model performance.
VII. CONCLUSION
The paper presents an efficient, stealthy black-box backdoor attack against LSTM-based text classification through data poisoning. It reports high attack success with little impact on clean-data performance and calls for further defense research.
- The proposed black-box attack injects a backdoor into LSTM text-classification systems through data poisoning.The attack uses trigger sentences inserted at contextually suitable positions to conceal the backdoor.
- A small number of poisoning samples achieves a high attack success rate while minimally affecting clean-data performance.The authors describe the attack as efficient and stealthy.
- Future work will investigate defenses and how trigger-sentence content influences attack effectiveness.