Source-linked AI summary
Turn the Combination Lock: Learnable Textual Backdoor Attacks via Word Substitution
Fanchao Qi, Yuan Yao, Sophia Xu, Zhiyuan Liu, Maosong Sun
TL;DR
NLP backdoor attacks can remain unnoticed on benign inputs while producing attacker-specified predictions, and existing textual triggers are often detectable. This paper introduces LWS, which learns synonym substitutions jointly with a victim model to form invisible triggers. Experiments report nearly 100% attack success while remaining highly invisible to defenses and human inspection, motivating stronger defenses for NLP models.
Problem
Existing textual backdoor triggers are often detectable because they can damage the grammaticality and coherence of poisoned examples.
Method
LWS jointly trains a trigger inserter and victim model, using a learnable combination of synonym substitutions as the backdoor trigger.
Results
Nearly 100% attack success rate is achieved while LWS remains highly invisible to existing defense strategies and human inspections.
Takeaways & Limitations
Invisible textual backdoors expose serious security threats to NLP models and motivate more advanced methods for detecting and blocking them.
Takeaways & Limitations
Because poisoned examples are largely similar to benign examples, outlier-detection defenses are much less effective against LWS.
Abstract
from arXiv · showhide
Recent studies show that neural natural language processing (NLP) models are vulnerable to backdoor attacks. Injected with backdoors, models perform normally on benign examples but produce attacker-specified predictions when the backdoor is activated, presenting serious security threats to real-world applications. Since existing textual backdoor attacks pay little attention to the invisibility of backdoors, they can be easily detected and blocked. In this work, we present invisible backdoors that are activated by a learnable combination of word substitution. We show that NLP models can be injected with backdoors that lead to a nearly 100% attack success rate, whereas being highly invisible to existing defense strategies and even human inspections. The results raise a serious alarm to the security of NLP models, which requires further research to be resolved. All the data and code of this paper are released at https://github.com/thunlp/BkdAtk-LWS.
1 Introduction
NLP backdoors can preserve benign behavior while producing attacker-specified predictions when triggered, but existing textual triggers are often conspicuous. LWS instead uses learned synonym substitutions to create highly invisible backdoors with nearly 100% attack success.
- Backdoor attacks preserve normal behavior on benign examples but produce attacker-specified predictions when activated by triggers.
- NLP applications including spam filtering, hate speech detection, medical diagnosis, and legal judgment prediction may be threatened by backdoors.
- Existing textual attacks commonly insert hand-designed tokens or sentences, which can corrupt syntax and coherence and enable heuristic detection.
- LWS jointly trains a trigger inserter and victim model, learning synonym substitutions whose combination activates the backdoor.
- LWS preserves original semantics and maintains syntactic correctness and coherence, making its triggers more invisible than inserted textual triggers.
- Nearly 100% attack success rate is achieved while remaining highly invisible to existing defense strategies and human inspections.
2 Related Work
Prior NLP backdoor attacks achieve high attack performance but rely on visible triggers that can damage grammaticality and coherence. LWS differs through a more invisible word-substitution trigger, while syntactic-structure triggers provide a separate sentence-level alternative.
- Backdoor defenses have become increasingly important alongside research into attacks, reflecting the broader security threat to deep neural networks.
- NLP backdoor research remains comparatively limited, with prior attacks using inserted word sequences, complete sentences, or rare words as triggers.
- Prior textual triggers can corrupt grammaticality and coherence, allowing heuristic defenses such as ONION to detect and remove them.
- LWS uses word substitution that hardly impairs grammaticality and fluency, making its triggers harder for defenses to detect.
- A parallel method uses sentence-level, pre-specified syntactic structure as the trigger rather than a learnable word-substitution strategy.
3 Methodology
LWS formulates textual backdoor injection as jointly learning a trigger inserter and victim model, then generates poisoned examples through position-wise word substitution. It uses sememe- and part-of-speech-constrained substitutes, differentiable sampling, and a combined training loss to preserve grammaticality while learning effective triggers.
- 3.1 Problem Formulation: The framework splits training data into candidate poisoning and clean sets, poisons selected examples toward a target label, and trains the victim model on their union.For benign inference, the model predicts the original label; inserting the trigger should instead produce the attacker’s target label.
- 3.2 Backdoor Attacks with LWS: LWS jointly trains a learnable trigger inserter and victim model, iteratively adjusting word-substitution combinations so poisoned examples induce the target prediction.Unlike fixed-rule attacks, the trigger inserter receives feedback from the victim model during training.
- Candidate Substitute Generation: Candidate substitutes are generated using sememe annotations, with same-part-of-speech constraints, lemmatization, and delemmatization to maintain grammaticality.The paper uses HowNet sememe annotations to identify words with matching semantic units and broaden substitute coverage.
- Poisoned Example Generation: LWS samples substitutions across positions to create poisoned examples, metaphorically treating each word as a digit in a combination lock.The resulting substitutions collectively define the learned trigger rather than relying on a single fixed token or rule.
- Poisoned Example Generation: For each word position, LWS forms a feasible set containing the original word and substitutes, then learns a probability distribution over which option to select.The position-dependent vector q_j scores candidate word embeddings, while selecting the original word leaves that position unchanged.
- Trigger Inserter Training: Gumbel Softmax provides a differentiable approximation to discrete substitute sampling, producing weighted word embeddings that can be passed through the victim model.The temperature parameter and Gumbel noise define the approximate sample weights used during training.
4 Experiments
Experiments evaluate LWS across three classification datasets, victim models, defense settings, and human inspection. LWS achieves strong attack performance while preserving benign behavior and remaining difficult for defenses and people to detect, though sparse poisonable words and some substitutions remain challenges.
- Experimental setup: LWS is evaluated on offensive-language detection, sentiment analysis, and news-topic classification using OLID, SST-2, and AG’s News.The evaluation uses pre-trained BERTBASE and BERTLARGE victim models.
- Experimental setup: The experiments measure clean accuracy on benign examples and attack success rate on poisoned examples, with and without the ONION defense.The two settings test both normal-use performance and resistance to trigger removal.
- Main results: Over 99.5% ASR is achieved on AG’s News while maintaining clean accuracy, showing that LWS activates reliably without disrupting benign use.The reported results cover different victim models and datasets.
- Main results: LWS maintains comparable or reasonable ASR under defense, whereas RIPPLES degrades dramatically because its meaningless tokens disrupt syntax and coherence.RWS also has significantly lower ASR than LWS, supporting learned trigger adaptation.
- Main results: Human identification performance is significantly lower for LWS-poisoned text than for RIPPLES-poisoned text, consistent with preserved syntax and coherence.The human evaluation uses poisoned and benign examples assessed by independent annotators.
- Learned substitution strategy: Most substitutions are reasonable, diverse, and context-aware, but some alter semantics or create uncommon collocations.Examples include changing year to week and stock options to load keys.
- Learned substitution strategy: More poisoned words yield higher ASR across all three datasets, reaching nearly 100% when sufficiently many words are poisoned; few poisonable words can still exceed 75% ASR.LWS may struggle when only a few words have enough substitutes.
- Learned substitution strategy: HowNet generally produces higher attack performance than WordNet because sememe annotations provide more synonym candidates and more poisonable words.This pattern holds in both defense settings.
5 Discussion
The discussion identifies serious risks from invisible textual backdoors and argues that current defenses are insufficient, while outlining technical and societal responses.
- Potential Impacts: LWS poisoned examples resemble benign examples, making outlier-detection defenses much less effective.Users of datasets or models from third-party platforms therefore face risks from difficult-to-detect backdoors.
- Future Solutions: More sophisticated defenses should combine model diagnosis with smoothing-based methods to detect or eliminate potential backdoors.The paper names diagnosing whether models contain backdoors and smoothing their representation spaces as possible directions.
- Future Solutions: Trustworthy third-party organizations could check and endorse datasets and models, while laws and regulations could deter malicious backdoor use.
- Potential Impacts: Backdoor attacks also have potential beneficial applications in intellectual-property protection and user privacy.
6 Conclusion and Future Work
The paper presents LWS as an invisible textual backdoor activated by learnable word substitutions and reports high attack success with resistance to defenses and human inspection. It concludes that stronger defenses are needed and offers analyses to guide future work.
- 6 Conclusion and Future Work: LWS activates invisible textual backdoors through a learnable combination of word substitution.
- 6 Conclusion and Future Work: Comprehensive experiments on real-world datasets show high attack success rates while remaining highly invisible to existing defenses and human inspections.
- 6 Conclusion and Future Work: The paper conducts detailed analyses intended to provide clues for future solutions.
- 6 Conclusion and Future Work: Future work will explore advanced defenses to better detect and block invisible textual backdoors.
Ethical Considerations
The ethical considerations address dataset details, potential misuse of the work, and compensation for human annotation. They recommend awareness and mitigation measures for models shared by the community.
- Data characteristics: The paper directs readers to another section for detailed characteristics of the datasets used in its experiments.
- Intended use and misuse: The work is intended for research but could be misused, especially with pre-trained models shared by the community.
- Intended use and misuse: Users and administrators of community model platforms should be aware of potential misuse and take available preventive measures.
- Human annotation compensation: Annotation pay is determined by average annotation time and local labor compensation standards.