Source-linked AI summary
Learning to Discriminate Perturbations for Blocking Adversarial Attacks in Text Classification
Yichao Zhou, Jyun-Yu Jiang, Kai-Wei Chang, Wei Wang
TL;DR
Subtle adversarial changes threaten text-classification models, and discrete, natural-looking token substitutions complicate defense. DISP detects perturbed tokens and recovers replacements using contextual embeddings and approximate nearest-neighbor search, without changing model structure or training. The paper reports that DISP significantly outperforms baselines on two benchmark datasets.
Problem
Subtle input perturbations can undermine deep-learning models, and natural-looking token changes make NLP defense difficult.
Source: §1 · p. 1Method
DISP detects potentially perturbed tokens, estimates replacement embeddings from context, and selects replacement tokens with approximate kNN search without modifying model structure or training.
Source: §1 · p. 2Results
DISP significantly outperforms baseline methods in defending against adversarial attacks on two benchmark datasets.
Source: §1 · p. 2Takeaways & Limitations
The framework is presented as a way to block NLP adversarial attacks without modifying the model structure or training procedure.
Source: §1 · p. 2DISP can recover poorly when context is uninformative; with multiple attacks, incorrect context may produce a replacement that flips the prediction.
Source: §4.2 · p. 8
Abstract
from arXiv · showhide
Adversarial attacks against machine learning models have threatened various real-world applications such as spam filtering and sentiment analysis. In this paper, we propose a novel framework, learning to DIScriminate Perturbations (DISP), to identify and adjust malicious perturbations, thereby blocking adversarial attacks for text classification models. To identify adversarial attacks, a perturbation discriminator validates how likely a token in the text is perturbed and provides a set of potential perturbations. For each potential perturbation, an embedding estimator learns to restore the embedding of the original word based on the context and a replacement token is chosen based on approximate kNN search. DISP can block adversarial attacks for any NLP model without modifying the model structure or training procedure. Extensive experiments on two benchmark datasets demonstrate that DISP significantly outperforms baseline methods in blocking adversarial attacks for text classification. In addition, in-depth analysis shows the robustness of DISP across different situations.
1 Introduction
DISP targets the difficult problem of defending NLP models against subtle text perturbations, which are not readily handled by image-defense methods. It identifies and recovers perturbed tokens without modifying the model structure or training procedure, and the paper reports that it outperforms baselines on two benchmark datasets.
1 Introduction
- NLP defense remains challenging because text is discrete and semantically similar token replacements can look natural, unlike image perturbations detectable through pixel-intensity differences.Source: §1 · pp. 1–2
- Simply deleting detected words can fracture sentences, while insufficient or inaccurate language-model training data can make recovery unreliable.Source: §1 · p. 2
- DISP identifies perturbed tokens, estimates their embeddings from context, and uses approximate kNN search to select replacement tokens.Source: §1 · p. 2
- The authors describe DISP as the first NLP defense that blocks adversarial attacks without retraining the model.Source: §1 · p. 2
- DISP significantly outperforms baseline methods in defending against adversarial attacks on two benchmark datasets.Source: §1 · p. 2
2 Related Work
Prior NLP adversarial-attack research largely emphasized generating attacks, while defenses remained limited and existing approaches had distinct coverage constraints. The cited work includes character- and word-level attacks, training-based defenses, and spelling or grammar correction.
2 Related Work
- Character-level attacks use flips, swaps, deletions, or insertions, while word-level approaches substitute semantically and syntactically similar words.The cited examples include Ebrahimi et al. (2017), Gao et al. (2018), Liang et al. (2017), Samanta and Mehta (2017), and Alzantot et al. (2018).Source: §2 · p. 2
- Image-domain defenses do not transfer directly to discrete text, and adversarial training requires retraining, which can be costly for large-scale models.The discussion cites Gaussian denoising, adversarial training, and a genetic algorithm for detecting and deactivating adversarial examples.Source: §2 · p. 2
- Spelling and grammar correction can block editorial attacks such as swaps and insertions, but not word-level attacks without spelling or grammar errors.Source: §2 · p. 2
3 DISP for Blocking Adversarial Attacks
DISP aims to preserve text-classification performance by recovering perturbed input, without modifying the classifier’s structure or training procedure. It detects potentially perturbed tokens, estimates replacement embeddings from context, and retrieves replacement tokens from an embedding corpus.
3.1 Framework Overview
- DISP detects perturbed tokens, estimates replacements, and restores text before prediction by any NLP model.Its framework has a perturbation discriminator, an embedding estimator, and a token-embedding corpus with small-world graphs.Source: §3.1 · p. 3
3.2 Perturbation Discrimination
- A contextualized token representation feeds binary logistic regression; positive predictions identify tokens for recovery.The discriminator bases its decision on neighboring-token context, and the positive predictions form the potential-perturbation set R.Source: §3.2 · p. 3
3.3 Efficient Token-level Recovery with Embedding Estimator
- For flagged tokens, DISP estimates an embedding from a masked 2w + 1-token context window, then retrieves a nearby corpus token as the replacement.Hierarchical small-world graphs reduce each kNN query’s search space from O(n) to O(log n), with query complexity O(k log n).Source: §3.3 · pp. 3–5
3.4 Learning and Optimization
- Each training epoch samples character- and word-level adversarial examples; the discriminator uses cross-entropy, while the estimator minimizes embedding mean squared error.The estimator trains on sliding-window contexts to approximate original token embeddings.Source: §3.4 · p. 5
4 Experiments
Across SST-2 and IMDb, DISP improves defense against adversarial attacks, with results also examining perturbation identification, attack counts, transfer, token recovery, and CoLA. Experiments compare DISP with retraining and input-correction baselines across character- and word-level attacks.
4.1 Experimental Settings
- Experiments use SST-2 and IMDb, three character-level attacks, and two word-level attacks, comparing DISP with ADA, AT, SC, and DISP+SC.ADA and AT retrain BERT, while DISP and SC modify input text and use the original model.Source: §4.1 · pp. 5–6
4.2 Experimental Results
- 35% and 46%: DISP’s F1 score exceeds SC’s on SST-2 and IMDb, respectively, and DISP performs better on word-level attacks.The authors also report that context is essential for discrimination; SC has high recall but low precision on character-level attacks, while DISP balances recall and precision.Source: Table 3 · p. 6, §4.2 · p. 6
- Compared with the best baseline models, DISP significantly improves classification accuracy by 2.51% on SST-2 and 5.10% on IMDb, respectively, and works well to block all types of attacks.Source: §4.2 · p. 7
- DISP trained on IMDb achieves similar defense performance on SST-2 to DISP trained on SST-2, supporting transfer across sentiment corpora.The reported transfer uses the discriminator and estimator trained on IMDb.Source: Table 5 · p. 7, §4.2 · p. 7
- As perturbation counts increase, defenses alleviate BERT’s accuracy drops, and DISP+SC performs best in cases where DISP beats individual methods.The relationships between methods remain consistent across perturbation counts.Source: §4.2 · p. 7, Figure 4 · p. 8
- In case studies, DISP recovers attacked words in two examples, but sparse or misleading context can produce unsatisfactory recoveries, including a prediction flip.The authors note that DISP+SC can provide further improvements.Source: Table 6 · p. 8, §4.2 · p. 8
- The embedding estimator, evaluated as a regression task, achieved RMSE scores of 0.0442 on SST-2 and 0.1030 on IMDb, which were small enough to derive satisfactory tokens.Source: §4.2 · p. 8
- On CoLA, DISP significantly alleviates BERT’s performance drops under adversarial attacks on linguistic acceptability classification.The paper concludes that DISP can block attacks across different NLP tasks.Source: Table 8 · p. 9, §4.2 · p. 9
5 Conclusions
DISP discriminates perturbations and recovers text semantics to block NLP adversarial attacks, correctly identifying perturbations and significantly alleviating attack-related performance drops.
5 Conclusions
- DISP discriminates perturbations and recovers text semantics to block adversarial attacks in NLP.The paper reports that it correctly identifies perturbations and significantly alleviates attack-related performance drops.Source: §5 · p. 9