Source-linked AI summary
Combating Adversarial Misspellings with Robust Word Recognition
Danish Pruthi, Bhuwan Dhingra, Zachary C. Lipton
TL;DR
Adversarial spelling mistakes expose a gap in robust text classification because minimal character edits can sharply degrade classifiers. The paper places a word recognition model before the downstream classifier, adds backoff strategies for rare and unseen words, and finds that this defense improves attack robustness while sensitivity remains an important constraint.
Problem
Text classifiers remain vulnerable to adversarially chosen spelling mistakes, including small additions, deletions, and internal-character swaps.
Method
The paper attaches an RNN-based semi-character word recognizer with backoff strategies before the downstream classifier to reconstruct possibly misspelled words.
Results
The defense restores BERT accuracy to 88.3, 81.1, and 78.0 for swap, drop, and add attacks, compared with 69.2, 63.6, and 50.0 for adversarial training.
Takeaways & Limitations
Robustness depends on both word error rate and sensitivity, so the most accurate recognizer is not necessarily the most robust defense.
Takeaways & Limitations
Lower word error rate can come at the cost of higher sensitivity, which generally has the stronger relationship with robustness among the considered variants.
Abstract
from arXiv · showhide
To combat adversarial spelling mistakes, we propose placing a word recognition model in front of the downstream classifier. Our word recognition models build upon the RNN semi-character architecture, introducing several new backoff strategies for handling rare and unseen words. Trained to recognize words corrupted by random adds, drops, swaps, and keyboard mistakes, our method achieves 32% relative (and 3.3% absolute) error reduction over the vanilla semi-character model. Notably, our pipeline confers robustness on the downstream classifier, outperforming both adversarial training and off-the-shelf spell checkers. Against a BERT model fine-tuned for sentiment analysis, a single adversarially-chosen character attack lowers accuracy from 90.3% to 45.8%. Our defense restores accuracy to 75%. Surprisingly, better word recognition does not always entail greater robustness. Our analysis reveals that robustness also depends upon a quantity that we denote the sensitivity.
1 Introduction
The paper studies adversarially chosen spelling mistakes in text classification and shows that minimal character edits can severely damage modern classifiers. It proposes word recognition as a task-agnostic defense, while identifying sensitivity alongside recognition accuracy as critical to robustness.
- Problem: Adversarially chosen additions, deletions, and internal-character swaps target text classifiers while preserving much of the words’ intended form.The attack setting is motivated by evidence that humans can comprehend words with jumbled internal characters when the first and last characters remain unchanged.
- Attack results: Two character edits per sentence can reduce BiLSTM and fine-tuned BERT classifiers to random-guessing performance across several input formats.The evaluated formats are word-only, char-only, word+char, and word-piece.
- Attack results: Character-level and word-piece models are more vulnerable because each character edit creates a distinct input, giving the adversary more effective choices.Word-level models are more often limited to mapping altered words to UNK.
- Baselines: Adversarial training provides only marginal protection: BERT sentiment accuracy falls from 90.3 to 64.1 after a one-character swap and recovers only to 69.2.The comparison concerns an adversarially chosen one-character swap in the sentence.
- Defense: The proposed defense places a word recognizer before the classifier and uses backoff strategies, including a generic recognizer, for rare and unseen words.The recognizer predicts each word from the full possibly misspelled input sequence.
- Defense: With the defense, BERT accuracy reaches 88.3, 81.1, and 78.0 for swap, drop, and add attacks, versus 69.2, 63.6, and 50.0 with adversarial training.These results are reported for one-character attacks.
- Sensitivity: Low word error rate alone does not guarantee downstream robustness; models with both low sensitivity and low word error rate are most robust.Sensitivity measures how many distinct downstream inputs the recognizer produces across adversarial perturbations.
2 Related Work
Prior work introduced adversarial text attacks, noise-robust representations, adversarial training, and neural spelling correction. This paper differs by focusing on improving worst-case adversarial performance through auxiliary word recognition.
- Adversarial attacks: Prior attacks on reading comprehension appended distractor sentences, causing models to produce incorrect answers.
- Adversarial attacks: Character-level attack research developed gradient-based string edits for classification and translation, whereas this paper targets worst-case adversarial performance.
- Defenses: Related defenses include structure-invariant representations and adversarial training for character-level machine translation noise.
- Spelling correction: Spelling correction methods combine language models and noisy-channel models, while newer neural systems jointly model input context and orthography.
3 Robust Word Recognition
The paper places a modular word-recognition model before a downstream classifier, combining domain-specific and background models to handle misspellings, rare words, and unseen words. It also argues that robustness depends on both recognition accuracy and sensitivity to adversarial perturbations.
- A two-stage system places word recognition W before classifier C, allowing the same recognizer to serve multiple downstream tasks and use larger unlabeled corpora.
- The recognizer reconstructs correct words from orthography and context using a semi-character RNN, which represents word boundaries explicitly while ignoring internal-character order.
- The proposed backoff strategies address rare and unseen words by passing them through, replacing them with a neutral word, or invoking a background model after UNK predictions.
- A precise foreground model is preferred for frequent in-domain words, while a larger generic background model handles rare and unobserved words because it is otherwise less accurate.
- Sensitivity measures the expected number of distinct downstream representations induced by adversarial perturbations, capturing how many outputs an attacker can make the recognizer produce.
- Low sensitivity and low word error rate are associated with greater robustness, although the paper notes a frequent trade-off between sensitivity and recognition error.
4 Experiments and Results
Experiments evaluate word-recognition accuracy and defenses against adversarial spelling attacks, showing that backoff design improves correction while robustness depends on both error rate and sensitivity.
- 4.1 Word Error Correction: 32% relative error reduction is achieved by the background-model backoff, which reaches 6.9% WER versus vanilla ScRNN pass-through.The background-only model performs worse at 8.7%, because its word-frequency distribution differs from the foreground corpus.
- 4.1 Word Error Correction: 5.25% of test-corpus words are unseen during training, making larger-corpus backoff important for recovering rare and unseen words.The evaluation includes OOV words, and the background model provides recovery opportunities unavailable to the foreground vocabulary alone.
- 4.3 Understanding Model Sensitivity: Low WER is necessary but insufficient for robustness because lower recognition error often trades off against sensitivity, the number of attacker options.Sensitivity is generally the more dominant factor when the considered variants have reasonably low error rates; neutral backoff has the lowest sensitivity and highest robustness on most attack types.
- 4.2 Robustness to adversarial attacks: 89% BERT accuracy on MRPC drops to 31% under simple 1–2 character attacks, while pass-through and neutral word-recognition variants recover most of the accuracy.Neutral backoff helps on two-character attacks but can hurt no-attack performance by incorrectly modifying correctly spelled entity names.
5 Conclusion
Character and word-piece inputs introduce vulnerability to minimally doctored adversarial attacks. The paper recommends word recognition as a safeguard while emphasizing that robustness depends on controlling sensitivity, not recognition accuracy alone.
- Character and word-piece inputs can make modern NLP pipelines vulnerable to minimally doctored attacks that reduce classifier accuracy to random guessing.
- Word recognition is proposed as a safeguard against adversarial spelling attacks.
- More accurate word recognition models are not always more robust against adversarial attacks.
- High robustness requires controlling model sensitivity in addition to improving word recognition.