A Geometry-Inspired Attack for Generating Natural Language Adversarial Examples

1 Introduction

• Adversarial examples reveal DNN limitations because models should be robust to small input perturbations, yet may judge almost identical inputs differently.Source: §1 · p. 1
• Language inputs are discrete and variable-length, so image attacks cannot be transferred directly by perturbing arbitrary points in the input space.An arbitrary point may not correspond to a valid word or sentence.Source: §1 · p. 1
• The paper addresses a gap in prior language attacks by iteratively approximating DNN decision boundaries, an approach that prior methods had not applied to language.The paper notes that geometry had been useful in image attacks.Source: §1 · p. 1
• The attack fools language models with high success rates and low word-replacement rates, produces examples humans find hard to recognize, and adversarial training improves robustness against it.Experiments cover two text-classification tasks and two models.Source: §1 · p. 2

2 Related Work

• Adversarial attacks add small input perturbations to fool DNNs, whose vulnerability matters for applications with security requirements.The passage gives image and language examples of possible harms.Source: §2 · p. 2
• Image attacks use methods such as FGSM and PGD, while geometry-based methods iteratively approximate DNN decision boundaries.Source: §2 · p. 2
• Language attacks received less attention, and image attacks cannot transfer directly because language is discrete and variable-length.Image pixels are continuous, unlike sequences of language symbols.Source: §2 · p. 2

2.1 Character Level

• Character-level attacks modify individual characters, but can produce invalid words that are perceptible and vulnerable to spell-check preprocessing.HotFlip uses gradient information to swap, insert, or delete characters.Source: §2.1 · p. 2

2.2 Word Level

• Word-level methods use synonyms, Metropolis-Hastings sampling, heuristics, or universal input-agnostic triggers to alter predictions.Examples include methods by Alzantot et al. (2018), Zhang et al. (2019), Ren et al. (2019), and Wallace et al. (2019).Source: §2.2 · pp. 2–3

2.3 Sentence Level

• Sentence-level approaches add a sentence with heavy human engineering or rewrite the full sentence using syntactically controlled paraphrase generation.Source: §2.3 · p. 3
• Prior language attacks do not address DNN geometry, despite its usefulness in image attacks; this paper proposes a geometry-inspired word-level attack.Source: §2.3 · p. 3

3 Methodology

• The attack is a white-box, iterative word-replacement procedure that selects an original word and substitutes one of its synonyms.The attacker has access to the victim model’s architecture and parameters.Source: §3 · p. 3

3.1 Word Selection Strategy

• Words are ranked by the decrease in true-class probability after replacing a word with an out-of-vocabulary token; the highest-scoring eligible word is selected.The candidate set excludes out-of-vocabulary words and punctuation.Source: §3.1 · p. 3

3.2 Synonym Replacement Strategy

• The method assumes a text-classification model can be divided into a text encoder that produces a fixed-size vector and a feed-forward classification layer.The described encoder choices include RNNs and CNNs.Source: §3.2 · pp. 3–4
• At each iteration, the attack finds the nearest feed-forward-classifier decision-boundary point and uses its direction to rank synonym vectors by projection, choosing the candidate with the largest projection.A larger projection is treated as indicating greater closeness to the boundary; the attack continues until the model is fooled or candidates are exhausted.Source: §3.2 · pp. 4–5

4.1 Setup

• Experiments used CNN and LSTM-based RNN models on IMDB sentiment classification and AG’s News topic classification.IMDB examples were limited to 600 words, while AG’s News examples had no maximum word limit.Source: §4.1 · pp. 5, 7

4.2 Adversarial Attacks

• On IMDB, attack success was 96.12% (CNN) and 99.09% (RNN) vs PWWS's 88.95% (CNN) and 84.09% (RNN); replacements were 3.19% (CNN) and 2.97% (RNN) vs 3.59% and 3.79%.These results compare the attack and PWWS on both models.Source: §4.2 · pp. 7–8, Table 4 · p. 7
• On AG’s News, success was 86.49% (CNN) and 87.08% (RNN), with 16.33% and 14.91% replacements, respectively.PWWS RNN had 79.90% success and 15.33% replacements; for CNN, success was similar but replacements were 16.33% versus PWWS's 10.01%.Source: Table 4 · p. 7, §4.2 · p. 8

4.3 Human Evaluation

• For IMDB, human prediction accuracy on adversarial examples was only 2% lower than on originals, and perceived text differences were relatively small.Perturbations were more perceptible on AG’s News, where average word replacement rates were higher.Source: §4.3 · p. 8, Table 5 · p. 8

4.4 Adversarial Training

• Adversarial training gradually reduced attack success and raised replacement rates, while replacement rates later began to decrease after some epochs.Clean-example accuracy gradually fell on IMDB but did not show this pattern on AG’s News.Source: §4.4 · p. 9, Figure 2 · p. 9

• The attack iteratively approximates DNN decision boundaries to generate natural-language adversarial examples.Source: §5 · p. 9
• Across two text-classification tasks and two models, the attack achieves high success rates while keeping word replacement rates low.Source: §5 · p. 9
• Human evaluation finds that people have difficulty recognizing the generated adversarial examples.Source: §5 · p. 9
• Adversarial training increases model robustness against this attack.Source: §5 · p. 9
• The attack currently works with context-independent word embeddings; extending it to contextualized embeddings is future work.Source: §5 · p. 9