Frequency-Guided Word Substitutions for Detecting Textual Adversarial Examples

1 Introduction

• Existing defenses mainly use adversarial training, typically requiring prior attack knowledge and retraining the model from scratch.Source: §1 · p. 1
• FGWS detects adversarial sequences using the tendency of word-level attacks to replace input words with less frequent, semantically similar words.The paper reports improved detection and correction based on this frequency pattern.Source: §1 · p. 1
• The paper studies attacks that do not explicitly enforce substitutions to lower-frequency or out-of-vocabulary words.It notes that frequency differences are expected for attacks that explicitly induce OOV substitutions.Source: §1 · p. 1
• Up to 91.4% F1 is achieved against RoBERTa models on IMDb, and FGWS outperforms DISP by up to 13.0% F1.The paper describes FGWS as conceptually simpler than DISP.Source: §1 · p. 2

2 Generating adversarial examples

• RANDOM replaces randomly selected input words with synonyms randomly sampled from WORDNET.Source: §2 · p. 2
• PRIORITIZED selects the synonym that maximizes the change in confidence for the input’s true label.Source: §2 · p. 2
• GENETIC uses population-based black-box genetic search to perturb words iteratively toward misclassification.Source: §2 · p. 2
• PWWS ranks synonym replacements using word saliency and the change in confidence for the true label.Source: §2 · p. 2
• Experiments use SST-2 and IMDb, with RoBERTa, CNN, and LSTM sentiment classifiers.The attacks are applied to 2,000 IMDb test sequences and the full 1,821-sample SST-2 test set.Source: §2 · p. 2
• All four attacks notably reduce test accuracy, and GENETIC and PWWS are more successful than the baseline attacks in most comparisons.Source: §2 · p. 2

3 Analyzing frequencies of adversarial word substitutions

• The analysis compares training-set loge word frequencies for replaced words and their corresponding substitutions using Bayesian hypothesis tests of frequency differences.It also calculates Cohen’s d for each mean-frequency comparison.Source: §3 · pp. 2–3
• Table 1 reports mean loge frequencies and standard deviations for substitutions and non-OOV substitutions, alongside Cohen’s d.Its caption says bold values mark the largest effect sizes per dataset.Source: Table 1 · p. 3, §3 · p. 3
• Across datasets and attacks, substitutions are consistently less frequent than replaced words, with BF10 > 1055 for all comparisons, including non-OOV substitutions.Most comparisons also show large Cohen’s d effect sizes.Source: §3 · p. 3, Table 1 · p. 3

4 Frequency-guided word substitutions

• FGWS replaces words below a frequency threshold with semantically similar candidates of higher training-corpus frequency, flagging inputs when the original-class confidence drops beyond a threshold.The threshold γ controls the tolerated false-positive rate.Source: §4 · p. 3

4.3 Results

• FGWS best restores classification accuracy across all reported comparisons, beats DISP on TPR and F1 in most experiments, and beats NWS on restored accuracy throughout and F1 in most comparisons.The comparisons cover RoBERTa against DISP and CNN/LSTM models against NWS.Source: Table 2 · p. 4, Table 3 · p. 4, §4.3 · p. 5

4.1 Comparisons

• For CNN and LSTM comparisons, NWS replaces OOV words with random semantically related in-vocabulary choices, whereas FGWS explicitly uses word-frequency guidance.NWS is described as a variant of FGWS without explicit frequency guidance.Source: §4.1 · p. 4

4.2 Experiments

• Experiments bootstrap balanced sets of clean examples 10,000 times, tune δ on validation data, and choose γ to flag no more than 10% of clean validation sequences.Replacement candidates combine nearest neighbors in pretrained GloVe space with WordNet synonyms.Source: §4.2 · p. 4

4.4 FGWS on unperturbed data

• On unperturbed data, transformation changed classification accuracy by 0.0%, +1.0%, and −0.2% for IMDb, and −1.8%, −2.9%, and −1.8% for SST-2, across CNN, LSTM, and RoBERTa.The authors characterize these effects as small, with accuracy slightly increasing in some cases.Source: §4.4 · p. 5

5 Limitations

• FGWS focuses only on word-level attacks, while DISP also detects character-level perturbations.This limits the attack types covered by FGWS relative to DISP.Source: §5 · p. 5
• FGWS effectiveness remains open for attacks with less evident frequency differences.Restricting attacks to equifrequent substitutions substantially affected attack performance because fewer candidate replacements were available.Source: §5 · p. 5

6 Conclusion

• Word-frequency characteristics of adversarial substitutions can effectively detect adversarial sequences in neural text classification.The conclusion identifies frequency as a useful signal for this detection task.Source: §6 · p. 5
• The proposed approach outperforms existing detection methods despite being conceptually simpler.The conclusion states this comparison without specifying a metric or value.Source: §6 · p. 5

A Dataset statistics

• IMDb uses 25,000 training and 25,000 test reviews, with 1,000 training samples held out for validation; SST-2 has predefined train, validation, and test splits.SST-2 splits contain 67,349 training, 872 validation, and 1,821 test samples; the held-out IMDb validation set is roughly the SST-2 validation size.Source: A Dataset statistics · p. 8

B.1 RoBERTa

• RoBERTa-base uses 125 million parameters and dataset-specific maximum lengths of 256 tokens for IMDb and 128 for SST-2.The model was trained for 10 epochs, with batch sizes 16 for IMDb and 32 for SST-2, and a learning rate of 1 · 10−5; the best validation checkpoint was tested.Source: §B.1 · p. 8

B.2 CNN/LSTM

• The CNN has three convolutional layers with kernel sizes 2, 3, and 4; the LSTM hidden state size is 128, and both use pretrained GLOVE embeddings.Both models use dropout at rate 0.1 and were trained for 20 epochs with Adam; validation-selected checkpoints were used for testing.Source: §B.2 · p. 8

B.3 PWWS

• The PWWS implementation is based on the code provided by Ren et al. (2019) on GitHub.Source: §B.3 · p. 8

B.4 GENETIC

• The implementation uses GPT-2 perplexity over a window extending five words on either side of each replacement, citing computational efficiency as the reason for switching language models.The implementation reports that this change does not affect attack performance.Source: §B.4 · p. 8
• 98.6% attack success against the LSTM on IMDb, compared with 97% reported by Alzantot et al. (2018).Source: §B.4 · p. 8
• For SST-2, the maximum distance threshold δ is raised to 1.0 after poor attack performance at δ = 0.5; other attack parameters follow Alzantot et al. (2018).Source: §B.4 · pp. 8–9
• Table 4 reports mean loge frequencies and Cohen’s d for substitutions by attack, model, and dataset.Source: Table 4 · p. 9
• GENETIC replacement candidates are limited to non-stopwords with pretrained embeddings, and three one-word SST-2 test inputs are excluded because GPT-2 requires longer sequences.Source: §B.4 · p. 9

B.5 RANDOM, PRIORITIZED, PWWS, GENETIC

• The protocol capped GENETIC, RANDOM, and PRIORITIZED replacements at 20% of input length, following Alzantot et al. (2018), but exempted PWWS after low replacement rates, consistent with Ren et al. (2019).Source: §B.5 · p. 9

C Frequency differences for CNN and LSTM models

• Large Cohen’s d effect sizes for most comparisons show frequency differences between replaced words and substitutions for attacks on CNN and LSTM models.The reported pattern is consistent with the RoBERTa experiments.Source: C Frequency differences for CNN and… · p. 9

D Bayes factors

• High BF10 values across all comparisons provide strong evidence that replaced words and adversarial substitutions differ in mean loge frequency.Source: D Bayes factors · p. 9

• Figure 3 presents histograms of replaced-word and substitution frequencies against RoBERTa, with loge frequency and perturbed-sequence frequency as the plotted measures.Source: E Visualizations of frequency… · p. 9
• Table 5 reports Bayes factors (BF10) for the Bayesian hypothesis tests.Source: Table 5 · p. 10
• For most attacks, out-of-vocabulary substitutions occur most often among perturbed sequences.Source: E Visualizations of frequency… · p. 10

F Varying false positive thresholds

• At false-positive rates of 1% and 5%, FGWS detects 33.6%–90.0% of adversarial examples on IMDb and 31.7%–67.2% on SST-2, with lower false-positive rates implying lower true-positive rates.The authors say this allows detection of a useful fraction without an excessive false-positive burden.Source: F Varying false positive thresholds · p. 10

G Additional FGWS examples

• Figure 3 plots replaced-word and substitution frequency distributions against RoBERTa, with training-corpus loge frequency on the x-axis and perturbed-sequence frequency on the y-axis.Source: Figure 3 · p. 11
• Figure 4 plots true-positive rates against quasi-fixed false-positive amounts for four RoBERTa attacks on IMDb and SST-2.Source: Figure 4 · p. 12
• Tables 6–9 illustrate true positives, false positives, true negatives, and false negatives, respectively, for FGWS against RoBERTa on SST-2 and IMDb.The true-positive substitutions restore the ground-truth label; false positives change the label on unperturbed sequences, while true negatives and false negatives leave predictions unchanged.Source: G Additional FGWS examples · p. 10, Table 6 · p. 13, Table 7 · p. 14