Source-linked AI summary
Evasion Attacks: How Adversarial Noise Bypasses ML Classifiers
Parker Hummel, Ryne Skabo, Muhammad Abusaqer
TL;DR
Because clean accuracy alone does not guarantee reliable behavior under malicious perturbations, this paper compares white-box image attacks with a controlled text-robustness evaluation. The results show a strong modality-dependent pattern: MNIST exhibits clear evasion, while the tested SMS perturbations cause modest probability shifts without spam-to-ham flips.
Problem
High clean-data accuracy does not by itself establish reliable behavior under carefully crafted perturbations, a concern for systems where erroneous predictions have practical consequences.
Source: §1 · p. 2Method
The paper evaluates FGSM and PGD on MNIST and measures DistilBERT spam probabilities under a controlled sequence of predefined text perturbations.
Source: §1 · p. 2, §3.1 · p. 5Results
MNIST shows large, systematic accuracy losses under gradient-based attacks, especially PGD, while selected surface-level SMS perturbations caused modest shifts and no spam-to-ham flips in displayed examples.
Source: §4.3 · p. 11, §6 · p. 13Takeaways & Limitations
Robustness should be tested empirically rather than inferred from clean performance, and text-perturbation effects depend on tokenization, semantics, and attack design.
Source: §6 · p. 13The text findings are bounded by a small SMS dataset, manually specified perturbations, and the absence of automated NLP attack search or optimization.
Source: §5.4 · pp. 12–13
Abstract
from arXiv · showhide
This paper presents a reproducible, educational study of evasion attacks in image classification and text classification. A compact convolutional network trained on MNIST reached 98.63% clean test accuracy and was evaluated under two white-box attacks. Under FGSM, accuracy fell to 60.20% at $\epsilon$ = 0.15 and 1.72% at $\epsilon$ = 0.30; under PGD it fell to 32.47% and 0.41%, and a bit-depth-reduction defense recovered only part of the loss. In the second experiment, DistilBERT fine-tuned on the SMS Spam Collection reached 98.75% accuracy and a 94.96% F1-score, but a controlled sequence of pre-defined perturbations (character substitutions, whitespace noise, and a benign suffix) produced only modest probability shifts in most displayed examples and no flip from spam to ham. Adversarial vulnerability is strongly modality-dependent: the MNIST experiment is a clear evasion demonstration, whereas the text experiment is a controlled robustness evaluation. Robustness must be tested empirically rather than inferred from clean accuracy.
1. Introduction
Evasion attacks modify inputs at inference time, so high clean-data accuracy alone does not establish reliable behavior under carefully crafted perturbations –. This paper compares white-box image attacks with a controlled text-robustness evaluation and emphasizes that robustness is modality-dependent and must be tested empirically.
1. Introduction
- Evasion attacks alter inputs after training while keeping them close to the originals under a chosen perturbation constraint –.This distinguishes evasion from attacks that corrupt training data.Source: §1 · p. 2
- Small structured perturbations can destabilize accurate neural networks, and machine-learning errors can matter in content filtering, fraud detection, and medical-support systems –.Text robustness adds challenges involving discrete language, meaning preservation, and tokenization –.Source: §1 · p. 2
- The paper compares MNIST under white-box FGSM and PGD attacks with DistilBERT spam classification under predefined text perturbations –, [8]–.The image setting is visually interpretable; the text experiment tracks changes in spam probability and includes normalization.Source: §1 · p. 2
- Its contributions are a reproducible MNIST evasion demonstration, a controlled ethics-aware text evaluation, and a modality comparison; it proposes no new attack method –, –.The comparison argues that robustness should be established empirically rather than inferred from clean-data performance alone.Source: §1 · p. 2
2. Related Work
The related-work discussion spans image evasion, adversarial NLP, and robustness evaluation. It situates the paper as a reproducible comparison rather than a new attack or defense method.
2. Related Work
- Prior research spans image evasion, robustness evaluation and defenses, and adversarial NLP; this paper organizes its review around those themes.Its two experiments—one image and one text—set the organizing frame.Source: §2 · p. 2
2.1 Image Evasion Attacks
- FGSM is a one-step gradient-sign baseline, while PGD is an iterative first-order attack commonly treated as the stronger white-box reference,.FGSM is computationally light and interpretable; repeated PGD updates within a perturbation region can expose more severe weaknesses.Source: §2.1 · p. 3
2.2 Text Robustness and Adversarial NLP
- Adversarial NLP must account for discrete language, meaning, fluency, and tokenization; prior work includes gradient-based edits and reusable attack frameworks –.DistilBERT is presented as a lower-cost transformer retaining much of BERT’s performance, [8], and transformer models are used as text-classification baselines,.Source: §2.2 · p. 3
2.3 Robustness Evaluation and Defenses
- Prior defense evaluations warn that apparent protection may fail under stronger adaptive attacks, motivating both FGSM/PGD testing and cautious treatment of preprocessing as partial mitigation.The paper’s text study uses predefined surface perturbations rather than automated search, framing its contribution as an educational, reproducible comparison rather than methodological novelty.Source: §2.3 · pp. 3–4, §2.2 · p. 3
3. Experimental Setup and Evaluation Protocol
The study uses reproducible image and text classification experiments to compare perturbed-input behavior. It evaluates standard white-box attacks on MNIST and predefined surface changes to SMS messages, with separate evaluation protocols for each modality.
3. Experimental Setup and Evaluation Protocol
- The two-modality design contrasts MNIST image attacks with DistilBERT text robustness, implemented in standalone notebooks with fixed seeds and saved plots and tables.Table 1 summarizes experimental configuration and primary outputs.Source: §3 · p. 4, Table 1 · p. 5
3.1 Experiment 1: Image-Based Evasion Attacks on MNIST
- On MNIST, a compact CNN trained on 60,000 images for two epochs is tested with FGSM and 20-step PGD across ε values from 0.00 to 0.30.The protocol measures clean and attacked accuracy, saves clean/adversarial/perturbation examples, and tests 16-level bit-depth reduction on PGD inputs at ε = 0.15 as an illustrative defense baseline.Source: §3.1 · pp. 4–5
3.2 Experiment 2: Text Robustness Evaluation Using DistilBERT on SMS Spam
- The text experiment fine-tunes DistilBERT on 5,574 SMS messages and applies predefined character substitutions, whitespace noise, and a benign suffix to selected spam examples.It records spam probabilities after each perturbation; the changes are non-optimized and intended to preserve an educational, ethics-aware scope.Source: §3.2 · pp. 6–7
3.3 Evaluation Protocol and Reproducibility
- Both experiments use seed 42 and save figures and CSV tables; evaluation separates clean baselines from perturbed inputs and uses classification metrics and per-example probability trajectories.The text study remains local and controlled, without automated search or deployment guidance for bypassing real-world filters.Source: §3.3 · p. 7
4. Results
MNIST showed systematic evasion under gradient-based attacks, while the selected text perturbations caused modest shifts without flipping spam messages to ham. Bit-depth reduction partly recovered MNIST accuracy, and normalization altered some text-probability trajectories without removing the text experiment’s sensitivity.
4.1 Results of the MNIST Image Experiment
- The MNIST classifier reached 98.63% clean test accuracy before perturbation, yet an FGSM example at ε = 0.15 changed a correctly predicted 7 into a confidently predicted 1.The clean confidence was 0.9986 and the adversarial confidence was 0.7069.Source: §4.1 · pp. 7, 9, Figure 4 · p. 9
- At ε = 0.15, FGSM accuracy was 60.20% and PGD accuracy was 32.47%; at ε = 0.30, they were 1.72% and 0.41%, respectively.PGD was more damaging than FGSM across most perturbation levels.Source: Table 2 · p. 8, §4.1 · p. 8
- At ε = 0.15, bit-depth reduction raised PGD accuracy from 32.47% to 46.56%, still far below the 98.63% clean baseline.Across five random starts, PGD accuracy averaged 32.44% and the defense gain ranged from 14.08 to 14.60 points.Source: §4.1 · p. 9
4.2 Results of the Text DistilBERT Experiment
- On the held-out text test split, DistilBERT achieved 98.75% accuracy and a 94.96% F1-score, with 485 ham and 66 spam messages correctly classified.The confusion matrix also showed 2 ham-to-spam errors and 5 spam-to-ham errors.Source: Table 3 · p. 9, §4.2 · pp. 9–10, Figure 5 · p. 10
- Most displayed text examples retained high spam probability through the perturbation sequence, but Example 5’s character substitutions raised its initially near-zero spam probability to near 1.Example 5 was already missed by the clean model, so this perturbation made it more detectable rather than evading detection.Source: §4.2 · p. 10, Figure 6 · p. 11
- Normalization shifted several text probability traces downward, but affected examples generally remained above the spam decision boundary.For Example 5, normalization kept spam probability near 0 at every step, returning the message to a missed classification.Source: §4.2 · p. 10, Figure 7 · p. 11
4.3 Cross-Experiment Summary
- MNIST showed large accuracy losses under gradient attacks, while selected text perturbations caused modest probability shifts and no spam-to-ham flip in displayed examples.The paper characterizes the image experiment as a clear evasion demonstration and the text experiment as a controlled robustness evaluation.Source: §4.3 · p. 11
5. Discussion
The discussion contrasts strong MNIST evasion with a text evaluation whose predefined perturbations produced limited changes, then sets boundaries for interpreting both results and proposes broader text-side testing.
5.1 Interpretation of the MNIST Findings
- MNIST showed stronger degradation under PGD than FGSM at the same perturbation budgets, while bit-depth reduction only partially recovered PGD accuracy at ε = 0.15.The defense remained far below the clean baseline, so the recovery does not establish robustness.Source: §5.1 · p. 11
5.2 Interpretation of the Text Findings
- Under the current perturbation set, spam probabilities generally stayed high after mild substitutions, whitespace changes, and a short benign phrase.The result is bounded to the local model, dataset, and perturbation design; it does not establish broad robustness of transformer spam classifiers.Source: §5.2 · p. 12
5.3 Comparative Summary
- Table 4 summarizes the experiments’ different modality-specific conclusions: clear evasion in the image setting and limited bypass evidence in the displayed text cases.The text-side interpretation is limited to the predefined perturbations and displayed examples.Source: §5.1 · p. 11, §5.3 · p. 12, Table 4 · p. 12
5.4 Limitations
- The study’s scope is bounded by simplified MNIST images, a small SMS dataset of short messages, and manually specified text perturbations.The ethics-aware text pipeline also omits automated attack search and optimization procedures.Source: §5.4 · pp. 12–13
5.5 Next Steps
- A next text-side study could test more messages, report aggregate probability-drop statistics and label flips, and compare models or preprocessing choices.The proposed extension is intended to better balance the image and text components.Source: §5.5 · p. 13
6. Conclusion
The paper concludes that image and text classifiers showed different evasion patterns under the study’s attacks and perturbations. It argues that robustness must be evaluated empirically, and recommends broadening the text-side analysis.
6. Conclusion
- FGSM and PGD reduced MNIST accuracy as perturbation budgets increased, with PGD causing stronger degradation, while predefined text perturbations caused limited confidence changes and weaker bypass evidence.The text results apply to the displayed cases and current perturbation set.Source: §6 · p. 13
- High clean-data performance is not evidence of robustness, and observed robustness behavior is modality-dependent and must be established empirically.In text classification, perturbation effects depend on tokenization, semantics, and attack design.Source: §6 · p. 13
- Future work should evaluate more text messages with summary statistics and examine how preprocessing and model choice affect observed robustness.Source: §6 · p. 13