Source-linked AI summary
Towards Crafting Text Adversarial Samples
Suranjana Samanta, Sameep Mehta
TL;DR
The paper addresses the limited development of meaningful adversarial text samples, which are harder to modify than images because vocabulary, grammar, and semantics constrain valid changes. It proposes greedily deleting, inserting, or replacing salient words using classifier contributions and genre-specific candidates, and reports effective crafting on IMDB and Twitter data. The authors also note that the heuristics could be improved and automated further.
Problem
Adversarial text crafting has received little attention because discrete vocabularies and grammatical and semantic constraints make meaningful, human-undetectable modifications difficult.
Method
The method ranks words by classifier contribution and greedily deletes, inserts, or replaces them using synonyms, typos, and genre-specific keywords.
Results
The method produced adversarial samples on IMDB sentiment and Twitter gender-classification datasets, with genre-specific keywords improving crafting quality and Twitter samples remaining semantically close to originals.
Takeaways & Limitations
The approach is best suited to datasets with sub-categories and can create text samples that alter classifier labels while retaining similar semantics and grammar.
Takeaways & Limitations
The word-at-a-time heuristics are not fully automated and may be improved by modifying whole sentences instead.
Abstract
from arXiv · showhide
Adversarial samples are strategically modified samples, which are crafted with the purpose of fooling a classifier at hand. An attacker introduces specially crafted adversarial samples to a deployed classifier, which are being mis-classified by the classifier. However, the samples are perceived to be drawn from entirely different classes and thus it becomes hard to detect the adversarial samples. Most of the prior works have been focused on synthesizing adversarial samples in the image domain. In this paper, we propose a new method of crafting adversarial text samples by modification of the original samples. Modifications of the original text samples are done by deleting or replacing the important or salient words in the text or by introducing new words in the text sample. Our algorithm works best for the datasets which have sub-categories within each of the classes of examples. While crafting adversarial samples, one of the key constraint is to generate meaningful sentences which can at pass off as legitimate from language (English) viewpoint. Experimental results on IMDB movie review dataset for sentiment analysis and Twitter dataset for gender detection show the efficiency of our proposed method.
1 Introduction
The paper addresses the limited work on crafting adversarial text while preserving sentence semantics and grammatical validity. It proposes minimally modifying salient words, especially in datasets with useful sub-categories such as movie genres.
- Research gap: Adversarial text is difficult to craft because discrete vocabularies and grammatical constraints make small, human-undetectable modifications challenging.Unlike image pixels, arbitrary Word2Vec vectors do not necessarily map to valid words.
- Research gap: The proposed method targets datasets with sub-categories, using genre-dependent words whose sentiment contribution can differ across movie genres.For example, “hilarious” may indicate positive sentiment in comedy reviews but negative sentiment in horror reviews.
- Contribution: Adverbs can alter class probabilities without substantially disrupting grammar, making their addition or removal useful for adversarial modification.The paper illustrates this with “fair” versus “extremely fair” and notes that removing words can shift class probability in the opposite direction.
- Contribution: The work crafts adversarial text by deleting, replacing, or inserting words while preserving semantic meaning as much as possible and minimizing alterations.The stated goal is to change classifier behavior without substantially changing the sentence’s meaning.
2 Related Work
Prior adversarial-sample research largely targets images, whereas text requires valid vocabulary, syntax, and meaning preservation. Existing text methods can produce classifier-confusing samples that remain detectable as noisy or unnatural text.
- Image-domain foundations: FGSM generates image perturbations from classifier cost-function gradients and can change a panda image’s predicted label to gibbon.The paper presents FGSM as a fast method for creating image adversarial samples.
- Image-domain foundations: Papernot et al. showed that adversarial samples generated on a separately trained model can also confuse a deployed classifier with little information about that model.Their work introduced and applied the concept of gradient masking to real-world images such as traffic signs.
- Text-domain methods: Liang et al. used character-level CNNs and phrase replacements for adversarial text, but the resulting gibberish could be readily identified by humans.Other approaches modify words through synonyms, typos, punctuation insertion, or deletion, yet some remain easily detectable.
3 Proposed Method
The method ranks words by their contribution to the classifier’s decision, builds candidate replacements using linguistic and genre information, and greedily applies minimal modifications until the label changes.
- Algorithm overview: The algorithm offers three modification types—replacement, insertion, and removal—with the goal of changing the class label using as few alterations as possible.It receives a text sample and a classifier trained for sentiment analysis.
- Word contribution: Words are ranked by their class-contribution factor, prioritizing words whose removal substantially changes the classifier’s class probability.For large texts, the method uses an FGSM-based approximation because exact contribution calculation for every word is time-consuming.
- Candidate pool: For each word, the candidate pool combines synonyms, valid typos, and genre-specific keywords derived from class-distinctive term frequencies.Genre-specific candidates are selected from words distinctive to the opposite class within the relevant genre.
- Modification heuristics: A highly contributing adverb is removed; otherwise, a selected candidate is inserted before an adjective or used to replace the target word.Genre-keyword replacements require matching parts of speech to reduce grammatical corruption and human detectability.
- Stopping rule: The process changes one word at a time in descending contribution order until the original and modified samples receive different labels.The greedy, minimal-change strategy is intended to keep semantics and grammar similar.
- Robustness: After crafting adversarial samples, the existing text classifier is retrained to improve robustness against adversarial attacks.This retraining step follows the adversarial-sample construction procedure.
4 Experimental results
Experiments evaluate adversarial text crafting on IMDB sentiment and Twitter gender classification, measuring accuracy, semantic similarity, and modification counts across model-training configurations.
- Datasets and setup: Experiments use IMDB movie reviews for sentiment analysis and Twitter data for gender classification.The study compares model performance on original and adversarially tainted test data.
- Datasets and setup: IMDB preprocessing identifies Action, Comedy, and Drama genres to select genre-specific distinctive keywords.Reviews are filtered by movie genre before feature extraction and adversarial crafting.
- Models and evaluation: The classifier is a CNN, with separate architectures used for IMDB sentiment classification and Twitter gender prediction.The experiments retrain models using adversarial samples generated from original training data.
- Models and evaluation: Evaluation compares classification accuracy, semantic similarity, and the number of text changes required to create successful adversarial samples.Lower modification counts and higher semantic similarity are treated as desirable evaluation characteristics.
- IMDB results: 0.9164 and 0.9732 are the average semantic similarities with and without genre-specific keywords, respectively, on the IMDB test set.The values are measured with the Spacy toolbox for successfully crafted adversarial counterparts.
- Results across datasets: Genre-specific keywords produce more tainted samples than non-genre-specific keywords for the same number of changes, with the red curve above the blue curve.The paper also reports a larger number of adversarial samples with semantic similarity close to the originals on Twitter compared with TextFool.
5 Conclusion
The paper presents heuristic, greedy word-level text modifications for crafting adversarial samples while preserving semantic meaning and grammar. Figure 5 provides examples from Twitter and IMDB using TextFool and the proposed method.
- The method modifies text one word at a time, selecting each modification greedily to create adversarial samples.The authors note that sentence-level modification could be a stronger alternative.
- Text modifications must preserve both semantic meaning and grammaticality so the resulting samples remain plausible.
- The modification steps are heuristic and could be improved or automated further.
- Figure 5 shows adversarial samples from Twitter and IMDB produced with TextFool and the proposed method.