PaperlayerView exact revision on arXiv

Source-linked AI summary

Contextualized Perturbation for Textual Adversarial Attack

Dianqi Li, Yizhe Zhang, Hao Peng, Liqun Chen, Chris Brockett, Ming-Ting Sun, Bill Dolan

arXiv:2009.07502v2cs.CL

TL;DR

NLP adversarial-example generation must induce model errors while preserving text quality, but heuristic rules are context-agnostic and can produce unnatural outputs. CLARE uses context-aware mask-then-infill with three perturbations, and experiments and human evaluation report better attack success rate, textual similarity, fluency, and grammaticality than baselines.

  • Problem

    Heuristic adversarial-text generation is context-agnostic, limiting natural, fluent, and grammatical outputs while adversarial examples remain useful for evaluating model robustness.

  • Method

    CLARE applies context-aware mask-then-infill perturbations using a pretrained masked language model, with Replace, Insert, and Merge actions.

  • Results

    CLARE outperforms baselines in attack success rate, textual similarity, fluency, and grammaticality across multiple NLP tasks and human evaluation.

  • Takeaways & Limitations

    The three perturbation patterns provide varied-length outputs and enable attacks with fewer edits while preserving textual quality.

  • Takeaways & Limitations

    The evaluation assumes black-box access to the victim classifier and examines only the matched dataset set because the mismatched set is easier to attack.

Abstract

from arXiv · show

Adversarial examples expose the vulnerabilities of natural language processing (NLP) models, and can be used to evaluate and improve their robustness. Existing techniques of generating such examples are typically driven by local heuristic rules that are agnostic to the context, often resulting in unnatural and ungrammatical outputs. This paper presents CLARE, a ContextuaLized AdversaRial Example generation model that produces fluent and grammatical outputs through a mask-then-infill procedure. CLARE builds on a pre-trained masked language model and modifies the inputs in a context-aware manner. We propose three contextualized perturbations, Replace, Insert and Merge, allowing for generating outputs of varied lengths. With a richer range of available strategies, CLARE is able to attack a victim model more efficiently with fewer edits. Extensive experiments and human evaluation demonstrate that CLARE outperforms the baselines in terms of attack success rate, textual similarity, fluency and grammaticality.

1 Introduction

CLARE addresses the context-insensitivity of heuristic adversarial-text generation with a mask-then-infill model using three contextualized perturbations. Experiments and human evaluation show improvements in attack success rate, textual similarity, fluency, and grammaticality.

  • Adversarial examples perturb NLP inputs to trigger model errors while keeping the output close to the original.They also support analysis and interpretation of model decisions.
  • Rule-based methods are agnostic to context, limiting their ability to produce natural, fluent, and grammatical outputs.
  • CLARE uses a mask-then-infill procedure with a pretrained masked language model to modify inputs contextually.It detects model vulnerabilities, masks input text, and fills the masks with alternatives.
  • Replace, Insert, and Merge respectively replace a token, insert a new one, and merge a bigram, enabling outputs of varied lengths.This contrasts with token-replacement methods limited to outputs having the same length as their inputs.
  • CLARE outperforms baselines on attack success rate, textual similarity, fluency, and grammaticality across classification, inference, and paraphrase tasks.The evaluation attacks finetuned BERT models and includes human evaluation.

2 CLARE

CLARE generates adversarial examples through contextualized mask-then-infill perturbations, selecting fluent alternatives that confuse a black-box victim model while preserving similarity. Its Replace, Insert, and Merge actions support varied output lengths and iterative, low-edit attacks.

  • Contextualized infilling: CLARE applies local mask-then-infill actions using a pretrained masked language model and unmasked context.Each action masks text around a position, then fills the mask with an alternative token.
  • Perturbation actions: Replace substitutes one token, Insert adds one token after a position, and Merge replaces a bigram with one token.Insert increases sequence length by one, whereas Merge decreases it by one; Merge can function as token deletion.
  • Candidate selection: Candidate tokens must be fluent, preserve similarity, and reduce the victim model’s probability for the gold label.Higher fluency and similarity thresholds improve those properties but can reduce attack success, so CLARE trades them off.
  • Sequential attack: CLARE constructs actions for all positions in parallel, ranks them by their ability to confuse the victim model, and applies them iteratively.At each position only one action is selected, and iteration stops when an adversarial example is found or the action limit is reached.
  • Sequential attack: At input length n, CLARE considers 3n actions, assuming candidate token sets are nonempty, and seeks adversarial examples with minimal modifications.The algorithm starts from the original text and outputs an adversarial example after applying selected perturbations.
  • Comparison: Unlike BERTAttack and BAE, CLARE combines three perturbations, accounts for infilled tokens when selecting positions, and produces outputs of varied lengths.BERTAttack only replaces tokens, while BAE inserts only next to a replaced token.

3 Experiments

CLARE is evaluated against word-level black-box attack baselines across multiple datasets and NLP tasks using attack effectiveness, modification, fluency, grammaticality, and similarity metrics. It consistently achieves stronger overall performance, better trade-offs, and favorable human judgments.

  • Experimental setup: CLARE is evaluated on text classification, natural language inference, and sentence paraphrase tasks using several benchmark datasets and finetuned victim models.The experiments include DBpedia, SST-2, MRPC, QQP, Yelp Reviews, AG News, MNLI, and QNLI; sentence-pair attacks target the longer sentence.
  • Evaluation metrics: The evaluation measures attack success rate, modification rate, perplexity, grammar errors, and textual similarity.Higher attack success rate and similarity are better, while lower modification rate, perplexity, and grammar errors indicate preferable outcomes.
  • Automatic results: More than 5.4% higher attack success rate and fewer average modifications distinguish CLARE from BERTAttack, the strongest baseline.The paper attributes this result to combining Replace, Insert, and Merge at any input position.
  • Automatic results: CLARE consistently outperforms competing methods in perplexity, grammatical errors, and similarity, including a similarity advantage exceeding 0.02 over all baselines.BERTAttack has similar perplexity to language-model-augmented TextFooler on three of four datasets, whereas CLARE performs better than both.
  • Trade-off analysis: Trade-off curves show that CLARE combines higher attack success with the least similarity drop and a better attack-success-rate–perplexity balance.The larger area under a curve represents the better trade-off between the corresponding metrics.
  • Human evaluation: In human evaluation, 56% of responses preferred CLARE for similarity versus 16% for TextFooler, while fluency and grammaticality preferences were 42% versus 9%.The evaluation used 300 AG News instances successfully attacked by both models and collected paired judgments.
  • Human evaluation: CLARE’s label consistency was 68% with a 95% CI of (66%, 70%), slightly below TextFooler’s 70% with a 95% CI of (68%, 73%).The authors relate this difference to overlapping AG News categories, reflected in 71% label consistency for original inputs.

4 Analysis

The analysis examines CLARE’s perturbation strategies, candidate constraints, masked language model backbones, attack patterns, and adversarial training effects. Results show trade-offs among attack success, fluency, similarity, efficiency, and downstream robustness.

  • Perturbation strategies: Combining Replace, Insert, and Merge achieves the best performance with the fewest modifications, while Merge-only underperforms because it is restricted to bigram noun phrases.Insert-only performs best among individual strategies, with Replace-only a close second.
  • Attack efficiency: REPLACEONLY outperforms BERTAttack across the board, presumably because BERTAttack ignores the tokens that will be infilled when selecting attack positions.The comparison targets attack-order efficiency.
  • Constraint ablations: Removing the textual-similarity constraint decreases similarity but improves other aspects, whereas removing the masked language model constraint improves success rate while worsening perplexity, grammaticality, and similarity.The ablations expose explicit trade-offs between attack success and output quality.
  • Masked language models: Distilled RoBERTa is the fastest masked language model backbone without losing performance; the authors conjecture that attacking a BERT-based victim with BERT information is less efficient.Speed is compared across CLARE implementations using different masked language models.
  • Part-of-speech analysis: 64% of Replace actions target nouns, while Insert actions frequently place tokens within noun-phrase bigrams, consistent with topic-classification predictions relying on characteristic noun words or phrases.The analysis uses AG News and reports the most frequent part-of-speech patterns.
  • Adversarial training: In three of four cases, adversarial training decreases attack success rate by more than 10.3% and increases required modifications by more than 0.8; TextCNN with 10% data is the exception.With full training data, adversarial training slightly decreases test accuracy by 0.2% and 0.5%; under low-resource training, TextCNN gains 2.0% absolute accuracy.
  • Adversarial training: These results suggest CLARE can improve downstream robustness with a negligible accuracy drop.The claim summarizes the adversarial-training experiments.

5 Related Work

Related work includes character-, sentence-, and word-level textual attacks, along with masked-language-model text generation. CLARE differs by combining contextualized perturbations to preserve fluency and support variable-length attacks.

  • Textual adversarial attack: Character-based attacks use misspellings but can often be defended by a spell checker.The related-work discussion cites several character-level attack models and defenses.
  • Textual adversarial attack: Sentence-level models introduce sophisticated token or phrase perturbations but generally struggle to maintain semantic similarity with the original inputs.This contrasts sentence-level generation with similarity preservation.
  • Textual adversarial attack: Word-level models commonly use synonym substitution rules to preserve semantic meaning.The discussion places CLARE among recent word-level approaches while distinguishing its contextualized strategy.
  • Textual adversarial attack: CLARE’s contextualized Replace, Insert, and Merge perturbations produce more fluent outputs and allow variable-length adversarial examples.Its three perturbation types distinguish it from the related attack methods discussed.
  • Text generation with BERT: Masked language models have been applied to tasks ranging from lexical substitution to non-autoregressive generation.The paper situates CLARE’s masked-language-model foundation within broader text-generation research.

6 Conclusion

The paper concludes that CLARE uses pretrained masked-language-model knowledge and three contextualized perturbations to generate natural adversarial examples. It reports higher attack success with fewer edits and human-evaluated gains in similarity, fluency, and grammaticality.

  • Conclusion: CLARE uses contextualized knowledge from pretrained masked language models to generate natural, fluent, and grammatical adversarial examples.The conclusion identifies contextualized language-model knowledge as the basis of the method.
  • Conclusion: Replace, Insert, and Merge produce outputs of varied lengths while achieving higher attack success rates than baselines with fewer edits.The conclusion summarizes the method’s flexibility and attack efficiency.
  • Conclusion: Human evaluation shows significant advantages for CLARE in textual similarity, fluency, and grammaticality.These are the human-evaluation dimensions reported in the conclusion.

A.1 Additional Experiment Details

The additional experiment details specify model implementations, evaluation metrics, preprocessing, dataset handling, and test-set construction. They also document the computational and sampling settings used for evaluation.

  • Model implementation: CLARE uses distilled RoBERTa as its main masked-language-model backbone for fast inference, while baselines use their authors’ open-source implementations.The implementation uses Hugging Face Transformers and PyTorch; distilled RoBERTa has 82M parameters.
  • Evaluation metric: The local similarity function uses USE with a window size of 15, while global Sim measures similarity between the two complete texts.All baselines use the local similarity function during candidate construction, and human evaluation is emphasized.
  • Data processing: Punctuation is retained during victim-model training and attacking, unlike TextFooler preprocessing, because removing punctuation was found to make the victim model vulnerable.The paper explicitly identifies this as an empirical preprocessing difference.
  • Data processing: For GLUE datasets, the dev set is used as test data; sentence-pair attacks target the longer sentence while excluding tokens shared by both sentences.The exclusion preserves shared keywords relevant to inference tasks.

A.2 Additional Results

The section reports CLARE’s adversarial-example results across four benchmark datasets and states that it achieves the best attack success rate compared with baselines.

  • CLARE is evaluated on DBpedia, SST-2, MRPC, and QQP datasets.The section references dataset statistics in Table 9 and model results in Table 10.
  • CLARE achieves the best attack success rate compared with all baselines.

A.3 Human Evaluation Details

Human evaluation compares CLARE and TextFooler adversarial examples for meaning similarity, fluency, grammaticality, and label consistency. The evaluation uses paired judgments, selected crowd workers, and bootstrap confidence intervals.

  • Human evaluation samples 300 AG News sentences attackable by both CLARE and TextFooler, pairing their adversarial examples with each original and gold label.
  • Participants compare paired outputs for meaning similarity, fluency, and grammaticality, then annotate examples for label consistency.Higher label consistency indicates better preservation of human predictions while causing victim-model errors.
  • Each output pair is judged by 5 crowd-sourced evaluators, with the study using the top-ranked 30% of U.S. workers and providing task guidance.
  • Statistical significance is assessed from 95% confidence intervals using 10K paired bootstrap replications.

A.4 Qualitative Samples

The paper presents qualitative adversarial examples from CLARE and TextFooler across five datasets, alongside human-evaluation figures for pairwise quality and label consistency.

  • Tables 11 and 12 show adversarial examples produced by CLARE and TextFooler on AG News, DBpedia, Yelp, MNLI, and QNLI.
  • The qualitative tables identify each original example’s gold label and visually distinguish CLARE’s Replace, Insert, and Merge perturbations.Replace, Insert, and Merge use italic red, bold blue, and sans serif yellow formatting, respectively.
  • Figure 3 presents pairwise human comparisons of text similarity, fluency, and grammaticality.
  • Figure 4 presents the label consistency task used in human evaluation.
Loading 2009.07502v2…