Source-linked AI summary
Towards Improving Adversarial Training of NLP Models
Jin Yong Yoo, Yanjun Qi
TL;DR
NLP adversarial training is limited by expensive attacks and has been insufficiently evaluated beyond the training attack. The paper introduces A2T, a cheaper word-substitution adversary, and finds improvements in robustness and several broader model properties, with optimal α left for future tuning.
Problem
NLP adversarial-example generation can require expensive combinatorial search and auxiliary neural networks, while the benefits of vanilla adversarial training remain insufficiently investigated.
Method
A2T uses a cheaper gradient-based word-importance ranking and synonym substitutions to generate adversarial examples for training BERT and RoBERTa models.
Results
Across multiple attacks and tasks, A2T-based training improves adversarial robustness, and A2T can also improve standard accuracy and/or cross-domain accuracy.
Takeaways & Limitations
A cheaper adversary can support practical vanilla adversarial training while improving robustness beyond the training attack and benefiting standard model performance.
Takeaways & Limitations
The optimal α is left for future work to tune.
Abstract
from arXiv · showhide
Adversarial training, a method for learning robust deep neural networks, constructs adversarial examples during training. However, recent methods for generating NLP adversarial examples involve combinatorial search and expensive sentence encoders for constraining the generated instances. As a result, it remains challenging to use vanilla adversarial training to improve NLP models' performance, and the benefits are mainly uninvestigated. This paper proposes a simple and improved vanilla adversarial training process for NLP models, which we name Attacking to Training (A2T). The core part of A2T is a new and cheaper word substitution attack optimized for vanilla adversarial training. We use A2T to train BERT and RoBERTa models on IMDB, Rotten Tomatoes, Yelp, and SNLI datasets. Our results empirically show that it is possible to train robust NLP models using a much cheaper adversary. We demonstrate that vanilla adversarial training with A2T can improve an NLP model's robustness to the attack it was originally trained with and also defend the model against other types of word substitution attacks. Furthermore, we show that A2T can improve NLP models' standard accuracy, cross-domain generalization, and interpretability. Code is available at https://github.com/QData/Textattack-A2T .
1 Introduction
Adversarial training augments original NLP examples with adversarial examples, but conventional attacks are expensive to generate. The paper proposes A2T, a cheaper adversary, and reports broader robustness and performance benefits.
- Adversarial training combines original and adversarial examples to improve resistance to adversarial attacks.
- NLP adversarial-example generation can require combinatorial search and up to tens of thousands of model forward passes per example.This cost hinders vanilla adversarial training and leaves its performance benefits unclear.
- A2T uses a cheaper gradient-based word-importance ranking method and synonym replacements from counter-fitted word embeddings.The paper also evaluates A2T-MLM, which uses masked-language-model replacements.
- Adversarial training with A2T and A2T-MLM can improve robustness against attacks that were not used during training.The paper evaluates this cross-attack robustness using multiple attack types.
- A2T can improve standard accuracy, cross-domain generalization, and model interpretability, whereas A2T-MLM tends to hurt standard accuracy and cross-domain generalization.Interpretability is evaluated with LIME and the AOPC metric.
2 Background
Vanilla adversarial training perturbs inputs and augments training data with the resulting examples. NLP attacks use search, transformations, and constraints, but their large search spaces make generation difficult.
- 2.1 Vanilla Adversarial Training in NLP: Vanilla adversarial training augments training data with adversarial examples created by perturbing inputs in the input space.Nonvanilla training instead perturbs non-input representations such as word embeddings.
- 2.1 Vanilla Adversarial Training in NLP: NLP vanilla adversarial training has usually been evaluated only against the attack used to train the model.The paper therefore investigates robustness against different attacks and other performance aspects.
- 2.1 Vanilla Adversarial Training in NLP: The paper examines adversarial training effects on standard accuracy and cross-domain generalization in addition to adversarial robustness.
- 2.1 Vanilla Adversarial Training in NLP: Natural-language adversarial-example generation uses a search algorithm, a transformation module, and constraints intended to preserve semantics and fluency.Synonym substitution is one example of a transformation.
- 2.1 Vanilla Adversarial Training in NLP: With N words and M potential substitutions per word, the synonym-replacement search space contains (M + 1)^N − 1 perturbed inputs.Its exponential size makes exhaustive search impractical.
3 Method: A2T (Attacking to Training)
A2T makes vanilla adversarial training more practical by using a cheaper attack and a workflow that generates adversarial examples before each epoch. Its attack combines gradient-based word ranking, efficient synonym or masked-language-model replacements, and a lower-memory semantic constraint.
- 3.2 A Practical Training Workflow: A2T trains on clean and adversarial examples while generating adversarial data before training epochs rather than between every mini-batch.This workflow addresses the practical difficulty of generating NLP adversarial examples between mini-batch updates.
- 3.2 A Practical Training Workflow: A2T attacks a tunable percentage γ of the training dataset instead of requiring an adversarial example for every clean example.The default experimental setting attacks 20% of the training dataset, and failed attacks are skipped with additional sampling to compensate.
- 3.3 Cheaper Attack for Adversarial Training: Switching TextFooler from deletion-based to gradient-based ranking provides approximately 2× speedup.The runtime comparison attacks 1,000 samples against a BERT model trained on IMDB.
- 3.3 Cheaper Attack for Adversarial Training: A2T ranks word importance with loss gradients, requiring one forward and backward pass instead of additional forward passes for each word.For BERT and RoBERTa, word importance averages the importance of the sub-words constituting each word.
- 3.3 Cheaper Attack for Adversarial Training: A2T uses DistilBERT for semantic textual similarity constraints because it requires 10× less GPU memory than USE and fewer operations.The attack also generates cached top-k nearest-neighbor replacements from counter-fitted word embeddings; A2T-MLM instead uses BERT masked-language-model replacements.
4 Related Work
Related work includes embedding-level adversarial training and methods that enforce consistency over activation regions associated with word substitutions. The paper instead studies input-space vanilla adversarial training more broadly, including robustness beyond the training attack and effects on generalization and interpretability.
- Embedding-level adversarial training: Embedding-level adversarial training perturbs word representations rather than the actual input text.Several prior methods apply gradient-based optimization techniques adapted from computer vision.
- Activation-space defenses: Another approach computes activation-space regions containing texts generated by word substitutions and trains consistent predictions within those regions.Prior work uses axis-aligned hyper-rectangles with Interval Bound Propagation to defend against substitution attacks.
- Input-space vanilla adversarial training: Input-space vanilla adversarial training has been relatively unexplored beyond resistance to the attack used during training.This paper examines robustness across different word-substitution attacks, along with generalization and interpretability.
5 Experiment and Results
Experiments evaluate A2T and related training methods across multiple datasets, models, attacks, robustness measures, generalization settings, and interpretability metrics. A2T generally improves robustness and preserves or improves standard accuracy and cross-domain generalization, while A2T-MLM shows less favorable generalization behavior.
- Adversarial robustness: Adversarial training with the same attack decreases attack success rates by up to 70%, while training with the other A2T variant also reduces attack success rates.A2T training lowers A2T-MLM attack success, and A2T-MLM training lowers A2T attack success.
- Adversarial robustness: BERT-A2T raises accuracy on adversarial examples that fooled BERT-Natural from 0% to over 70% on IMDB and Yelp.The evaluation uses 1000 adversarial examples successfully attacking BERT-Natural.
- Adversarial robustness: Across four datasets and two models, A2T and A2T-MLM lower attack success rates for TextFooler, BAE, PWWS, and PSO in all but five cases.The reductions extend to PWWS and PSO despite their WordNet and HowNet transformations.
- Generalization and accuracy: A2T improves cross-domain accuracy in six of eight cases and generally improves standard accuracy, whereas A2T-MLM tends to hurt both accuracy measures.The authors describe this as a robustness-generalization trade-off for A2T-MLM but not for A2T.
- Interpretability: A2T improves LIME explanation faithfulness as measured by AOPC across three sentiment datasets, with larger gains for BERT than RoBERTa.The evaluation samples 1000 test examples per dataset, generates 1000 perturbed samples per instance, and uses K = 10.
6 Conclusion
The paper presents A2T as a practical vanilla adversarial training process using a fast adversarial attack, and reports improvements in robustness, standard accuracy, and/or cross-domain accuracy.
- A2T is a practical vanilla adversarial training process built around an adversarial attack designed to generate examples quickly.
- A2T improves model robustness against several adversarial attack types reported in prior literature.
- Models trained with A2T achieve better standard accuracy and/or cross-domain accuracy than baseline models.
A.1 A2T and A2T-MLM Attacks
The paper frames A2T and A2T-MLM attacks through four components: goal function, transformation, constraints, and search method. Their attack objective is untargeted and seeks to cause model misprediction.
- A2T and A2T-MLM are decomposed into a goal function, transformation, constraints, and search method.
- The attacks use an untargeted objective because untargeted attacks are generally easier than targeted attacks.
- The goal function maximizes an objective based on the model’s confidence in the label given the input and parameters.
Transformation
A2T and A2T-MLM iteratively replace important words with constrained candidates, using different replacement sources and limits designed to preserve fluency and semantics while controlling search cost.
- Replacement generation: A2T uses counter-fitted word embeddings, whereas A2T-MLM uses a BERT masked language model to propose replacements.
- Replacement generation: Both attacks retain the top 20 proposed replacement words to reduce consideration of less desirable candidates.
- Constraints: Candidate substitutions must preserve part-of-speech consistency and meet a minimum sentence-encoding cosine similarity of 0.9.
- Constraints: The attacks allow at most 10% of words to be replaced to limit semantic changes.
- Search method: A2T ranks word importance using one forward and backward pass, then iteratively searches ranked words until finding an adversarial example or exhausting its query budget.
- Constraints: A2T also requires minimum word-embedding cosine similarity of 0.8 between the original and perturbed texts.
- Search method: Training uses a 200-query budget, while evaluation uses 2000 queries for A2T and A2T-MLM and 5000 for other attacks.