Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models

Boxin Wang; Chejian Xu; Shuohang Wang; Zhe Gan; Yu Cheng; Jianfeng Gao; Ahmed Hassan Awadallah; Bo Li

Source-linked AI summary

Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models

Boxin Wang, Chejian Xu, Shuohang Wang, Zhe Gan, Yu Cheng, Jianfeng Gao, Ahmed Hassan Awadallah, Bo Li

arXiv:2111.02840v2Nov 4, 2021cs.CLcs.CRcs.LG

TL;DR

Language models remain vulnerable to textual adversarial examples, while existing robustness datasets lack a principled and comprehensive benchmark. AdvGLUE constructs such a benchmark by applying 14 attacks to GLUE tasks and validating examples through human annotation; tested models and defenses perform poorly against it.

Problem
Existing robustness evaluations lack a unified, principled benchmark for comparing language models and adversarial attacks across NLU tasks.
Method
AdvGLUE applies 14 textual adversarial attacks to GLUE tasks and uses crowd-sourcing and curation to obtain reliable adversarial examples.
Results
AdvGLUE reveals a significant performance drop for state-of-the-art models relative to their standard GLUE accuracy, and existing robust training methods provide only incremental improvements.
Takeaways & Limitations
AdvGLUE provides a comprehensive diagnostic benchmark for developing language models with greater adversarial robustness.
Takeaways & Limitations
Some grammatical errors and typos remain after curation because certain attacks intentionally inject or manipulate them, although annotators retain high label consensus.

Abstract

from arXiv · show

Large-scale pre-trained language models have achieved tremendous success across a wide range of natural language understanding (NLU) tasks, even surpassing human performance. However, recent studies reveal that the robustness of these models can be challenged by carefully crafted textual adversarial examples. While several individual datasets have been proposed to evaluate model robustness, a principled and comprehensive benchmark is still missing. In this paper, we present Adversarial GLUE (AdvGLUE), a new multi-task benchmark to quantitatively and thoroughly explore and evaluate the vulnerabilities of modern large-scale language models under various types of adversarial attacks. In particular, we systematically apply 14 textual adversarial attack methods to GLUE tasks to construct AdvGLUE, which is further validated by humans for reliable annotations. Our findings are summarized as follows. (i) Most existing adversarial attack algorithms are prone to generating invalid or ambiguous adversarial examples, with around 90% of them either changing the original semantic meanings or misleading human annotators as well. Therefore, we perform a careful filtering process to curate a high-quality benchmark. (ii) All the language models and robust training methods we tested perform poorly on AdvGLUE, with scores lagging far behind the benign accuracy. We hope our work will motivate the development of new adversarial attacks that are more stealthy and semantic-preserving, as well as new robust language models against sophisticated adversarial attacks. AdvGLUE is available at https://adversarialglue.github.io.

1 Introduction

AdvGLUE addresses the lack of a unified, principled benchmark for evaluating language-model robustness against textual adversarial attacks. It combines broad attack coverage with human validation and evaluates models and defenses across GLUE tasks.

Existing language models can be fooled by human-imperceptible perturbations, creating security concerns for systems built on them.
Different robustness studies use inconsistent adversary setups, making model comparisons and vulnerable-attack identification difficult.
AdvGLUE is a multi-task benchmark designed for principled robustness evaluation of language models.
The benchmark covers word-level, sentence-level, and human-written adversarial phenomena across GLUE tasks.
AdvGLUE systematically annotates adversarial examples and evaluates state-of-the-art language models and defense methods.

2 Related Work

Prior robustness resources include programmable evaluation toolkits and human-crafted benchmark datasets. Toolkits offer broad transformations but have quality challenges, whereas human-crafted datasets provide control at the cost of scalability and comprehensiveness.

Evaluation toolkits integrate ad hoc transformations and programmable APIs, but their input-transformation quality can be difficult to guarantee.
AdvGLUE’s benchmark statistics organize word-level, sentence-level, distraction-based, and human-crafted perturbations across GLUE tasks.
Human-crafted adversarial datasets can control example quality and validity, but human annotation limits their scalability and comprehensiveness.

3 Dataset Construction

AdvGLUE constructs a unified robustness benchmark by applying diverse adversarial perturbations to selected GLUE tasks, incorporating human-crafted examples, and filtering generated samples for semantic validity and annotation reliability.

Task and data selection: AdvGLUE covers five GLUE task groups, excluding tasks lacking well-defined automatic attacks or sufficient attack data.The included tasks are SST-2, QQP, and NLI tasks MNLI, RTE, and QNLI.
Task and data selection: The benchmark uses GLUE training data and evaluation metrics, enabling models to be compared on benign and adversarial test sets after one-time training.AdvGLUE also supports robustness analysis across attack levels through its diagnostic report.
Adversarial perturbations: Four representative word-level attacks and one compositional attack generate perturbations using typos, embedding similarity, contextual prediction, knowledge bases, or combined optimization.TextBugger, TextFooler, BERT-ATTACK, SememePSO, and CompAttack represent these strategies; the attacks use surrogate models for perturbation search.
Human-crafted examples: Human-crafted examples add capability tests and human-in-the-loop or distraction-based adversarial data to the automatically generated perturbations.CheckList examples test temporal reasoning and negation, ANLI uses iterative human attacks, and AdvSQuAD adds distracting sentences in QNLI format.
Data curation: Most attacks have Curated ASR below 11% and filter rates above 85%, indicating that many generated examples are invalid or ambiguous after curation.Around 60–80% are filtered for low transferability or high word modification, while additional samples are removed for low human agreement or semantic label changes; post-curation Fleiss Kappa is near 0.6.
Data curation: TextBugger is reported as the most effective and valid attack, achieving the highest Curated ASR and Curated Fleiss Kappa across tasks.The curation phase also provides a common standard and systematic human annotations for comparing attack methods.

4 Diagnostic Report for Language Models

AdvGLUE evaluation shows that strong language models remain highly vulnerable to adversarial attacks, while robust training methods provide only limited improvements. The diagnostic report identifies human-crafted and distraction-based perturbations as particularly challenging.

A performance gap as large as 55% separates benign GLUE and AdvGLUE performance for SMART (BERT) on average score.
DeBERTa (Large) and ALBERT (XXLarge) achieve the highest average AdvGLUE scores among the tested language models.
Curated adversarial examples transfer across architectures, with ELECTRA (Large) scoring even lower than RoBERTa (Large) on AdvGLUE.
BERT performs below random-guess accuracy on every AdvGLUE task.
SMART and FreeLB improve RoBERTa robustness, but existing robust training methods yield only incremental gains.SMART (RoBERTa) improves RoBERTa (Large) by over 3.71% on average, while also improving benign accuracy.
Models are most vulnerable to human-crafted examples and, among sentence-level perturbations, to distraction-based attacks.The report associates human-crafted difficulty with phenomena such as numerical reasoning, negation, and coreference resolution.

5 Conclusion

The paper introduces AdvGLUE as a curated, multi-task benchmark built from 14 adversarial attacks and human validation to evaluate language-model robustness. Its results show that modern models remain vulnerable, while related defense approaches face important limitations under unseen attacks and model constraints.

AdvGLUE is a principled multi-task benchmark for evaluating and analyzing language-model and robust-training-method robustness.
The benchmark systematically applies 14 adversarial attacks to GLUE tasks and uses crowd-sourcing to ensure adversarial-example quality and validity.
Modern language models perform poorly on AdvGLUE, indicating that vulnerabilities to adversarial attacks remain unsolved.
AdvGLUE is intended to serve as a comprehensive diagnostic benchmark for developing more robust models.
Textual adversarial attacks span word-level transformations, sentence-level attacks, and human-crafted samples.
Existing defenses include adversarial training, interval bound propagation, and randomized smoothing, but each has stated limitations.Adversarial training may be less effective against unseen attacks, IBP relies on strong architectural assumptions, and randomized smoothing adapts continuous-noise ideas to text with synonym words.

A.3 Task Descriptions, Statistics and Evaluation Metrics

This section documents the benchmark’s task and attack glossaries, task definitions, label distributions, and model-performance reporting.

Statistics and metrics: GLUE and AdvGLUE label distributions and evaluation metrics are reported in Table 8.
Attack glossary: Table 6 provides explanations and examples for word-level and sentence-level adversarial attacks.
Attack glossary: Table 7 provides explanations and examples for human-crafted adversarial attacks.
Task descriptions: SST-2 predicts positive or negative sentiment from movie-review sentences.
Task descriptions: QQP classifies whether two Quora questions are semantically equivalent.
Task descriptions: MNLI, QNLI, and RTE evaluate textual entailment, answer containment, and sentence-pair relationships, respectively.

A.4 Implementation Details of Adversarial Attacks

This section specifies implementation choices for the adversarial attacks, including perturbation strategies, optimization settings, templates, and benchmark examples.

Word-level attacks: TextBugger uses character-level insertions, deletions, keyboard substitutions, visually similar substitutions, and swaps to create small perturbations.The first four strategies use word edit distance 1, while character swaps are limited to distance 2.
Word-level attacks: TextFooler uses cosine-similarity thresholds of 0.8 for sentiment tasks and 0.7 for other tasks, with 50 synonyms per word.
Word-level attacks: BERT-ATTACK uses 48 candidate words and a cosine-similarity threshold of 0.4 to filter antonyms from synonym dictionaries.
Optimization-based attacks: SememePSO uses inertia weights and movement probabilities ranging from 0.2 to 0.8, with population size 60 for every task.
Optimization-based attacks: CompAttack minimizes perturbation magnitude plus a weighted attack objective while constraining perturbations to predefined typo, knowledge, or contextualized-embedding spaces.The objective is L(e*) = ||e*||p + c · g(x′), where c balances attack goal and attack cost.
Sentence-level attacks: SCPN uses ten frequent ParaNMT-50M templates, while AdvFever applies 23 entailment-preserving templates to produce semantically equivalent sentences.

A.5 Examples of AdvGLUE benchmark

The benchmark examples demonstrate that adversarial inputs at different perturbation levels can change predictions across several surrogate models.

Benchmark examples: Examples with different perturbation levels successfully change predictions for BERT, RoBERTa, and a RoBERTa ensemble.

A.6 Fine-tuning Details of Large-Scale Language Models

This section records the computational setup and fine-tuning configurations used for the evaluated language models and robust-training methods.

Compute setup: Experiments use a GPU cluster with 8 V100 GPUs and 256GB memory.
Language-model fine-tuning: BERT, ELECTRA, RoBERTa, T5, and DeBERTa are trained with task-specific epoch, batch-size, and learning-rate settings.
Language-model fine-tuning: T5 converts GLUE tasks into generation tasks using templates from its original paper.
Language-model fine-tuning: ALBERT uses default hyperparameters, with maximum training steps varying across SST-2, MNLI, QNLI, QQP, and RTE.The listed maximum steps are 20935, 10000, 33112, 14000, and 800, respectively.
Robust-training methods: SMART searches parameter grids and reports the best trained-model performance, while FreeLB tests every official parameter combination and selects the best.

A.7 Human Evaluation Details

Human evaluation used qualified annotators trained on GLUE tasks, then filtered adversarial examples through transferability, agreement, and semantic-preservation criteria.

Human Training: Human accuracy was comparable to prior work, indicating that most selected annotators understood the GLUE tasks well.Annotators were tested on 100 non-overlapping GLUE development-set samples for each task.
Human Filtering: Around 60–80% of examples were filtered for low transferability and high word modification rates.These criteria removed examples unlikely to provide suitable adversarial evaluations.
Human Filtering: Among remaining samples, around 30–40% were removed for low human agreement and around 20–30% for semantic changes that altered labels.The filtering process therefore addressed both annotation reliability and preservation of the original task label.
Annotation Instructions: Annotation instructions for MNLI were provided for both the training and filtering phases.Additional instructions and task-specific FAQs were made available online.

A.8 Discussion of Limitations

The discussion identifies computational coverage and surrogate-model construction as limitations, while positioning AdvGLUE as a supplementary robustness diagnostic alongside GLUE.

Limitations: The evaluation could not comprehensively cover all existing language models because of computational-resource constraints.The authors invite researchers to submit additional models through the leaderboard.
Future Work: The authors leave adversarial robustness of large-scale autoregressive models in few-shot settings for future work.This marks a scope boundary beyond the evaluated setting.
Limitations: AdvGLUE generates examples against BERT and RoBERTa surrogate models, raising concerns about fairness if models exploit errors unlike those surrogates.The authors argue that systematic data curation and high transferability can address this concern.
Scope: AdvGLUE is intended mainly for robustness evaluation and should complement, rather than replace, standard GLUE evaluation.Using both benchmarks distinguishes generalization from robustness.

B Data Sheet

The data sheet describes AdvGLUE’s motivation, provenance, documentation, distribution, licensing, and update mechanisms.

Purpose: AdvGLUE addresses the lack of a unified adversarial setup and limited coverage of linguistic transformations in existing robustness methods.The stated motivation includes evaluating transformations such as typos, synonymous substitutions, and paraphrases under a common standard.
Creators: The dataset was created by the University of Illinois at Urbana-Champaign and Microsoft Corporation.
Documentation: The project directs readers to the paper and website for further dataset information.
Distribution: The dev set is public, while the hidden test set is evaluated through an automatic CodaLab submission API.The website hosts the development data and CodaLab hosts the hidden test set.
License and Updates: AdvGLUE is distributed under the CC BY-SA 4.0 license and may be updated to correct errors or add tasks.The authors state that updates will be reflected on the website and leaderboard.
Contributions: Contributors can contact the dataset creators by email to propose extensions or additions.