Source-linked AI summary
Adversarial NLI: A New Benchmark for Natural Language Understanding
Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, Douwe Kiela
TL;DR
Existing NLU benchmarks struggle to keep pace with model improvement, motivating a benchmark that can last longer. The paper uses adversarial human-and-model-in-the-loop collection across repeated rounds, where annotators target model vulnerabilities. Training on the resulting data achieves state-of-the-art performance on existing NLI benchmarks while producing increasingly difficult test sets.
Problem
Existing NLU benchmarks struggle to keep pace with model improvement, raising the question of whether a large benchmark dataset can last longer.
Method
The paper uses human annotators as adversaries who generate hypotheses intended to fool an NLI model, repeating the process across rounds with new models and test sets.
Results
Training on the new dataset yields state-of-the-art performance on existing NLI benchmarks, while successive rounds produce more robust models and more difficult test sets.
Takeaways & Limitations
The iterative procedure creates a moving target for NLU rather than a static benchmark, while the dataset supports analysis of model weaknesses using annotator explanations and inference labels.
Takeaways & Limitations
The training set includes examples that were not verified, so learning from noisy and potentially unverified data remains an additional challenge.
Abstract
from arXiv · showhide
We introduce a new large-scale NLI benchmark dataset, collected via an iterative, adversarial human-and-model-in-the-loop procedure. We show that training models on this new dataset leads to state-of-the-art performance on a variety of popular NLI benchmarks, while posing a more difficult challenge with its new test set. Our analysis sheds light on the shortcomings of current state-of-the-art models, and shows that non-expert annotators are successful at finding their weaknesses. The data collection method can be applied in a never-ending learning scenario, becoming a moving target for NLU, rather than a static benchmark that will quickly saturate.
1 Introduction
Existing NLU benchmarks can become obsolete quickly, while high benchmark scores may reflect spurious dataset patterns rather than robust language understanding. ANLI addresses these concerns with iterative adversarial collection that makes models and test sets progressively harder.
- NLU benchmarks struggle to keep pace with rapid model improvement, motivating benchmarks designed to remain challenging longer.
- High-performing NLU models may exploit spurious statistical patterns instead of learning flexible, generalizable meaning.
- ANLI uses iterative human-and-model-in-the-loop collection, adding examples that current models misclassify and retraining stronger models across rounds.
- Across three progressively harder rounds, the procedure produces a moving target for NLU rather than a static benchmark that eventually saturates.
- The benchmark introduces a human-and-model-in-the-loop NLI dataset and reports state-of-the-art performance on popular NLI benchmarks.
2 Dataset collection
ANLI collects difficult NLI examples by having human annotators adversarially target model weaknesses, verifying their labels, and repeating the process with stronger models and broader contexts. The resulting benchmark uses verified examples for development and testing while exposing remaining challenges from unverified training data.
- 2.1 HAMLET: HAMLET asks human annotators to write hypotheses that fool a base NLI model for a specified target label and explain the suspected misclassification.
- 2.1 HAMLET: Misclassified examples are checked by additional human verifiers, with disagreement resolved by a third verifier or the example discarded.
- 2.1 HAMLET: Each round constructs development and test sets solely from verified-correct examples, restricting test data to exclusive annotators and balanced label classes where applicable.
- 2.4 Round 2: The collection progresses from BERT to RoBERTa-based adversaries and then expands contexts across Wikipedia, news, fiction, spoken, causal, and procedural domains.
- 2.6 Comparing with other datasets: ANLI is designed to be harder than prior datasets and uses longer, multi-sentence contexts than SNLI.
- 2.6 Comparing with other datasets: Hypothesis-only models perform poorly on ANLI test sets, while exclusive annotators help measure and reduce overfitting to individual writing-style biases.
- 2.6 Comparing with other datasets: The training set includes model-correct examples that were not verified, creating a challenge involving noisy and potentially unverified data.
3 Dataset statistics
ANLI grows across three rounds while model error rates decrease overall, indicating that later examples are harder to elicit. Round 3 broadens domains, slightly raising overall error while reducing error on Wikipedia contexts.
- 19k examples were collected in Round 1, around 47k in Round 2, and over 103k in Round 3.
- Table 3 reports performance on ANLI round and aggregate test sets, exclusive-worker subsets, SNLI, and MNLI development data.
- Model error rates decrease across rounds, falling from roughly 1 in 5 verified errors in Round 1 to less than 1 in 10 overall.
- Round 3’s diverse-domain contexts slightly increase overall error relative to the preceding round, while Wikipedia-context error decreases substantially.
- Average attempts and time per verified error increase in later rounds, demonstrating that the rounds become increasingly difficult.
4 Results
The results show that adversarially collected data produces increasingly difficult test sets, improves robustness across model architectures, and supports strong performance on standard and stress-test benchmarks. Training on ANLI also preserves performance on existing tasks while revealing model- and hypothesis-specific weaknesses.
- Main results: Base models perform poorly on their corresponding ANLI test sets, with rounds 2 and 3 falling below the 33% chance level.For round 1, the entire test set was designed to be misclassified; later-round performance remained below chance, suggesting broadly applicable weaknesses within model classes.
- Main results: Later ANLI rounds are increasingly difficult, with round 3 yielding lower performance than round 2 and round 2 lower than round 1 across architectures.
- Main results: Training on more ANLI rounds improves performance and mitigates adversarial vulnerabilities across all evaluated model architectures.Adding more ordinary NLI data alone would not provide the same robustness to adversarial attacks.
- Main results: RoBERTa reaches 91.0 and 90.7 on MNLI matched and mismatched development sets, compared with 90.2 for both reported baselines, and 92.9 on SNLI versus 91.6.
- Robustness and generalization: Adding ANLI training data does not harm SNLI or MNLI performance, and exclusive-annotator test results show negligible differences from other test examples.
- Data efficiency: Adversarial data improves performance at equal training sizes, with later-round data more data-efficient and verified examples more valuable than unverified examples.The verified-versus-unverified advantage is especially pronounced in later rounds, where unverified examples approach random usefulness.
- Stress tests: ANLI-trained models outperform prior models on common NLI stress tests, with RoBERTa strongest on SNLI-Hard and high-80s accuracy on several stress-test subsets.The largest reported benefits are for antonym, numerical reasoning, negation, and word-overlap tests.
- Hypothesis-only results: Hypothesis-only models perform poorly on ANLI, while RoBERTa trained only on roughly 163k ANLI examples matches BERT trained on 943k SNLI+MNLI examples at approximately 86 on MNLI.In rounds 2 and 3, RoBERTa is not much better than a hypothesis-only prior, indicating exceptionally difficult test sets.
5 Linguistic analysis
The linguistic analysis categorizes the inference types used to fool models across three rounds and tracks how these strategies change as the dataset evolves. Numerical and quantitative reasoning dominates early rounds, while its use declines in round 3 as standard inferences increase.
- Analysis setup: The analysis manually annotates 500 development examples from each round to track model errors and the inference types used to fool models.
- Inference ontology: The ontology includes numerical and quantitative, reference and names, standard, lexical, and tricky inference types.
- Round-level patterns: Numerical and quantitative reasoning appears in 38% of round 1 and 32% of round 2 development examples, but only 17% in round 3.Most numerical cases involve cardinal numbers referring to dates and ages.
- Round-level patterns: Reference-and-name inferences occur in about 10% of rounds 1 and 3, reaching 20% in round 2, with coreference prominent.
- Round-level patterns: Standard inference types increase over rounds as writers develop examples intended to stump the models.
6 Related work
Prior work documents spurious-pattern exploitation and adversarial failures across language tasks, while dynamic-dataset methods iteratively probe or revise models. This paper combines human and model strategies to create new NLI examples in a potentially never-ending loop.
- Biases and adversarial failures: Earlier NLI studies found that hypothesis-only models exploit artifacts, lexical substitutions can break systems, and models struggle with quantifiers and superficial syntax.
- Biases and adversarial failures: Related work shows that question- or passage-only models can perform well, while adversarial sentences can sharply reduce question-answering performance.
- Dynamic datasets: AFLite uses adversarial filtering to iteratively probe and improve models, whereas counterfactual NLI augmentation uses humans to find problems with one model.
- Dynamic datasets: This work combines human and model-based strategies iteratively to create completely new examples in a potentially never-ending training loop.
- Related training paradigms: Human-and-model-in-the-loop training has also been explored in gamified collaborative learning, security, dialogue safety, and continuously refined question answering.
- Scope and distinction: The ANLI dataset is distinct from abductive NLI, which uses the αNLI name, and from other adversarially constructed datasets.
7 Discussion & Conclusion
ANLI uses adversarial human-and-model interaction to expose model weaknesses, produce harder successive test sets, and support broader analysis of NLI robustness. The benchmark achieves state-of-the-art results on existing NLI benchmarks while retaining scope for improvement and extension.
- Non-expert annotators were remarkably creative at finding and exploiting model weaknesses in a gamified human-and-model-in-the-loop setting.
- As rounds progressed, models became more robust while the corresponding test sets became more difficult.
- Training on the collected data yielded state-of-the-art performance on existing NLI benchmarks.
- ANLI includes annotator explanations for model errors and inference labels for development examples, enabling fine-grained studies of NLI performance.
- Training examples were not individually verified, although development and test examples were verified as correct.
- The procedure is model-agnostic, extendable to other classification and ranking tasks, but its application to generative cases is less clear.
- If ANLI saturates quickly, another collection round can be added to create a new challenge.
A Performance on challenge datasets
The paper evaluates ANLI-trained models on hard NLI challenge datasets designed to expose learned biases and stress specific reasoning abilities.
- SNLI-Hard contains examples that hypothesis-only models label incorrectly, while NLI stress tests target phenomena including antonyms and negation.
B Further linguistic analysis
ANLI contains more hard linguistic phenomena than FEVER and SNLI, broadly resembles MultiNLI on several surface features, and emphasizes numerals and time terms. Genre and context-source differences may explain part of this pattern.
- FEVER and SNLI generally contain many fewer hard linguistic phenomena than MultiNLI and ANLI.
- ANLI and MultiNLI have roughly the same percentages of hypotheses exceeding twenty words or containing negation, ‘or’, and modals.
- ANLI contains more cardinal numerals and time terms than MultiNLI.
- The higher incidence of numerals and time terms may reflect Wikipedia-derived contexts whose opening lines frequently include dates.
- A1 and A2 have similar tag incidences, whereas A3 has the highest incidence of every tag except numbers and time.
C Dataset properties
The dataset-properties section directs readers to tables and figures covering label distributions, annotation effort, sequence lengths, example types, and verifier agreement.
- Table 9 reports the label distribution across the collected data.
- Figures 4 and 5 summarize tries and time per good verified example across the three rounds.
- Figure 6 shows token-count histograms for contexts and hypotheses across the three rounds.
- Figure 7 shows the proportions of different collected-example types across rounds.
- Table 10 reports verifier inter-annotator agreement on development and test sets, while Table 11 reports agreement with intended author labels.
- Table 12 provides additional collected-data examples.
E User interface
The paper presents interfaces for creating, validating, and verifying examples, alongside figures and tables documenting annotation effort, agreement, labels, and linguistic phenomena. These materials organize the three-round collection process and its supporting analyses.
- Interfaces: Examples of the user interface are shown for creation, collection, and verification tasks.The creation interfaces provide context to annotators and give model feedback during collection.
- Collection statistics: Histograms summarize the number of tries, time spent, and token counts for good verified examples across three rounds.The token-count analysis covers contexts and hypotheses.
- Verification outcomes: The three-round verification outcomes distinguish model-correct examples, verifier agreement with writers, verifier overrides, and cases without verifier agreement.The figure specifies which categories are added to the training set.
- Linguistic analysis: A development-set table compares linguistic-phenomenon tag percentages in AdvNLI, SNLI, MultiNLI, and FEVER contexts and hypotheses.Bold values indicate higher incidence in adversarial hypotheses, while italicized values indicate roughly similar incidence.
- Annotation analysis: The tables report inter-annotator agreement, verifier agreement with author labels, and label distributions across dataset splits and rounds.Validators are identified separately for SNLI and MNLI in the verifier-agreement table.