Source-linked AI summary
Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference
R. Thomas McCoy, Ellie Pavlick, Tal Linzen
TL;DR
NLI models may achieve high test performance by relying on shallow syntactic heuristics rather than deeper generalizations. The paper introduces HANS, a controlled evaluation set targeting three such heuristics, and finds that MNLI-trained models perform very poorly on it, supporting the heuristic-adoption hypothesis. HANS and related training examples provide tools for diagnosing and reducing these shortcomings.
Problem
NLI models may learn shallow heuristics supported by standard training data instead of generalizing to challenging inference cases.
Method
The paper introduces HANS, a controlled evaluation set designed to diagnose lexical overlap, subsequence, and constituent heuristics, and evaluates models including BERT.
Results
All four evaluated NLI models performed substantially below chance on HANS, barely exceeding 0% accuracy in most cases.
Takeaways & Limitations
HANS can motivate and measure progress, while augmenting training with HANS-like examples can make current models’ shortcomings less severe.
Takeaways & Limitations
The experiment leaves open how many heuristic-contradicting examples are required for a model to avoid a heuristic.
Abstract
from arXiv · showhide
A machine learning system can score well on a given test set by relying on heuristics that are effective for frequent example types but break down in more challenging cases. We study this issue within natural language inference (NLI), the task of determining whether one sentence entails another. We hypothesize that statistical NLI models may adopt three fallible syntactic heuristics: the lexical overlap heuristic, the subsequence heuristic, and the constituent heuristic. To determine whether models have adopted these heuristics, we introduce a controlled evaluation set called HANS (Heuristic Analysis for NLI Systems), which contains many examples where the heuristics fail. We find that models trained on MNLI, including BERT, a state-of-the-art model, perform very poorly on HANS, suggesting that they have indeed adopted these heuristics. We conclude that there is substantial room for improvement in NLI systems, and that the HANS dataset can motivate and measure progress in this area
1 Introduction
Neural NLI models can exploit shallow heuristics that fit frequent training patterns rather than learning intended generalizations. HANS evaluates whether models rely on three such syntactic heuristics and exposes severe failures in models trained on MNLI.
- Statistical learners may adopt shallow heuristics that succeed on frequent training examples but fail on challenging cases.When heuristics usually produce correct outputs, the loss function provides little incentive to learn deeper generalizations.
- A lexical-overlap strategy can label The actor was paid by the judge as entailing The actor paid the judge, despite the reversed semantic roles making the label incorrect.The heuristic assumes that a premise entails any hypothesis whose words all appear in the premise.
- HANS is a controlled NLI evaluation set designed to diagnose lexical overlap, subsequence, and constituent heuristics on examples where they fail.Its construction targets particular failure subsets rather than merely measuring lower overall accuracy.
- All four evaluated NLI models, including BERT, performed substantially below chance on HANS, barely exceeding 0% accuracy in most cases.The authors interpret this behavior as consistent with adoption of the targeted heuristics.
- HANS illuminates interpretable shortcomings in state-of-the-art MNLI-trained models and can motivate and measure progress in NLI.The paper also reports that augmenting training with HANS-like examples can make these shortcomings less severe.
2 Syntactic Heuristics
The paper targets three nested syntactic heuristics that can be supported by standard NLI datasets and model representations. Their prevalence may encourage models to learn shortcuts instead of valid inference strategies.
- The targeted heuristics are lexical overlap, subsequence, and constituent, forming a hierarchy from the broadest to the narrowest pattern.The constituent heuristic is a special case of subsequence, which is a special case of lexical overlap.
- Standard NLI training data contains many more heuristic-supporting than heuristic-contradicting examples, giving statistical learners limited evidence against the shortcuts.The paper notes that MNLI contains only 261 contradicting cases in the relevant analysis.
- Some MNLI contradictions can be explained by learning additional cues, such as predicting contradiction when negation appears in the premise but not the hypothesis.This may prevent those examples from strongly overruling the lexical-overlap heuristic.
- Model representations may favor different heuristics: bag-of-words models disregard order, sequential models process adjacent chunks, and tree-based models use parse constituents.These representational properties correspond respectively to lexical overlap, subsequence, and constituent heuristics.
3 Dataset Construction
HANS uses controlled templates to generate plausible, non-overlapping examples that separately test each targeted heuristic. The construction also controls lexical and syntactic selection effects.
- For each heuristic, the authors created five supporting templates and five contradicting templates.The templates were designed to test cases where the heuristic succeeds or fails.
- Each heuristic receives 1,000 examples per template, producing 10,000 examples per heuristic.Examples for one heuristic were prevented from also qualifying under a narrower heuristic.
- The construction uses plausible sentences to avoid allowing hypothesis-plausibility heuristics to solve the examples.The authors avoid implausible reversals such as The student read the book ↛ The book read the student.
- Examples requiring knowledge of verb frames use verbs appearing at least 50 times in MNLI in all relevant frames.This controls whether models have had sufficient exposure to the relevant syntactic constructions.
4 Experimental Setup
The experiments compare NLI models with bag-of-words, sequential, syntactic-tree, and Transformer representations, all trained or fine-tuned using MNLI. HANS is evaluated with binary entailment versus non-entailment labels.
- The study evaluates DA, ESIM, SPINN, and BERT to represent bag-of-words, sequential, syntactic-tree, and Transformer-based strategies.DA is bag-of-words, ESIM uses a sequential encoder, SPINN uses a parse tree, and BERT uses attention.
- DA uses attention to align premise and hypothesis words without word-order information.It can therefore be viewed as a bag-of-words model.
- ESIM encodes sentences with a modified bidirectional LSTM using a sequential encoder rather than a tree-based encoder.The experiment uses the sequential variant of ESIM.
- SPINN combines phrases according to a syntactic parse tree and takes a parse tree as input.For HANS, the authors use parse templates created from parses produced by the Stanford PCFG Parser.
- All models are trained on MNLI, while HANS uses entailment and non-entailment labels obtained by mapping contradiction or neutral predictions to non-entailment.The highest-scoring MNLI label is selected before this mapping.
5 Results
All four MNLI-trained models performed well on MNLI but failed on HANS cases where the targeted heuristics made incorrect predictions. Their error patterns were consistent with broadly predicting entailment, although performance varied by model and example type.
- All models achieved high MNLI test accuracy but scored below 10% in most HANS cases where the heuristics made incorrect predictions, versus 50% chance.They nearly always assigned the correct label when it matched the heuristics.
- All four models behaved consistently with using HANS-targeted heuristics rather than the correct inference rules.
- Comparison of models: DA and ESIM had near-zero performance across all three heuristics, suggesting they treated them primarily as lexical overlap.DA lacks access to word order, while ESIM theoretically has it but did not appear to use it here.
- Comparison of models: SPINN performed best on subsequence cases and outperformed DA and ESIM on constituent cases, consistent with benefits from its tree-based representations.Its structural input may have helped it represent non-constituent subsequences and constituent contributions.
- Comparison of models: BERT performed less poorly than the other models on lexical-overlap and constituent cases, though it remained far below chance.Its lexical-overlap performance particularly suggested greater use of word-order information than the other models.
- Analysis of particular example types: BERT’s accuracy varied sharply across subcases, reaching 39% on conjunction but 0% on subject-object swaps and 49% on one conditional case but 0% on another.
6 Discussion
The discussion examines whether HANS failures reflect model architecture, insufficient training signal, or dataset difficulty. Comparisons across models and human judgments suggest training data contributes substantially, while HANS is challenging for models but tractable for people.
- All models failed on examples targeting all three heuristics, because the heuristics are hierarchically related and all models access individual words.Subsequence and constituent heuristics are special cases of lexical overlap.
- The heuristic hierarchy does not necessarily predict model performance, since specific test cases may differ in difficulty or models may exempt special cases.
- The models’ poor HANS results could reflect architectural limitations, insufficient MNLI signal, or both.
- SPINN’s stronger constituent and subsequence performance than DA and ESIM, despite identical training data, suggests MNLI contains signal that structural biases can exploit.
- BERT’s 0% accuracy on subject-object swaps despite access to relevant syntactic information suggests MNLI did not clarify how syntax applies to inference.The authors therefore argue that poor compositional behavior arises more from training data than architecture alone.
- BERT’s comparative HANS success cannot be attributed solely to architecture because it was pretrained on far more data than the other models.
- Mechanical Turk participants averaged 76% accuracy and experts 97%, indicating HANS is difficult but not generally beyond human understanding.Human accuracy was balanced across entailment and non-entailment labels at 75% and 77%, unlike the models’ label-imbalanced errors.
7 Augmenting the training data with HANS-like examples
The authors augment MNLI with HANS-like examples to test whether training data can reduce heuristic behavior and generalize beyond HANS. Augmentation generally improves HANS performance, but gains on MNLI and transfer across cases are uneven.
- Augmentation design: 30,000 HANS-like examples, roughly 8% of MNLI, were added without reusing HANS test examples.The augmented data used the same 30 subcases but contained no specific examples appearing in HANS.
- HANS results: Augmented models generally performed very well on HANS, except DA on subcases where bag-of-words representations were inadequate.
- Open questions: The augmentation experiment leaves open how many contradicting examples are required for models to avoid a heuristic.
- MNLI results: MNLI test accuracy changed inconsistently: BERT rose from 84.1% to 84.4%, while DA fell from 72.4% to 66.0%.ESIM also improved from 77.3% to 77.6%, whereas SPINN declined from 67.0% to 63.9%.
- Transfer: BERT showed some transfer to withheld categories, including sentence-initial adverbs, for both entailment and non-entailment labels.These successes suggest that some learned subcases can rule out broader heuristics rather than merely memorize templates.
- External evaluation: On an external lexical-overlap dataset, HANS-like augmentation improved performance modestly on long examples and dramatically on short examples.The external dataset used comparatives, different vocabulary, and many semantically implausible sentences.
8 Related Work
The paper builds on work diagnosing dataset biases, heuristic behavior, and linguistic representations in NLI and related tasks. It distinguishes HANS through a behavioral evaluation of targeted critical cases and a broader treatment of lexical overlap.
- NLI bias studies: Prior NLI studies showed that hypothesis-only models can achieve better-than-chance accuracy because of dataset biases.
- Heuristic diagnosis: Other work investigated semantic heuristics, while this paper focuses on syntactic heuristics and behavioral evaluation.
- Lexical overlap: The paper studies lexical overlap across a larger set of word-order phenomena than prior work centered on individual phenomena such as passives.
- Subsequence heuristic: The subsequence analysis extends a pilot study and draws several subcases from psycholinguistic research on human representations.
- Constituent heuristic: The constituent subcases extend prior implicational studies of verbs to conjunctions, prepositions, and other words with specific implicational behavior.
- Generalization: The paper aligns with evidence that standard test sets may not reveal failures that appear on generalization sets departing from the training distribution.
9 Conclusions
The authors conclude that high standard-test accuracy can coexist with reliance on invalid heuristics, and that targeted generalization data are needed to expose and reduce this behavior. HANS-like augmentation improves performance, although substantial questions remain.
- Conclusions: Four existing NLI models performed very poorly on HANS, suggesting that their standard-test accuracy may reflect invalid heuristics rather than deeper language understanding.
- Conclusions: HANS makes models using the three hypothesized heuristics fail on targeted examples, providing an evaluation of interpretable shortcomings.
- Conclusions: Training-data augmentation with HANS-like examples substantially improved performance on HANS and on a separate structure-dependent dataset.
- Conclusions: The findings indicate that targeted, challenging datasets are important for determining whether models learn the intended task.
D Results for models trained on MNLI with neutral and contradiction merged
This appendix reports results for MNLI-trained models when neutral and contradiction are merged into non-entailment. The evaluation is organized by heuristic templates, label type, withheld categories, and model experiments.
- Merged-label results: Merging neutral and contradiction into non-entailment produced results similar to merging labels after training.Models generally output entailment for HANS examples regardless of the correct label.
- Heuristic templates: Tables 4, 5, and 6 provide templates for the lexical overlap, subsequence, and constituent heuristics, respectively.
- Result organization: Results are separated into subcases whose correct label is entailment and subcases whose correct label is non-entailment.
- Withheld-category experiments: Five experiments withheld six of HANS’s 30 subcases per model before evaluation on the withheld categories.
F Human experiments
The study collected human judgments across HANS’s 114 example categories and used expert annotations and controls to evaluate the dataset. Supplementary tables provide numerical results for the model experiments and figures.
- Human annotation: HANS was divided into 114 categories covering template variations and the specific words used in each example.Categories included constituent-heuristic cases such as clauses embedded under verbs.
- Human annotation: For each category, 20 HANS examples received judgments from five Mechanical Turk participants, with each participant also completing 10 controls.Participants judged 57 examples plus controls and were paid $2.00.
- Human annotation: Participant response times averaged 6.5 seconds per example, while retained participants averaged 8.9 seconds and discarded participants averaged 4.2 seconds.The full experiment averaged 17.6 minutes, including 9.1 minutes answering prompts.
- Human annotation: Three linguistically trained native English expert annotators each labeled 124 examples, including one example from every category and 10 controls.The experts comprised two graduate students and one postdoctoral researcher who had not previously heard about the project.