Evaluating Models' Local Decision Boundaries via Contrast Sets

1 Introduction

• Systematic gaps, including annotator bias, can allow simple decision rules to score well on test data despite models failing on perturbations and challenge examples.This mismatch can also appear under covariate and label shifts.Source: §1 · p. 1
• Dataset authors manually make small, typically label-changing test-instance edits that preserve original artifacts, creating contrast sets to probe the correct local decision boundary.The authors recommend creating perturbations without a model in the loop and using them to measure decision-boundary consistency.Source: §1 · p. 2
• About a person-week of work can produce high-quality perturbed test sets of approximately 1,000 instances for many benchmarks, though effort varies greatly.Source: §1 · p. 2
• Across 10 diverse NLP datasets, state-of-the-art models performed dramatically worse on contrast sets than on original test sets, motivating their release as benchmarks.The authors recommend making contrast-set creation standard practice for NLP datasets.Source: §1 · p. 2

2.1 The Problem

• Systematic gaps can let simple decision boundaries perform well on i.i.d. test data; contrast sets probe neighborhoods around test pivots to check local alignment with the correct boundary.The section notes that fully filling such gaps is usually impossible.Source: §2.1 · pp. 2–3

2.2 Definitions

• A local decision boundary groups nearby inputs and their labels around a pivot, while a contrast set samples from that region, typically including examples whose labels differ from the pivot.Contrast consistency requires correct predictions on every example in the set.Source: §2.2 · p. 4

2.3 Contrast sets in practice

• For NLP, experts can judge which small input changes are meaningfully different, addressing both the distance function and sampling choices.Expert-built sets provide only an upper bound on local alignment; the authors draw pivots from an i.i.d. test set and withhold contrast sets from training to avoid additional artificially simple boundaries.Source: §2.3 · p. 4

2.4 Design Choices of Contrast Sets

• Construct contrast sets at dataset release, have original authors build them, and keep models out of annotation to avoid delay and model-specific bias.Easier perturbations can be automated, but pushing examples across a decision boundary is challenging to automate.Source: §2.4 · p. 5

2.5 Limitations of Contrast Sets

• Contrast sets can reveal local misalignment but cannot confirm alignment, and each dataset requires experts to specify the phenomena its sets should test.Their necessarily incomplete coverage gives them only negative predictive power.Source: §2.5 · p. 5

DROP

• DROP contrast sets add reasoning steps, invert question semantics, and reorder events to target patterns that can make questions artificially easy.The section identifies frequent answer patterns and questions that do not require compositional reasoning as issues in the original data.Source: §3 · p. 6

NLVR2

• For NLVR2, contrast sets modify sentences or replace images, including changes to quantifiers, entities, and object properties.The dataset pairs a sentence about two images with a true-or-false judgment.Source: §3 · p. 6

UD Parsing

• For UD parsing, the authors target prepositional-phrase attachment by changing a noun phrase so the sentence requires a different attachment, then re-annotating it.They search English treebanks for potentially ambiguous attachments to a noun or verb.Source: §3 · p. 6

Summary

• Although the process is unified, each dataset’s contrast sets must target its intended phenomena and remain similar to those in the original test set.The aim is more thorough evaluation of the test set’s original intent, not introducing new challenges.Source: §3 · p. 6

4.1 Original Datasets

• The ten datasets span diverse tasks and formats, including reading comprehension, sentiment analysis, visual reasoning, classification, span extraction, and structured prediction.They include both high-level tasks with known artifact concerns and formalism-based tasks where artifacts have been less studied.Source: §4.1 · p. 7

4.2 Contrast Set Construction

• Researchers deeply familiar with each dataset constructed contrast sets of up to about 1,000 examples, typically averaging 1–3 minutes per perturbation.Creating 1,000 examples took approximately 17–50 hours for most datasets, while dependency-parsing examples averaged 15 minutes each.Source: §4.2 · p. 7

4.3 Models Struggle on Contrast Sets

• Models struggle more on contrast sets, particularly on contrast consistency, though scores vary and are not directly comparable across datasets.IMDb and PERSPECTRUM show reasonably high consistency; Table 2 reports results across the datasets and metrics.Source: §4.3 · p. 7, Table 2 · p. 7

4.4 Humans Succeed On Contrast Sets

• Human performance was comparable on original and contrast examples across four datasets.The authors tested at least 100 test instances and corresponding perturbed examples per dataset, with testers different from the example creators.Source: §4.4 · p. 8, Table 3 · p. 8

4.5 Fine-Grained Analysis of Contrast Sets

• Fine-grained labels show MATRES models did better on appearance order (66.5% correct) than temporal conjunction words (60.0% correct).On DROP, MTMSN scored 47.3 F1 on event re-ordering versus 67.5 F1 on added compositional reasoning steps; dependency-parsing contrasts expose attachment decisions that do not respond reliably to prepositional-phrase content.Source: §4.5 · p. 8

• The paper extends minimally different examples to contrast sets, using consistency to assess local alignment with decision boundaries.It also offers a geometric interpretation of dataset-collection bias.Source: §5 · p. 8
• Some prior minimally contrastive examples and data augmentation methods were designed to provide training data or mitigate biases during training.The paper instead frames contrast sets as an evaluation resource.Source: §5 · pp. 8–9
• Unlike prior challenge sets, contrast sets group perturbed instances to measure local decision-boundary alignment and augment existing test examples rather than creating data from scratch.The paper argues this often requires less effort and keeps examples grounded in the original data distribution.Source: §5 · p. 9
• Recollecting test sets examines possible adaptive overfitting to reused test data, whereas contrast sets target biases in the original annotation process.Simply collecting more data would not address those biases, the paper argues, because new data would capture similar ones.Source: §5 · p. 9

6 Conclusion

• The conclusion presents the annotation paradigm as preserving most established dataset-creation processes while filling systematic gaps typically present in datasets.Source: §6 · p. 9
• Shifting evaluation from accuracy on i.i.d. test sets to consistency on contrast sets helps examine whether models learned intended capabilities or dataset idiosyncrasies.The authors created contrast sets for 10 NLP datasets and released them as evaluation benchmarks.Source: §6 · p. 9
• The authors recommend that future data collection efforts create contrast sets for more comprehensive evaluations of existing and new NLP datasets.Source: §6 · p. 9
• The paper describes its contrast-set work as only small steps toward the rigorous evaluations desired for NLP.Source: §6 · p. 9

A Dataset Details

• NLVR2 asks whether a sentence about two photographs is true and targets object detection, counting, spatial relations, and language understanding.Source: A Dataset Details · p. 13
• IMDb classifies movie-review sentiment, while MATRES classifies before-or-after relations between events whose event annotations are given.The MATRES temporal relations are re-annotated using a multi-axis formalism.Source: A Dataset Details · p. 13
• The English UD parsing data combine four treebanks across genres and focus on potentially ambiguous prepositional-phrase attachments; PERSPECTRUM focuses on predicting whether a perspective supports or opposes a claim.Source: A Dataset Details · p. 13
• DROP requires numerical reasoning over paragraphs, QUOREF requires coreference resolution, and ROPES asks readers to apply causal background knowledge to situations.For DROP, consistency is measured using the average number of contrast sets in which F1 for all instances exceeds 0.8; QUOREF and ROPES use the same definition as DROP.Source: A Dataset Details · p. 13
• BoolQ uses yes-or-no reading-comprehension questions without a clear set of intended phenomena, while MC-TACO tests common-sense knowledge of event durations, orderings, and frequency.MC-TACO's paragraphs do not contain the information necessary to answer its questions.Source: A Dataset Details · pp. 13–14

B.1 NLVR2

• For NLVR2, annotators perturb quantifiers, numbers, entities, and entity properties in text, or select perceptually or conceptually close images that minimally change a concrete image aspect.LXMERT trained on NLVR2 training data was used; five annotators created 983 perturbed instances forming 479 contrast sets.Source: §B.1 · p. 14
• NLVR2 annotation took about 30 seconds per textual perturbation and two minutes per image perturbation.Source: §B.1 · p. 14

B.2 IMDb

• For IMDb, reviews are minimally perturbed to flip the label while remaining coherent and factually consistent, as illustrated by edited review examples.The setup uses the same BERT model and training data as Kaushik et al. (2020) to compare crowd and expert revisions.Source: §B.2 · p. 14
• Three annotators spent approximately 70 hours constructing and validating the IMDb contrast set, using 100 validation reviews and 488 test reviews.Source: §B.2 · p. 14

B.3 MATRES

• The MATRES test set is the Platinum section, from which the authors use 239 instances, or 30% of the dataset.Source: §B.3 · p. 14
• MATRES perturbations change appearance order, verb tense, or temporal conjunction words, as illustrated by revisions that alter event order or tense.Source: §B.3 · pp. 14–15
• Two annotators created 401 perturbed instances forming 239 contrast sets and spent approximately 25 hours constructing and validating the dataset.Source: §B.3 · p. 15
• Appearance-order changes made up 49% of perturbations, tense changes 31%, temporal-conjunction changes 24%, and other changes 10%, with examples involving multiple perturbations counted more than once.Source: §B.3 · p. 15
• Table 4 presents an accuracy breakdown by MATRES perturbation type.Source: Table 4 · p. 15

B.4 Syntactic Parsing

• PP-attachment perturbations changed the noun phrase beside an unchanged preposition, altering the preposition’s syntactic path to its grandparent.For example, replacing “local US commanders” with “great urgency” changed the grandparent from the noun “talks” to the verb “demanded.”Source: §B.4 · p. 15
• Only a small fraction of sentences could be altered as desired, and each parsing perturbation took 10–15 minutes, with broader feature coverage requiring substantial annotator time.Annotators also had to find a natural alternative noun phrase and re-annotate relevant POS tags, UD word features, and arc labels.Source: §B.4 · p. 15
• 64.7% accuracy on target prepositions’ grandparents in the original tree was below 78.41% for all words and 78.95% for all prepositions, indicating structural ambiguity.The contrast consistency on the parsing set was 17.3%.Source: §B.4 · p. 15

B.5 PERSPECTRUM

• PERSPECTRUM annotators created non-trivial claim negations and labeled each altered claim against its perspective, changing the example label from True to False.The example negates “Should we live in space?” while keeping the perspective text fixed.Source: §B.5 · pp. 15–16

B.6 DROP

• The appendix refers readers to Section 3 of the main text for details of the DROP perturbation strategies.Source: §B.6 · p. 16
• The augmented DROP test set contains 947 examples and 623 contrast sets, constructed and validated by three annotators in approximately 16 hours.The model used is MTMSN, a DROP question-answering model built on BERT Large.Source: §B.6 · p. 16
• The analysis evaluates MTMSN on 100 perturbed instances grouped into the three perturbation categories described in Section 3.Table 5 is identified as an accuracy breakdown by DROP perturbation type; the supplied passages do not provide its cell values.Source: §B.6 · p. 16, Table 5 · p. 16

B.7 QUOREF

• QUOREF perturbations change entity answers to properties, add compositionality, insert sentences between references and antecedents, or replace antecedents with less frequent same-type entities.Four annotators created 700 instances in 415 contrast sets, averaging 2.7 (±1.2) examples per set; construction and validation took about 35 hours. The model was XLNet-QA, a span extraction model built on XLNet.Source: §B.7 · p. 16

B.8 ROPES

• ROPES perturbations alter background causes or effects, reassign entities to situation details, add reasoning steps, or align question presuppositions with the situation and background.Two annotators created 974 perturbed instances and used approximately 65 hours to construct and validate them. The model was a RoBERTa-based span extractor first finetuned on RACE (Liu et al., 2019).Source: §B.8 · pp. 16–17

B.9 BoolQ

• BoolQ perturbations covered adjective, event, and entity changes, illustrated by changing a question about the last movie to the first of multiple movies, win to play in, and the president to George Washington.Annotators created 339 perturbed questions forming 70 contrast sets; one created the dataset and another verified it, taking approximately 16 hours. The model was ROBERTA base, following the standard finetuning process from Liu et al. (2019).Source: §B.9 · p. 17

B.10 MC-TACO

• MC-TACO perturbations retain a similar question and temporal knowledge while changing related context and answers to provide plausible and implausible candidates.Two annotators created 646 perturbed question-answer pairs forming 646 contrast sets in approximately 12 hours; the model was the original paper’s RoBERTa-base baseline (Liu et al., 2019).Source: §B.10 · p. 17