PaperlayerView exact revision on arXiv

Source-linked AI summary

ERASER: A Benchmark to Evaluate Rationalized NLP Models

Jay DeYoung, Sarthak Jain, Nazneen Fatema Rajani, Eric Lehman, Caiming Xiong, Richard Socher, Byron C. Wallace

arXiv:1911.03429v2cs.CLcs.AIcs.LG

TL;DR

Interpretable NLP methods have been evaluated on different datasets and with different metrics, making comparison and progress tracking difficult. ERASER addresses this gap with a multi-task benchmark containing human rationales and metrics for rationale agreement and faithfulness; its baselines reveal adaptation challenges across datasets.

  • Problem

    Interpretable NLP research uses different datasets and performance metrics, making methods difficult to compare and progress difficult to track.

  • Method

    ERASER releases diverse repurposed and augmented NLP datasets with human rationales, standardized formats, tools, baselines, and metrics for agreement and faithfulness.

  • Results

    Baseline experiments find that no single off-the-shelf architecture readily adapts to datasets with different instance lengths and rationale snippets.

  • Takeaways & Limitations

    ERASER provides a resource for developing interpretable NLP models and comparing them across tasks, datasets, and rationale criteria.

  • Takeaways & Limitations

    The evaluation is constrained by computational limits and by a pipeline model that assumes rationales are sentences, making it unsuitable for short-span rationale datasets.

Abstract

from arXiv · show

State-of-the-art models in NLP are now predominantly based on deep neural networks that are opaque in terms of how they come to make predictions. This limitation has increased interest in designing more interpretable deep models for NLP that reveal the `reasoning' behind model outputs. But work in this direction has been conducted on different datasets and tasks with correspondingly unique aims and metrics; this makes it difficult to track progress. We propose the Evaluating Rationales And Simple English Reasoning (ERASER) benchmark to advance research on interpretable models in NLP. This benchmark comprises multiple datasets and tasks for which human annotations of "rationales" (supporting evidence) have been collected. We propose several metrics that aim to capture how well the rationales provided by models align with human rationales, and also how faithful these rationales are (i.e., the degree to which provided rationales influenced the corresponding predictions). Our hope is that releasing this benchmark facilitates progress on designing more interpretable NLP systems. The benchmark, code, and documentation are available at https://www.eraserbenchmark.com/

1 Introduction

ERASER addresses the difficulty of comparing interpretable NLP methods by providing a standardized benchmark spanning diverse tasks, human rationales, and rationale metrics. It also supplies baselines and exposes the need for models that handle varied input lengths and rationale granularities.

  • ERASER draws inspiration from GLUE and SuperGLUE as standardized collections intended to support progress and comparison in interpretable NLP.The authors hope the benchmark will aid development of standardized metrics for different properties of interpretability.
  • The benchmark focuses on rationales as snippets supporting outputs, which should be sufficient for predictions but may not be comprehensive.Some datasets therefore include comprehensive rationales marking all supporting evidence on test instances.
  • Its metrics assess both agreement with human rationales and faithfulness, or whether model-provided rationales informed the corresponding predictions.The authors present these metrics as an initial set and do not privilege a single metric.
  • No single off-the-shelf architecture readily adapts to datasets with different instance lengths and rationale snippets.The authors identify a need for models that consume lengthy inputs and provide rationales at task-appropriate granularity.
  • ERASER standardizes diverse NLP datasets and tasks with human-marked rationales, starter tools, baseline models, and initial rationale metrics.

2 Related Work

Prior interpretable NLP work includes rationale acquisition, rationale-aware learning, rationale evaluation, and model explanation methods. ERASER emphasizes that matching human rationales measures plausibility, whereas faithfulness asks whether those inputs actually drove predictions.

  • Learning to explain: Attention can extract highly weighted tokens, but attention weights do not generally provide faithful explanations for predictions.The text attributes this difficulty partly to encoders entangling inputs in contextualized representations.
  • Learning to explain: Hard attention extracts snippets for a classifier and thereby provides faithful explanations by construction, while proposed systems differ in optimization and extraction design.Examples include jointly trained extractor-classifier models, heuristic pseudo-labeling, classwise rationales, and differentiable binary masks.
  • Post-hoc explanation: Post-hoc explanation methods seek to explain why a model made a specific prediction for a given input.
  • Learning from rationales: Rationale-learning methods have extended discriminative, generative, convolutional, pipeline, and Transformer-based models to use or produce human rationales.
  • Evaluating rationales: Agreement with human rationales or subjective judgments assesses plausibility but does not establish whether the model relied on those rationales.ERASER calls rationales corresponding to inputs most relied upon for a prediction faithful.
  • Evaluating rationales: Faithfulness evaluations commonly measure how perturbing or erasing identified important tokens changes model output.The authors build on these perturbation-based methods and note that faithfulness may be better treated as a continuous property.

3 Datasets in ERASER

ERASER distributes standardized datasets with labels and human rationales, using repurposed corpora across tasks such as evidence inference, question answering, sentiment, and textual verification. The benchmark includes dataset-specific rationale annotations and documented split decisions.

  • All ERASER datasets provide reference labels and human-marked rationales in a standardized format.
  • The benchmark defines train, validation, and test splits for all corpora and generally keeps source documents disjoint to avoid contamination.BoolQ is an explicit exception because its original train and validation source documents were not disjoint.
  • Evidence Inference classifies whether an intervention significantly increases, decreases, or does not significantly affect an outcome relative to a comparator, using supporting rationales.Exhaustive rationale annotations were collected for subsets of validation and test data.
  • BoolQ contains Wikipedia passages with generated yes/no questions, while ERASER recovered source passages and obtained comprehensive annotations for part of its test set.
  • Movie Reviews provides positive/negative sentiment labels and includes comprehensive span-level rationales for the final two folds.
  • MultiRC converts each rationale/question/answer triplet into an instance with a True or False label for each answer candidate.Its questions have multiple correct answers that depend on information from multiple sentences.
  • CoS-E contributes highlighted supporting snippets, while ERASER excludes its free-text explanations because the benchmark focuses on extractive rationales.
  • e-SNLI marks rationale words under entailment, contradiction, and neutral-specific annotation constraints across the premise and hypothesis.

4 Metrics

ERASER evaluates rationale quality through human agreement and faithfulness metrics, including measures of whether rationales influenced predictions and contained sufficient predictive signal. The benchmark also standardizes evaluation across diverse datasets while exposing limitations of threshold-based faithfulness assessment.

  • ERASER evaluates models using both task predictive performance and the quality of extracted rationales.
  • 4.1 Agreement with human rationales: Human-agreement metrics assess how closely extracted rationales match annotated rationales, using relaxed overlap measures for discrete selections and AUPRC for soft scores.Discrete evaluations include token-level IOU, precision, recall, and F1; soft evaluations use token-score rankings.
  • 4.2 Measuring faithfulness: Faithfulness metrics measure comprehensiveness by removing extracted rationales and sufficiency by testing whether rationales alone support a similar prediction.A high comprehensiveness score indicates that removing rationales reduces confidence, while sufficiency measures whether the retained snippets adequately support prediction.
  • 4.2 Measuring faithfulness: For soft importance scores, ERASER selects the top-k_d tokens, where k_d equals the dataset’s average human rationale length, before computing faithfulness.This discretization makes soft scores compatible with the comprehensiveness and sufficiency calculations.
  • 4.2 Measuring faithfulness: The soft-score approach is computationally cheap but requires choosing a threshold k, limiting evaluation to a particular discretization.
  • 4.2 Measuring faithfulness: AOPC evaluates token orderings across the top 1%, 5%, 10%, 20%, and 50% of tokens, with analogous comprehensiveness and sufficiency measures and random-score references.

5 Baseline Models

ERASER baseline models cover hard rationale selection and soft token scoring across datasets with substantially different input and rationale lengths. The benchmark therefore requires models that adapt their rationale granularity to each task.

  • Model families: Baseline models are grouped into hard selectors that extract snippets and soft scorers that assign continuous token importance.The evaluated soft scorers include simple gradients, attention, and LIME; hard models include jointly trained and pipeline architectures.
  • Design constraints: ERASER datasets differ considerably in document and rationale lengths, motivating models that flexibly adapt to varying input lengths and rationale granularities.The benchmark authors state that such flexibility is needed to compare models across all datasets.
  • Hard selection: Hard-selection models use an encoder to extract snippets and a decoder that predicts from the extracted text.Lei et al. jointly train these modules, while pipeline models train separate rationale-identification and prediction stages.
  • Hard selection: Pipeline baselines independently identify sentence rationales and classify using those rationales, with a BERT-to-BERT variant replacing both stages with BERT.The rationale-identification stage uses source-annotation sentence boundaries and sampled negative examples during training.
  • Evaluation setup: The reported hard-selection results use accuracy for CoS-E and macro-averaged F1 for the other datasets.Table 3 distinguishes rationale-supervised models from models trained only with instance-level supervision and marks degenerate rationale training.
  • Soft selection: Soft scoring models assign importance scores to tokens, and their evaluation includes task performance, rationale metrics, and random-score reference results.Table 4 reports comprehensiveness and sufficiency as AOPC and averages random orderings over 10 runs.

6 Evaluation

ERASER evaluates rationale agreement and faithfulness across hard and soft selection methods. The results show trade-offs among human-rationale agreement, faithfulness, predictive performance, and rationale granularity.

  • Hard selection: Hard-selection models are treated as faithful by construction because the decoder predicts only from the snippets selected by the encoder.This evaluation therefore reports agreement-with-human-rationale metrics for these methods.
  • Limitations: Computational constraints prevented running the BERT-based Lei et al. implementation on larger corpora, and sentence-level pipelines are unsuitable for very short-span rationales.These limitations motivate extraction models that select contiguous rationales at task-appropriate lengths.
  • Hard selection: Rationale-level supervision often improves agreement with human rationales for Lei et al.’s rationalizing model, but this improvement is not strongly correlated with predictive performance.The authors qualify the improvement as occurring often, though not always.
  • Hard selection: Lei et al. outperforms the simple pipeline with a BERT encoder and the comparable BERT-to-BERT pipeline on final prediction tasks.The authors suggest this may reflect Lei et al.’s ability to select arbitrary subsets rather than only sentences.
  • Soft selection: Simple gradients and LIME produce more comprehensive rationales than attention weights, while attention performs better on AUPRC agreement with human rationales.LIME performs particularly well across these tasks on faithfulness.
  • Faithfulness analysis: Random removal is especially damaging for models with high sufficiency, whereas models with high comprehensiveness are sensitive to rationale length.Models with poor final-task performance also tend to have poor overall token ordering and marginal differences in comprehensiveness and sufficiency.

7 Conclusions and Future Directions

ERASER releases a standardized, publicly available benchmark with human rationales and metrics for evaluating rationale quality. The authors position it as a foundation for comparing interpretable NLP models and improving faithfulness evaluation.

  • Contributions: ERASER contains seven datasets with instance-level labels and human-marked supporting snippets, augmented annotations, and a standard input-rationale-output format.The datasets are publicly released as a benchmark for explainable NLP.
  • Evaluation: The benchmark proposes metrics for both agreement with human annotations and faithfulness of extracted rationales.The authors regard these metrics as reasonable means for comparing specific aspects of interpretability.
  • Future directions: ERASER is intended to support comparison of interpretable models across varied tasks, datasets, and desired criteria.The authors also identify better interpretability metrics, causal analysis, and multilingual rationale datasets as future directions.

A Dataset Preprocessing

ERASER preprocessing standardizes diverse datasets while applying dataset-specific filtering, repartitioning, document recovery, and tokenization procedures.

  • ERASER datasets were converted to a unified format, with tokenization applied using spaCy or SciSpacy for Evidence Inference.
  • MultiRC uses its validation set for testing, while e-SNLI separates premise and hypothesis statements into distinct documents.
  • Evidence Inference removes prompts lacking grounded evidence spans because some provided evidence spans have no offsets.
  • Movie reviews use the ninth fold for validation and collect comprehensive annotations on the tenth fold for evaluation.
  • FEVER removes the Not Enough Info class and multi-document claims, repartitions validation data, and prevents document overlap across splits.
  • BoolQ reconstructs source paragraphs from a Wikipedia archive using fuzzy matching and removes instances below a Levenshtein ratio of 90.

B Annotation details

ERASER collected comprehensive rationales on subsets of test sets to evaluate rationale recall, using different annotators, expertise requirements, and annotation procedures across datasets.

  • Comprehensive rationales were collected for subsets of test sets to evaluate how completely models recover supporting evidence.
  • For Movie reviews, two fluent English speakers annotated 200 test documents, with each document taking approximately 5 minutes on average.
  • Evidence Inference used four medical professionals to annotate 125 documents, averaging 31 minutes and USD 13 per document.
  • BoolQ collected three annotations for each of 199 randomly selected test documents from qualified MTurk workers paid USD 1.50 per HIT.
  • The authors report that BoolQ annotations took longer than anticipated, producing an unintended effective low pay rate.

C.1 (Lei et al., 2016) models

The Lei et al. (2016) baseline uses recurrent encoders and attention-based rationale extraction, with BERT or GloVe embeddings and standardized training settings.

  • The model uses a bidirectional LSTM with 128-dimensional hidden states in each direction and dropout before the decoder’s attention layer.
  • BERT provides token embeddings for most datasets, while BoolQ, Evidence Inference, and FEVER use GloVe embeddings.
  • The models use sparsity rate 0.01 and contiguity loss weight equal to twice the sparsity rate.
  • Training uses Adam with learning rate 2e-5, 20 epochs, early stopping after five patience epochs, and validation-based model selection.
  • Inputs are encoded as [CLS] document [SEP] query [SEP], and implementation uses AllenNLP.
  • The BERT-LSTM reuses the decoder and hyperparameters, whereas GloVe-LSTM is trained with learning rate 1e-2.

C.3 Lehman et al. (2019) models

The Lehman et al. (2019) models use pretrained encoders for rationale identification and classification, with SciBERT as the Evidence Inference exception.

  • Earlier models use GloVe vectors except for Evidence Inference, which uses PubMed word vectors.
  • Training uses Adam, validation-loss monitoring, and retention of the best validation model; the reported schedule is 50 epochs with patience 10 for earlier models and 10 epochs for the BERT pipeline.
  • BERT-base-uncased supplies both components of the identification and classification pipeline for most datasets.
  • Evidence Inference instead uses SciBERT for the pipeline.
Loading 1911.03429v2…