PaperlayerView exact revision on arXiv

Source-linked AI summary

Are Human Explanations Always Helpful? Towards Objective Evaluation of Human Natural Language Explanations

Bingsheng Yao, Prithviraj Sen, Lucian Popa, James Hendler, Dakuo Wang

arXiv:2305.03117v2cs.CL

TL;DR

Human explanations can be subjective and differ despite being correct, so similarity to a reference does not by itself establish their quality. The paper evaluates explanation helpfulness to model performance at fine-tuning and inference using a unified format; across five datasets and two architectures, its metric consistently ranks explanation quality while Simulatability falls short.

  • Problem

    Human explanations can vary while remaining correct, making similarity metrics an inadequate basis for treating annotated explanations as gold-standard quality references.

    Source: §1 · p. 1

  • Method

    The paper extends Simulatability to evaluate explanation helpfulness at fine-tuning and inference, using a unified prompt-based format for classification and multiple-choice tasks.

    Source: §1 · p. 2, §5.1 · p. 6

  • Results

    Across five datasets and two model architectures, the proposed metric consistently ranks explanation quality while Simulatability falls short.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The evaluation supports the hypothesis that human explanations can benefit model predictions even when prior human evaluation criticized them as low-quality.

    Source: §1 · p. 2

    A higher metric score does not necessarily indicate higher human-perceived quality because the metric measures explanation benefit from the modeling perspective only.

    Source: §8 · p. 8

Abstract

from arXiv · show

Human-annotated labels and explanations are critical for training explainable NLP models. However, unlike human-annotated labels whose quality is easier to calibrate (e.g., with a majority vote), human-crafted free-form explanations can be quite subjective. Before blindly using them as ground truth to train ML models, a vital question needs to be asked: How do we evaluate a human-annotated explanation's quality? In this paper, we build on the view that the quality of a human-annotated explanation can be measured based on its helpfulness (or impairment) to the ML models' performance for the desired NLP tasks for which the annotations were collected. In comparison to the commonly used Simulatability score, we define a new metric that can take into consideration the helpfulness of an explanation for model performance at both fine-tuning and inference. With the help of a unified dataset format, we evaluated the proposed metric on five datasets (e.g., e-SNLI) against two model architectures (T5 and BART), and the results show that our proposed metric can objectively evaluate the quality of human-annotated explanations, while Simulatability falls short.

1 Introduction

Human explanations can vary for the same instance, making similarity metrics an unreliable measure of their quality. The paper evaluates explanations by their helpfulness to model performance and reports that its metric ranks explanation quality more consistently than Simulatability.

1 Introduction

  • Different people can give distinct, valid explanations for the same observation, so BLEU or ROUGE similarity does not establish explanation quality.The paper instead treats helpfulness to model prediction as the core value of explanations.Source: §1 · pp. 1–2
  • The proposed evaluation extends Simulatability and uses a unified prompt-based format to measure human explanations’ helpfulness to model performance.The format maps classification and multiple-choice tasks into a unified multiple-choice generation format.Source: §1 · p. 2
  • Across five datasets and two model architectures, the proposed metric ranks explanation quality consistently, while Simulatability falls short.The evaluation also supports the hypothesis that explanations can benefit predictions even when prior human evaluations criticized them as low-quality.Source: §1 · p. 2

2 Related Work

The related work covers free-text explanation datasets, shortcomings of existing explanation evaluation, and research on incorporating explanations into models. These strands motivate evaluating explanation utility across task formats and stages of model use.

2.1 Natural Language Explanation Datasets

  • The study evaluates instance-level free-text human explanations from five public datasets, including CoS-E variants, ECQA, e-SNLI, and ComVE.The datasets cover both multiple-choice and classification tasks.Source: §2.1 · p. 2

2.2 Evaluation Metric for Explanations

  • Simulatability measures prediction changes when explanations are provided at inference, but does not cover explanation use during fine-tuning or task-format variation.BLEU and ROUGE, meanwhile, treat human explanations as gold-standard references rather than evaluating their quality.Source: §2.2 · pp. 2–3

2.3 Usage of Explanations for SOTA models

  • Prior work studies explanations as model inputs, generated outputs, or prompt additions, while findings include that prompt-added explanations do not always boost in-context performance.Some work also examines self-rationalization and unified frameworks for generating or extracting explanations.Source: §2.3 · p. 3

3 Unified Structure

The unified structure is designed to reduce task-format differences when measuring explanation helpfulness. It represents classification and multiple-choice tasks as multiple-choice generation with prompts that identify task content and explanations.

3 Unified Structure

  • The paper seeks a metric for explanation utility to model performance because text similarity metrics such as BLEU and ROUGE do not assess how beneficial explanations are.The metric is intended to systematically measure explanation quality, including what prior work calls “noisy” explanations.Source: §3 · p. 4
  • A unified prompt-based format maps classification and multiple-choice tasks to generation tasks, minimizing the influence of task-format variation on model performance.Classification labels can be treated as choices in a multiple-choice task.Source: §3 · p. 4
  • Prompts mark questions with “explain:”, candidate answers with “choice-n:”, and separate task content from explanations with “<sep>”; Infusion explanations begin with “because”.CoS-E and ECQA use their original task as question content, while e-SNLI and ComVE use fixed task-specific questions.Source: §3 · p. 4

4 Preliminary Experiment

The preliminary experiments contrast explanations supplied as input with explanations generated as output, then examine how explanation utility varies between fine-tuning and inference.

4.1 Utilizing Explanations as Part of Input vs Part of Output

  • Infusion improves performance on CoS-E, previously considered noisy, and approaches nearly complete correctness on ECQA; self-rationalization performs worse than Baseline.Source: §4.1 · p. 4

4.2 Explanations as Partial Input During Fine-Tuning

  • Increasing training data does not significantly improve performance, suggesting fine-tuning does not teach new knowledge conveyed by explanations.Source: §4.2 · p. 5
  • Infusion-trained models perform worse when explanations are omitted at inference, indicating that they rely on explanations for prediction.Source: §4.2 · p. 5
  • At inference, explanations hurt Baseline models on CoS-E but significantly improve them on ECQA.Source: §4.2 · p. 5
  • For Infusion-trained models, explanations can improve CoS-E inference performance, and fine-tuning with even a small amount of explanation-bearing data can yield substantial improvement.Source: §4.2 · p. 5
  • Explanation utility differs between fine-tuning and inference, so evaluation should consider both stages.Source: §4.2 · p. 5

5 Our Metric and Evaluation

The section introduces Treu, which evaluates explanation helpfulness during both fine-tuning and inference, and compares it with Simulatability across five datasets and two model architectures. Treu gives consistent cross-dataset rankings across T5 and BART, while Simulatability does not.

5.1 Our Treu Metric

  • Treu sums accuracy differences measured during fine-tuning and inference, whereas Simulatability captures the inference-stage component.A negative Treu score indicates a performance drop in at least one evaluation part, not necessarily that explanations are unhelpful overall.Source: §5.1 · p. 6

5.2 Evaluation

  • The evaluation applies a unified prompt-based format to five datasets with T5 and BART; e-SNLI models are fine-tuned for two epochs.The authors use the same fine-tuning hyperparameters across experiments except for the e-SNLI epoch count.Source: §5.2 · p. 6

5.3 Findings

  • T5-base models outperform BART-base models under the same settings, generally by a significant margin.Source: §5.3 · p. 6
  • Treu ranks explanation quality consistently across T5 and BART—ECQA > CoS-E v1.11 > CoS-E v1.0 > e-SNLI > ComVE—while Simulatability does not.Simulatability reverses the e-SNLI and ComVE rankings between BART and T5.Source: §5.3 · pp. 6–7, Table 3 · p. 7
  • ECQA ranks highest, while both CoS-E datasets have positive Treu scores despite prior descriptions of their explanations as low-quality and noisy (Sun et al., 2022).The authors report that ECQA scores substantially exceed both CoS-E datasets.Source: §5.3 · p. 6
  • Both e-SNLI Treu scores are negative, yet Infusion-fine-tuned models outperform Baseline-fine-tuned models when predicting with Infusion.Entailment/neutral/contradiction Treu scores are 0.13/-0.483/0.094 on T5-base and 0.015/-0.227/-0.271 on BART-base.Source: §5.3 · p. 7
  • Baseline-fine-tuned models show more than 40% prediction-accuracy drops on neutral-label data under Infusion prediction; BART also drops about 40% on contradiction labels.The authors hypothesize that negation or counterfactual explanation styles may make these categories harder for models to interpret.Source: §5.3 · p. 7
  • ComVE ranks worst, and its negation-heavy explanations may make predictions more difficult despite relatively good Baseline performance.The authors connect this possibility to ComVE's task framing, which asks which sentence is more likely against commonsense.Source: §5.3 · pp. 7–8

6 Conclusion

The conclusion presents an objective evaluation of human-annotated explanations based on their helpfulness to model predictions. Across five datasets and two sequence-to-sequence architectures, Treu consistently ranks explanation quality where Simulatability does not.

6 Conclusion

  • The paper evaluates human-annotated natural language explanations by measuring their helpfulness to model prediction.Source: §6 · p. 8
  • Treu considers explanation helpfulness at both fine-tuning and inference, alongside a unified prompt-based format mapping tasks to multiple-choice generation.Source: §6 · p. 8
  • Across five datasets and two sequence-to-sequence architectures, Treu consistently ranks explanation quality while Simulatability does not.Source: §6 · p. 8

7 Limitations

The authors caution that the metric and evaluation may not suit every training setup, and note the high cost of collecting human-annotated explanations.

7 Limitations

  • The evaluation requires generated explanations for training-split data and a second model fine-tuned with Infusion, which may not suit systems trained on that split.Source: §7 · p. 8
  • The metric is evaluated for human-annotated explanations, while generalizing it to model-generated explanations is described only as a possible extension.Source: §7 · p. 8
  • Human-annotated explanations are expensive to collect, and the authors identify active-learning approaches as one possible way to improve annotator performance.Source: §7 · p. 8

8 Ethics Statement

The ethics statement identifies misinterpretation of the findings as a minimal potential risk. It cautions that Treu measures modeling benefit, not necessarily explanation quality as perceived by humans.

8 Ethics Statement

  • A higher Treu score need not indicate higher quality as perceived by humans; the metric measures modeling benefit and is only one possible automatic evaluation.Source: §8 · p. 8

A Implementation of self-rationalization format

B Experiment Hyper-Parameters

The appendix reports experiment hardware and hyperparameters, including separate settings for preliminary experiments and a two-epoch exception for e-SNLI.

B Experiment Hyper-Parameters

B.1 Hyper-parameter for Preliminary Experiment

  • Preliminary experiments used max_len 512 and batch size 1; settings differed in target length, learning rate, and epochs.The input-versus-output experiment used target_max_len 64, learning_rate 5e−5, and 12 epochs; partial-input fine-tuning on CoS-E and ECQA used target_max_len 16, learning_rate 1e−4, and 6 epochs.Source: §B.1 · p. 13

B.2 Hyper-parameter for Explanation Evaluation with five Datasets

  • Five-dataset evaluations used max_len 512, target_max_len 64, batch size 1, and learning_rate 5e−5, with e-SNLI fine-tuned for two epochs.The other listed setting is 12 epochs; e-SNLI has about 10 times the training instances of the other datasets (549,367).Source: §B.2 · p. 13

C Results for Preliminary Experiment - Explanations as Partial Input During Fine-tuning

The preliminary partial-input fine-tuning experiment selected data subsets using three shuffled seeds and reports detailed results and average accuracy in Table 4.

D Examples of different explanations for each category in e-SNLI dataset

The e-SNLI examples illustrate differences in how explanations are phrased across classes: entailment explanations describe shared information, while neutral and contradiction explanations tend toward counterexamples or negation.

Loading 2305.03117v2…