Source-linked AI summary
Reframing Human-AI Collaboration for Generating Free-Text Explanations
Sarah Wiegreffe, Jack Hessel, Swabha Swayamdipta, Mark Riedl, Yejin Choi
TL;DR
High-quality written explanations are difficult and expensive to collect, limiting supervision for free-text explanation generation. The paper uses GPT-3 overgeneration with human-acceptability filtering and finds that it consistently selects explanations humans judge acceptable, while improving support for labels and novel information.
Problem
Collecting high-quality written explanations is difficult and expensive because crowdsourced explanations can be inconsistent, ungrammatical, and hard to compare across datasets.
Method
The paper repeatedly prompts GPT-3 for candidate explanations and trains a supervised filter on human acceptability ratings to select among them.
Results
Within five queries, GPT-3 usually generates an explanation humans unanimously accept, while filtration improves acceptability, label support, and novel information.
Takeaways & Limitations
Overgenerate-and-filter improves few-shot GPT-3 explanations using human acceptability supervision.
Takeaways & Limitations
GPT-3 can explain answer choices it did not predict, making generated explanations unreliable and potentially misleading for real-world deployment.
Abstract
from arXiv · showhide
Large language models are increasingly capable of generating fluent-appearing text with relatively little task-specific supervision. But can these models accurately explain classification decisions? We consider the task of generating free-text explanations using human-written examples in a few-shot manner. We find that (1) authoring higher quality prompts results in higher quality generations; and (2) surprisingly, in a head-to-head comparison, crowdworkers often prefer explanations generated by GPT-3 to crowdsourced explanations in existing datasets. Our human studies also show, however, that while models often produce factual, grammatical, and sufficient explanations, they have room to improve along axes such as providing novel information and supporting the label. We create a pipeline that combines GPT-3 with a supervised filter that incorporates binary acceptability judgments from humans in the loop. Despite the intrinsic subjectivity of acceptability judgments, we demonstrate that acceptability is partially correlated with various fine-grained attributes of explanations. Our approach is able to consistently filter GPT-3-generated explanations deemed acceptable by humans.
1 Introduction
The paper shows that few-shot GPT-3 prompting can produce surprisingly competitive free-text explanations, while a human-supervised acceptability filter improves their quality. This reframes crowd annotators from explanation authors into evaluators of machine-generated explanations.
- The proposed pipeline overgenerates five explanations per instance from prompts authored by the researchers, then uses binary human acceptability judgments to filter implausible outputs.Evaluation is conducted at both the explanation and instance levels.
- High-quality written supervision is difficult and expensive because crowdsourced datasets often contain logically inconsistent, ungrammatical, and heterogeneous explanations.More than 70% of existing free-text explanation datasets are crowdsourced.
- Human studies find that GPT-3 explanations still need improvement in avoiding repetition, supporting the label, and overall acceptability.Fewer than half of greedily decoded explanations were acceptable with unanimous human agreement.
- The filtration model consistently selects human-identified acceptable explanations and improves label support, novel information, and overall acceptability over strong baselines.These gains hold despite the intrinsic subjectivity of acceptability ratings.
- Few-shot GPT-3 prompting produces surprisingly competitive explanations, offering a promising alternative to crowd-authored free-text explanation corpora.
2 GPT-3 is Competitive with Crowdsourced Explanation Datasets
Few-shot GPT-3 explanations become competitive with crowdsourced explanations when prompted with higher-quality, author-written examples. They are strong in surface-level quality but remain weaker in novelty and label support, which are related to overall acceptability.
- 2.1 Dataset comparison: GPT-3 explanations were frequently preferred or comparable to CoS-E explanations, while remaining competitive with ECQA and e-SNLI in 47.3% and 36.4% of cases.ECQA and e-SNLI explanations were generally preferred, but GPT-3 was not uniformly worse than crowdsourced explanations.
- 2.2 Prompt quality: Author-written explanations produced better GPT-3 generations than crowdsourced explanations used as prompts for all three corpora.The head-to-head evaluation demonstrates that prompt quality materially affects generated explanation quality.
- 2.2 Prompt quality: With author-written prompts, GPT-3 explanations were at least as good as human-written explanations more than half the time across all three datasets.These generations were more competitive than those produced with dataset prompts, motivating the use of author-written explanations in subsequent experiments.
- 2.3 Explanation attributes: GPT-3 explanations scored strongly on generality, grammaticality, and factuality, but provided less new information and weaker label support than the strongest crowdsourced datasets.New-information means were 0.1 for GPT-3 on both tasks versus 0.6 for ECQA and 0.2 for SNLI, with statistically significant differences at p ≤0.01.
- 2.3 Explanation attributes: Acceptability was positively correlated with every measured attribute, most strongly with grammaticality, generality, and support for the label, and least strongly with new information.GPT-3 explanations were judged more acceptable than CoS-E and ECQA explanations but not e-SNLI explanations.
3 Beyond Greedy Explanations
Sampling additional GPT-3 explanations substantially increases the chance that a set contains at least one explanation acceptable to all three annotators, despite sampled explanations being individually less acceptable than greedy outputs.
- 3 Beyond Greedy Explanations: Only 46.3% of greedy CommonsenseQA explanations and 31.5% of greedy NLI explanations were rated acceptable by all three annotators, motivating alternatives to greedy decoding.The goal is to provide the most unambiguously acceptable explanation when multiple end-users may receive the system output.
- 3 Beyond Greedy Explanations: The method samples four additional GPT-3 explanations per instance and obtains three crowdsource acceptability annotations for each new explanation.This stochastic sampling strategy was motivated by prior work on other generation tasks (Holtzman et al., 2020; Massarelli et al., 2020; Holtzman et al., 2021).
- 3 Beyond Greedy Explanations: 79.5% of CommonsenseQA and 51.2% of NLI instances had at least one 3/3-acceptable explanation after adding sampled generations, versus 46.3% and 31.5% for greedy outputs.The set comprised the greedy explanation plus four sampled explanations per instance.
- 3 Beyond Greedy Explanations: Sampled explanations individually achieved lower 3/3 acceptability than greedy explanations: 25.1% for CommonsenseQA and 11.3% for SNLI.Despite this lower individual acceptability, sampled candidates improve the chance that at least one explanation in the five-output set is acceptable.
4 Improving Explanation Generation with Acceptability Filtering
The section reframes annotators as binary acceptability judges and trains a supervised filter to select acceptable explanations from GPT-3’s stochastic candidates. The filter outperforms GPT-3 likelihood baselines, improves several content-related explanation traits, and raises preference for GPT-3 explanations on CommonsenseQA.
- Classifier performance: On CommonsenseQA, T0-3B beats the NLL baseline by 7–8% in instance-level accuracy and 18% in explanation-level AP on the test set.Across both tasks and validation/test sets, T0-3B substantially outperforms GPT-3’s negative log-likelihood acceptability score.
- Classifier performance: Full models outperform explanation-only baselines by more than 5 points across metrics, indicating that instance–explanation feature interactions help predict human acceptability.Explanation-only models nevertheless perform surprisingly well, suggesting that factuality and grammaticality may be latent factors in human judgments.
- Human evaluation: Filtered explanations more often introduce new information, support the label, and provide sufficient information than greedy explanations, although greedy outputs retain some surface-level advantages.Greedy explanations remain stronger in grammaticality and, for CommonsenseQA, factuality; these differences are statistically significant.
- Human evaluation: Replacing greedy with filtered GPT-3 explanations increases preference for GPT-3 by 15–24% on both CommonsenseQA datasets, but not on SNLI.The smaller SNLI effect coincides with fewer explanations changing after filtering and lower overall acceptability.
- Summary: The method selects unanimously acceptable GPT-3 candidates using about 5k binary annotations, while the gap to an upper-bound oracle leaves substantial room for improvement, especially on SNLI.Consensus labels sometimes improve performance and usually reduce seed variance, but require three times as many annotations; performance transfers comparably to separately annotated NLI test data.
5 Related Work
Prior work spans supervised, unsupervised, weakly supervised, and template-based free-text explanation generation, while concurrent studies examine prompting and model size. This work extends human-preference modeling to explanation acceptability, with distinct success conditions and evaluation requirements.
- Free-text explanation generation: Earlier explanation generators targeted vision and NLI, primarily using supervised datasets, while later work explored supervised or unsupervised task-improvement settings.These directions include methods for generating explanations or clarifications in supervised and unsupervised manners.
- Free-text explanation generation: Other approaches use only task signal, weak supervision, or hand-crafted templates with mask-infilling to generate free-text or contrastive explanations.Latcinnik and Berant (2020) use task signal, Brahman et al. (2021) use weak supervision, and Paranjape et al. (2021) use templates with pretrained language models.
- Free-text explanation generation: Concurrent work finds GPT-3 Davinci outperforms smaller pretrained models but remains less plausible to crowdworkers than dataset explanations; this work studies finer criteria and improves performance through prompts and filtering.The comparison concerns prompted explanation plausibility and aligns with the paper’s first experimental result (Table 3).
- Supervising on human preferences: Unlike prior human-preference modeling for summarization, commonsense knowledge bases, and NLI datasets, this work applies acceptability filtering to explanations that must produce a prediction for every instance.In contrast, causal knowledge-graph generation can discard bad generations, whereas this task differs in domain, success conditions, and evaluation metrics.
6 Conclusion
GPT-3 can generate free-text explanations for NLP instances in few-shot settings, and an overgenerate-and-filter pipeline further improves this capability using human acceptability supervision. The authors also identify trust and misleading explanations for incorrect labels as important directions for future investigation.
- Conclusion: GPT-3 generates free-text explanations for NLP task instances in a few-shot setting, and overgenerate-and-filter further improves this capability using human acceptability ratings.The filter is trained on supervision from human acceptability judgments.
- Conclusion: Future work should investigate free-text explanations through neural or neurosymbolic systems and further examine the benefits of counterfactual explanations.The authors point to Brahman et al. (2021), Majumder et al. (2021), and Saha et al. (2021) as relevant future directions.
- Conclusion: Modeling human acceptability may support user trust, but that trust may be unwarranted, especially when explanations accompany incorrect label predictions and could mislead end users.Human rationales for decision making are not necessarily the same as model rationales.
7 Ethics & Broader Impacts · A Prompt Construction · B Crowdsourcing Details
The paper limits ethical risks by documenting potential overtrust, unreliable explanations, and population-dependent acceptability judgments. It also details a robust few-shot prompt-construction strategy and organizes crowdsourcing procedures around interfaces, quality control, payment, and annotator statistics.
- 7 Ethics & Broader Impacts: The study uses public datasets, plans to release collected explanations and annotations, and does not collect personal information from participants.These practices define the study’s stated data and participant-privacy safeguards.
- 7 Ethics & Broader Impacts: GPT-3 explanations conditioned on gold labels may cause humans to place unwarranted trust in AI systems, motivating attention to explanation faithfulness [Hase et al., 2020; Wiegreffe et al., 2021].The paper connects this risk to research on the faithfulness of model-generated free-text explanations.
- 7 Ethics & Broader Impacts: GPT-3 can explain answer choices it did not predict equally well, indicating that its explanations are unreliable.The paper suggests sub-optimal decoding and factual knowledge about alternative choices may partly explain this behavior [Zhao et al., 2021; Holtzman et al., 2021].
- 7 Ethics & Broader Impacts: Acceptability judgments may vary significantly across population groups because the model reflects aggregate judgments from primarily Western, English-speaking crowdsourcing participants.This limitation makes the human acceptability model population-dependent.
- A Prompt Construction: Prompt construction preserves a true few-shot setting by avoiding tuning on full training and development sets and using at most 115 sampled training instances for method development.Initial design used no more than 10 layouts with 15 examples per CoS-E and e-SNLI; later design sampled 6 prompts from those examples.
- A Prompt Construction: Prompt capacity is limited by GPT-3’s 2049-token API limit, allowing at most 24 examples for CoS-E, e-SNLI, and handwritten explanations, and 16 for ECQA.The added explanations lengthen each prompt instance, reducing the number of examples the API can accommodate.
- A Prompt Construction: The prompting strategy samples additional hyperparameters rather than tuning them, aiming for robust expected-performance estimates when no large validation set is available.This general strategy addresses variance from example order, count, and selection [Zhao et al., 2021; Lu et al., 2022].
- B Crowdsourcing Details: Crowdsourcing procedures are documented through shared study-design details, head-to-head, absolute, and acceptability interfaces, plus quality control, payment, and annotator statistics.The appendix assigns these topics to §§B.1–B.6.
B.1 Shared Interface Details … B.4 Acceptability Interface Details
The studies expose participants to the task input and gold label while using interfaces tailored to pairwise comparison, attribute-based evaluation, and acceptability filtering. Absolute evaluation separates context-independent qualities from context-dependent usefulness and sufficiency judgments.
- B.1 Shared Interface Details: All human-study interfaces show the input instance and gold label alongside the explanation(s), treating the gold label as a methodological control.Participants are not shown other incorrect answer choices, helping avoid confounds from subjective or ambiguous alternatives, especially in CommonsenseQA.
- B.2 Head-to-Head Interface Details: The head-to-head interface asks workers to select which of two explanations best explains the answer while treating the gold label as correct and ignoring minor grammar or spelling errors.Figures 5 and 6 document the evaluation interface.
- B.3 Absolute Interface Details: The absolute evaluation interface contains seven questions divided between context-independent and context-dependent evaluation.The interface was iteratively refined over three to four versions to improve agreement rates.
- B.3 Absolute Interface Details: Context-independent evaluation assesses factuality, generality, and grammaticality without revealing the question or context the explanation addresses.Factuality judgments also distinguish statements whose truthfulness can be assessed from those requiring more information.
- B.3 Absolute Interface Details: Context-dependent evaluation reveals the question and gold answer, then tests whether the explanation adds new facts, information, or reasoning beyond them.This distinguishes useful explanations from correct but merely declarative restatements.
- B.3 Absolute Interface Details: The remaining context-dependent questions assess whether the explanation provides enough information to justify the answer and whether it is acceptable overall.Questions about adequacy are asked conditionally after added information or reasoning is identified, enabling finer-grained error analysis than a single informativeness judgment.
- B.4 Acceptability Interface Details: The binary acceptability interface collects training and test data for the overgeneration filter model.For correlation analysis, 250 test explanations from five data sources receive three annotations each, totaling 3750 datapoints before conditional exclusions.
B.5 Quality Control and Payment
The study used AMT with standardized hourly-equivalent compensation, geographic and experience-based screening, task-specific qualification exams, and ongoing annotation-quality monitoring. The authors acknowledge that geographic eligibility is a broad and likely suboptimal proxy for English competency.
- Payment: Workers were paid at a target rate of $15/hour, with initial payments of $0.12 per head-to-head HIT, $0.25 per absolute-evaluation HIT, and $0.20 per acceptability HIT.The researchers periodically checked median completion times to maintain the target rate, although annotators tended to speed up over time.
- Eligibility: Annotators were restricted to five English-speaking countries, required >98% approval across >5000 approved HITs, and excluded if previously block-listed.The geographic restriction was used as a proxy for English competency.
- Limitations: The country restriction was acknowledged as a broad and likely suboptimal assumption, with prior experience suggesting that broadening it still yielded >90% of annotators from the US or Canada.This limitation reflects uncertainty in using geography as a language-competency proxy.
- Quality Control: Annotations were quality-controlled through qualification exams, inter-annotator agreement, completion-time checks, removal and re-annotation of unreliable work, and recruitment of many distinct crowdworkers.Agreement was measured with Krippendorff’s α, while hidden JavaScript tracked time per HIT.
B.6 Statistics
This section reports annotation statistics, including crowd annotator counts and HIT completion summaries across experiments. It also points to tables containing acceptability-annotation statistics and more detailed inter-annotator agreement breakdowns.
- B.6 Statistics: Table 17 reports the number of distinct crowd annotators and the median and mean numbers of HITs completed for each experiment.These statistics summarize annotator participation across the experiments.
- B.6 Statistics: Table 13 presents statistics for the acceptability annotations.
- B.6 Statistics: More detailed breakdowns of inter-annotator agreement for some experiments appear in Tables 14 and 15.
C Absolute Evaluation by Label Accuracy · D 2/3 Acceptability Statistics · E Filter Model Details
GPT-3 explanations showed little attribute variation between correctly and incorrectly predicted labels, limiting their faithfulness, while acceptability was high for greedy CommonsenseQA generations and often recoverable across five candidates. The filter model used leakage-safe instance splits and standard sequence-model training configurations.
- C Absolute Evaluation by Label Accuracy: 50.8% CommonsenseQA label accuracy exceeded the 20% random baseline when GPT-3 generated label predictions without the WHY? token or gold explanations.
- D 2/3 Acceptability Statistics: Across five candidate explanations, 97.7% of CommonsenseQA instances and 79.5% of SNLI instances had at least one acceptable explanation.
- E Filter Model Details: The filter data used 4,955 distinct annotated CommonsenseQA explanations and 5,000 SNLI explanations, split into train and development sets with all five explanations per instance kept together.The instance-level metric averaged scores computed over all five explanations for each instance.
- E Filter Model Details: The low SNLI label-prediction performance aligns with prior findings that GPT-3 struggles with sentence-comparison tasks (Brown et al., 2020; Zhao et al., 2021).
- C Absolute Evaluation by Label Accuracy: Explanations varied little across correctly versus incorrectly predicted labels, indicating that GPT-3 was not faithful enough for real-world use in its current form.Figure 4 reports this absolute evaluation for CommonsenseQA and SNLI.
- E Filter Model Details: The filter classified explanations as good or bad using either explanation-only inputs or inputs containing the question and gold label, with T5-Large and T0-3B training configurations specified.T5-Large used learning rate 1E −4, batch size 64, early stopping, and up to 200 epochs; T0-3B used batch size 50.
F Additional Filter Results
Relaxing the acceptability threshold from unanimous agreement to 2/3 annotators produces results comparable to the main experiments. Under this definition, T0-3B still outperforms the baselines on select-1 accuracy and average precision (AP).
- Additional threshold results: The relaxed evaluation labels an explanation acceptable when at least 2/3 annotators agree, rather than requiring agreement from all 3 annotators.The main experiments used a 3/3 annotator threshold.
- Additional threshold results: Overall, relaxing the threshold yields results comparable to those obtained with the original unanimous-agreement criterion.The comparison covers both select-1 accuracy and AP.
- Additional threshold results: T0-3B outperforms the baselines on both select-1 accuracy and AP when acceptability requires at least 2/3 annotators.This result is reported for acceptability classifiers trained on CommonsenseQA and SNLI.