Source-linked AI summary
WT5?! Training Text-to-Text Models to Explain their Predictions
Sharan Narang, Colin Raffel, Katherine Lee, Adam Roberts, Noah Fiedel, Karishma Malkan
TL;DR
Neural networks perform strongly on difficult NLP tasks, but their prediction processes are hard to interpret. WT5 trains a text-to-text model to generate a natural-language explanation alongside each prediction, achieving strong benchmark results while supporting limited explanation labels and cross-domain transfer. The approach improves communication about predictions, but its explanations may not reflect the model’s true causal decision process.
Problem
Neural networks can perform well on NLP tasks, yet it remains difficult to determine why they produced particular predictions.
Method
WT5 fine-tunes a pre-trained text-to-text model to output a prediction followed by a natural-language explanation, without requiring a special model architecture.
Results
WT5-11B achieved state-of-the-art results across diverse benchmark datasets, often matching human classification and explanation abilities, while supporting limited labeled explanations and cross-domain transfer.
Takeaways & Limitations
The framework lets models communicate not only predictions but also textual answers to why those predictions were made.
Takeaways & Limitations
WT5 explanations provide only surface-level interpretability and may be plausible-sounding reasons rather than accurate descriptions of causal decision-making.
Abstract
from arXiv · showhide
Neural networks have recently achieved human-level performance on various challenging natural language processing (NLP) tasks, but it is notoriously difficult to understand why a neural network produced a particular prediction. In this paper, we leverage the text-to-text framework proposed by Raffel et al.(2019) to train language models to output a natural text explanation alongside their prediction. Crucially, this requires no modifications to the loss function or training and decoding procedures -- we simply train the model to output the explanation after generating the (natural text) prediction. We show that this approach not only obtains state-of-the-art results on explainability benchmarks, but also permits learning from a limited set of labeled explanations and transferring rationalization abilities across datasets. To facilitate reproducibility and future work, we release our code use to train the models.
1 Introduction
Neural networks achieve strong NLP performance but remain difficult to interpret, motivating models that explain predictions in natural text. WT5 uses the text-to-text framework to produce predictions with explanations while supporting limited explanation labels and cross-domain transfer.
- Large pre-trained language models can achieve superhuman performance on previously difficult NLP tasks.
- Neural-network predictions are difficult to understand, and existing interpretability methods have had questioned reliability.
- WT5 trains a text-to-text model to output a label followed by a natural-language explanation, rather than the label alone.
- The approach supports explanation generation with limited labeled explanations by combining explanation-target examples with label-only examples.
- The model can transfer explanation generation from one domain to out-of-domain data.
2 Approach
WT5 adapts a pre-trained text-to-text model by conditioning explanation generation on an “explain” prefix and appending explanations to prediction targets. The framework supports both abstractive and extractive explanations, while extractive spans are less general for tasks requiring unstated reasoning.
- 2 Approach: WT5 fine-tunes T5 to generate explanations for predictions, naming the resulting model and approach “Why, T5?”T5 is the pre-trained Text-to-Text Transfer Transformer.
- 2.1 Text-to-text framework: Text-to-text models map an input token sequence to an output token sequence autoregressively.
- 2.2 WT5: Prepending “explain” requests an explanation, while omitting it trains the model to produce only the classification label.
- 2.2 WT5: The same training setup enables semi-supervised learning when only some examples have annotated explanations.
- 2.3 Extractive Explanations: WT5 can generate extractive explanations by producing annotated input spans and evaluating their overlap with source text.
- 2.3 Extractive Explanations: Extractive explanations are less general because some valid reasoning, such as pronoun disambiguation, does not appear verbatim in the input.
3 Related Work
Related work includes broad interpretability research, extractive explanation benchmarks, and datasets for abstractive explanations. WT5 differs from closely related systems by using a single text-to-text architecture with pre-training rather than specialized classification and explanation components.
- Interpretability research spans attribution methods, analyzable architectures, attention inspection, adversarial diagnosis, and generation-error analysis.
- Attention-based introspection can provide interpretability information, but its reliability has been questioned.
- The ERASER benchmark consolidates datasets with extractive explanations, whereas WT5 also supports abstractive explanations.
- e-SNLI adds human-annotated explanations to SNLI examples across training, validation, and test sets.
- Compared with e-SNLI work using separate classification and explanation components, WT5 requires no special architecture and uses a pre-trained natural-text generator.
- CoS-E provides abstractive explanations for commonsense question answering, while related work focuses partly on using explanations to improve predictions.
4 Experiments
The experiments evaluate WT5 across classification and explanation benchmarks using automatic metrics and human judgments. Because free-form explanations are difficult to score automatically, the study supplements BLEU and Token F1 with crowd-based support judgments.
- Datasets and tasks: WT5 is evaluated on e-SNLI, CoS-E, Movie Reviews, and MultiRC, which pair classification with abstractive or extractive explanations.
- Automatic evaluation: Accuracy measures prediction quality across all datasets, while BLEU is used for abstractive explanations and Token F1 for extracted explanation spans.
- Automatic evaluation: BLEU scores on CoS-E should be interpreted cautiously because many distributed ground-truth explanations are low quality or nonsensical.
- Human evaluation: Human evaluation presents raters with the input, predicted label, and explanation, asking whether the explanation supports the label.
- Human evaluation: For each dataset, 100 examples receive five independent ratings, with attention checks and majority voting used to filter and classify explanations.
4.3 Training Details
The experiments use T5 Base and 11B configurations with standard fine-tuning procedures and greedy decoding. Training is selected using validation accuracy after overfitting appears.
- Experiments use the Base and 11B configurations of T5.
- Fine-tuning uses AdaFactor with a constant learning rate of 0.001 and model-specific token batch sizes.
- Maximum input lengths are 512 for e-SNLI, 128 for CoS-E, 1024 for MultiRC, and 2048 for Movie Reviews, with 10% dropout.
- Predictions use greedy decoding, and training stops at the validation checkpoint with the highest accuracy after overfitting is observed.
4.4 Results on standard benchmarks
WT5-11B achieves strong classification and explanation performance across the benchmark suite, while explanation quality can remain robust with limited abstractive supervision but degrades faster for extractive explanations.
- 4.4 Results on standard benchmarks: WT5-11B achieves the highest explanation scores on most studied datasets and reaches state-of-the-art accuracy on e-SNLI and Movie Reviews.
- 4.4 Results on standard benchmarks: For CoS-E and MultiRC, WT5-11B is very close in accuracy to T5-11B, which does not generate explanations.
- 4.4 Results on standard benchmarks: WT5-Base generally performs worse than WT5-11B but frequently exceeds previous state-of-the-art results and sometimes human annotations.
- 4.5 Learning from limited explanations: With only 100 explanations, WT5-11B achieves a better e-SNLI BLEU score than the previous state of the art while accuracy remains roughly constant.
- 4.5 Learning from limited explanations: On CoS-E, accuracy and BLEU remain essentially constant as annotated explanations decrease toward 100.
- 4.5 Learning from limited explanations: Extractive explanation quality degrades quickly with fewer annotations: Movie Reviews approaches zero near 100 explanations, while MultiRC requires about 10,000 for reasonable results.
4.6 Transferring across datasets
WT5-11B transfers explanation generation from explanation-annotated datasets to datasets without annotated explanations, including different review domains. Human raters judged most transferred explanations correct while classification accuracy remained reasonable.
- Transfer settings: The study transfers explanations from e-SNLI to MNLI and from Movie Reviews to IMDb, combining annotated and unannotated examples during training.MNLI covers broader natural-language-inference domains than SNLI, while IMDb contains movie reviews from its website.
- Results: 82% of MNLI and 94% of IMDb validation explanations were judged correct by human raters.These explanations were generated for datasets whose examples lacked explanation annotations.
- Results: WT5-11B achieved 91.5% classification accuracy on MNLI and 97.2% on IMDb.The paper describes these accuracies as reasonable for the transferred datasets.
- Cross-domain transfer: Movie Reviews-to-Amazon Reviews transfer extended explanation generation across diverse product categories.Amazon Reviews labels were converted from five-star ratings to binary labels; WT5-11B reached 98.1% classification accuracy.
- Implication: The results suggest transfer can provide a plausible way to learn explanations for datasets lacking annotations, including datasets from another domain.The paper supports this conclusion with transfers to IMDb, MNLI, and Amazon Reviews.
4.7 Transferring across tasks
The paper tests whether explanation abilities learned on e-SNLI can transfer to the different CoS-E task. After task-format alignment and mixed training, WT5-11B generated explanations for most CoS-E validation examples, though not perfectly reliably.
- Experiment: The cross-task experiment transfers explanation generation from e-SNLI to CoS-E, which uses different inputs, labels, and answer formats.e-SNLI is three-class natural-language inference, whereas CoS-E answers questions using five example-specific choices.
- Method: The researchers aligned task formatting by replacing task-specific tokens and adding three fixed choices to e-SNLI inputs.CoS-E tokens were changed to resemble the e-SNLI format before training.
- Training: WT5-11B was fine-tuned on explanation-annotated e-SNLI, explanation-free e-SNLI, and explanation-free CoS-E examples.The model then generated zero-shot explanations for CoS-E.
- Results: WT5-11B produced explanations for most CoS-E validation examples, establishing that zero-shot explanation transfer across tasks is possible.The authors note that transfer was not 100% reliable and describe the displayed CoS-E examples as cherry-picked.
5 Conclusion
The paper concludes that text-to-text training can make models produce explanations alongside predictions, while supporting learning from limited explanations and transfer across domains and tasks. It frames this as progress toward better model communication but acknowledges that the interpretability improvement is only surface-level.
- Conclusion: WT5-11B achieved state-of-the-art results across diverse benchmark datasets and often matched human classification and explanation abilities.The conclusion presents this as the paper’s main empirical outcome.
- Conclusion: The approach supports learning from limited labeled explanations and transferring explanatory capabilities across domains and tasks.These capabilities are summarized as consequences of using the text-to-text framework.
- Interpretation: The authors characterize the work as a small step toward models that communicate more naturally by answering both what a prediction is and why.Their example contrasts sentiment prediction with asking for the reason behind that prediction.
- Limitation: The approach does not guarantee that generated explanations identify the specific reasons behind a model’s prediction.The authors warn that a model could produce a plausible-sounding explanation without accurately describing its causal decision process.
A Amazon Reviews explanations
Table 5 presents non-cherry-picked WT5-11B predictions and explanations for Amazon Reviews across different product categories, with explanatory spans shown in bold.
- Table 5: Table 5 shows non-cherry-picked Amazon Reviews predictions and explanations for different product categories.The examples are based on training WT5-11B.
- Table 5: Boldface marks the explanatory spans selected by WT5-11B.
B Human Study GUIs
Figures 4–7 present the MTurk GUIs used to evaluate explanations across MNLI/e-SNLI, CoS-E, Movie Reviews/IMDB, and MultiRC. Where datasets provide human-generated explanations, evaluations compare them with WT5-generated explanations.
- Evaluation interfaces: Figures 4–7 show MTurk evaluation GUIs for MNLI/e-SNLI, CoS-E, Movie Reviews/IMDB, and MultiRC.The MNLI/e-SNLI explanation is generated by WT5-11B, while the other captions identify validation-set explanations.
- Evaluation procedure: For datasets with human-generated label explanations, the human evaluation compares provided explanations with generated explanations.Several datasets use the same MTurk setup, as indicated in the figure captions.