Source-linked AI summary
Supervising Model Attention with Human Explanations for Robust Natural Language Inference
Joe Stacey, Yonatan Belinkov, Marek Rei
TL;DR
NLI models can learn dataset artefacts that impair out-of-distribution generalisation, while prior debiasing can come at an in-distribution performance cost. The paper trains models to attend to words identified by human explanations, improving performance in- and out-of-distribution and achieving a new state-of-the-art SNLI result with DeBERTa.
Problem
NLI models can learn dataset-specific artefacts that affect how well they generalise to out-of-distribution examples.
Source: Introduction · p. 1Method
The approach supervises model attention weights to focus on words identified in human explanations.
Source: Introduction · p. 1, Supervising Self-Attention Layers · p. 3Results
The approach improves in- and out-of-distribution performance, and with DeBERTa achieves a new state-of-the-art result for SNLI.
Source: Introduction · p. 2, Conclusion · p. 7Takeaways & Limitations
The supervised models have more interpretable attention, focusing more on important words, especially nouns, verbs, and adjectives.
Source: Conclusion · p. 7
Abstract
from arXiv · showhide
Natural Language Inference (NLI) models are known to learn from biases and artefacts within their training data, impacting how well they generalise to other unseen datasets. Existing de-biasing approaches focus on preventing the models from learning these biases, which can result in restrictive models and lower performance. We instead investigate teaching the model how a human would approach the NLI task, in order to learn features that will generalise better to previously unseen examples. Using natural language explanations, we supervise the model's attention weights to encourage more attention to be paid to the words present in the explanations, significantly improving model performance. Our experiments show that the in-distribution improvements of this method are also accompanied by out-of-distribution improvements, with the supervised models learning from features that generalise better to other NLI datasets. Analysis of the model indicates that human explanations encourage increased attention on the important words, with more attention paid to words in the premise and less attention paid to punctuation and stop-words.
Introduction
NLI models can exploit dataset artefacts that hurt out-of-distribution generalisation, while prior debiasing often trades away in-distribution performance. The paper uses human explanations to guide attention, aiming to improve robustness without sacrificing in-distribution results.
Introduction
- Dataset artefacts can undermine NLI models’ out-of-distribution generalisation, while prior debiasing approaches often improve it at the expense of in-distribution performance.Source: Introduction · p. 1
- Training with e-SNLI human explanations offers a different route to robustness than preventing models from learning dataset biases.The authors aim to address prior explanation-generation approaches’ computational expense and out-of-distribution performance reduction.Source: Introduction · p. 1
- The paper explores supervising existing self-attention, adding a supervised attention layer, and adapting an explanation-based attention method for NLI.Each approach identifies important premise and hypothesis words using e-SNLI explanations.Source: Introduction · p. 1
- The proposed approach improves in- and out-of-distribution NLI performance; combined with DeBERTa, it achieves a new state-of-the-art SNLI result.The model’s attention weights predict explanation words at prior-work performance and focus more on important premise words and less on hypothesis stop-words.Source: Introduction · p. 2
Related Work
Prior NLI explanation methods have generally failed to improve performance across distributions, while explanation-guided attention has shown benefits in other tasks. This paper positions its approach against those methods and against NLI debiasing strategies that target dataset artefacts.
Training NLI Models with Explanations
- Earlier NLI explanation approaches often failed to improve performance in-distribution or out-of-distribution; the paper aims to achieve both benefits within a single model.Prior methods include retrieval, class-specific explanation generation, and multi-model pipelines.Source: Training NLI Models with Explanations · p. 2
Training with Explanations Beyond NLI
- Unlike prior explanation-guided attention work, the paper individually selects attention heads, uses mean squared error, and evaluates out-of-distribution performance for NLI.It also uses free-text explanations or highlighted words and trains on 550,152 observations, compared with prior work’s 1,200 and 2,500 examples across two tasks.Source: Training with Explanations Beyond NLI · p. 2
Creating More Robust NLI Models
- Common NLI debiasing strategies use a weak model to identify biases, then reduce its influence through model-similarity constraints or example weighting.Other approaches include adversarial training, data augmentation, minority-example fine-tuning, gradient supervision, multi-task learning, and compressed representations.Source: Creating More Robust NLI Models · pp. 2–3
Attention Supervision Method
The method turns e-SNLI human explanations into token-level relevance targets for supervising transformer attention, using both existing self-attention and an additional attention layer. It also selects which existing attention heads to supervise and evaluates the approach across in- and out-of-distribution NLI sets.
Attention Supervision Method
- e-SNLI explanations become binary token-relevance labels, used to identify words in free-text explanations or annotator-highlighted words; the explanations are used only during training.Stop-words are excluded when matching free text, but highlighted stop-words remain selected.Source: Attention Supervision Method · p. 3
Supervising Self-Attention Layers
- The final transformer self-attention layer’s [CLS] weights are trained toward normalized explanation labels with an auxiliary loss, a choice that performed better than supervising earlier layers.The total loss combines the NLI cross-entropy with mean-squared error over supervised heads, weighted by λ.Source: Supervising Self-Attention Layers · p. 3
Selecting Attention Heads for Supervision
- Heads are tested individually before the top K are selected from K values of 1, 3, 6, 9, or 12 using five random seeds; this greedy selection may miss the optimal subset.The approach is intended to retain diversity between supervised and unsupervised heads.Source: Selecting Attention Heads for… · p. 3
Supervising an Additional Attention Layer
- An alternative adds an attention layer over transformer sequence representations, supervises its normalized weights, and feeds its resulting representation to a linear classifier and softmax.The added layer uses a single attention head for the loss.Source: Supervising an Additional Attention… · p. 4
Experimental Setup and Evaluation
- Evaluation uses BERT and DeBERTa, tunes λ on validation data, and tests significance on in- and out-of-distribution NLI sets.λ is 1.0 for BERT and 0.8 for DeBERTa; reported accuracy averages 25 random seeds across SNLI, MNLI, ANLI, and HANS sets.Source: Experimental Setup and Evaluation · p. 4, Table 1 · p. 4
Experiments
The experiments report gains in both in-distribution and out-of-distribution NLI performance, including with DeBERTa. They also compare explanation formats and attention-head choices, finding the strongest performance with combined explanations and supervision of three heads.
Performance in and out of Distribution
- SNLI-test accuracy improves by 0.4% and SNLI-hard by 0.79%; out-of-distribution accuracy gains are 0.84% and 0.91% on the MNLI validation sets and 1.59% on HANS, but ANLI does not improve.The SNLI-test and SNLI-hard gains are significant at p-values below 10^-8; the authors note ANLI uses multiple sentences per premise.Source: Performance in and out of Distribution · p. 4
Experiments with DeBERTa
- With DeBERTa, supervision reaches 92.69% SNLI performance versus 92.59% for DeBERTa alone, a statistically significant 0.1% gain and a new SNLI state of the art.The authors note the absolute gain is smaller than BERT’s 0.40% improvement, with less room to improve.Source: Experiments with DeBERTa · p. 5
Comparing Results with Prior Work
- Attention supervision improves SNLI accuracy by 0.40% without adding parameters; LIREx-adapted gains on SNLI but has MNLI performance of -0.85, while the Pruthi-adapted method improves SNLI, MNLI, and SNLI-hard.LIREx-adapted uses 453m parameters versus 109m in the BERT baseline, and its SNLI-test accuracy is 90.79% versus 92.69% for supervised DeBERTa.Source: Comparing Results with Prior Work · p. 5
Choosing Which Explanations to Use and Which Heads to Supervise
- Combining free-text explanations with highlighted words gives the greatest performance improvements; when highlights cover only hypothesis words, free-text attention also supervises the premise.The combined explanation distribution averages the free-text and highlighted-word attention distributions.Source: Table 3 · p. 5, Choosing Which Explanations to Use and… · pp. 5–6
- Supervising all attention heads improves performance, but supervising the top three performs best; those three heads also outperform the remaining heads when supervised individually.The authors suggest that supervising all heads in the same direction can adversely affect heads with different functions.Source: Choosing Which Explanations to Use and… · p. 6, Figure 3 · p. 6
Analysis
The supervised model improves token-level explanation prediction and shifts attention toward the premise and words highlighted as important. The analysis links this shift to more interpretable attention across both sentences.
Token Level Classification
- Hypothesis F1 improves by 3.1 points over previous baselines, and token-level predictions outperform a jointly supervised LSTM-CRF model.The passage also notes that a perturbation-based approach achieves higher premise F1 but did not improve the overall NLI task.Source: Token Level Classification · p. 6
Understanding the Changes in Attention
- Premise-plus-first-[SEP] attention rises from 22.86% to 50.89% with 12 supervised heads; preceding-layer attention rises from 31.1% to 54.2%, or 54.8% with three top-layer heads.The authors suggest the increased premise focus may explain better SNLI-hard performance and say the three-head result supports supervising fewer heads.Source: Understanding the Changes in Attention · p. 6
Words Receiving Most Attention
- Figure 4 illustrates attention to “grin” and “happy,” then “person” and “child”; unlike the baseline’s focus on hypotheses and special tokens, premise attention makes sentence interactions more interpretable.On the second example, the supervised model predicts neutral correctly while the baseline predicts contradiction.Source: Figure 4 · p. 7, Words Receiving Most Attention · p. 7
- Supervision shifts attention toward nouns, verbs, and adjectives and away from punctuation, determiners, and adpositions; nouns make up 46% of explanation-highlighted words.The most-attended words are often sentence-subject nouns.Source: Words Receiving Most Attention · p. 7
Conclusion
The paper finds its best performance when supervising the model’s existing self-attention weights using human explanations. This approach improves in- and out-of-distribution performance, and the DeBERTa combination achieves a new state-of-the-art result.
Conclusion
- Supervising existing self-attention with human explanations improves performance both in-distribution and out-of-distribution, with the best results from this approach.The approach encourages attention to words considered important in human explanations.Source: Conclusion · p. 7
- Combined with DeBERTa, the approach achieves a new state-of-the-art result.Source: Conclusion · p. 7
- Supervised attention focuses more on important words, mostly nouns, verbs, and adjectives, while the baseline attends more to special tokens, stop-words, and punctuation.The paper describes the supervised attention weights as more interpretable.Source: Conclusion · p. 7