Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU Models

1 Introduction

• Lexical shortcut reliance can lower NLU models’ out-of-distribution generalizability and adversarial robustness.The paper focuses on spurious correlations between shortcut words and labels.Source: §1 · p. 1
• The long-tailed account finds that models favor head features, learn tail features poorly, and pick up shortcuts during very early training iterations.The authors use these observations to explain shortcut learning and motivate measuring shortcut degree per training sample.Source: §1 · p. 1
• LTGR uses knowledge distillation to penalize overconfident predictions on training samples with large shortcut degree.The stated implicit effect is to downweight shortcut patterns and encourage attention to task-relevant features.Source: §1 · pp. 1–2
• The paper reports that LTGR improves OOD generalization while preserving in-distribution accuracy, and can partially mitigate shortcut-based Trojan attacks.The Trojan-attack result is described as a preliminary experiment.Source: §1 · p. 2

2 Long-Tailed Phenomenon

• Training, validation, and test sets share an IID split, so shortcut-learning models may perform reasonably on them yet generalize poorly to OOD data.The section motivates examining how simple features that reduce training loss most can contribute to low generalization.Source: §2.1 · p. 2, §2 · p. 2

2.1 Preference for Features of High Local Mutual Information

• Per-label LMI word statistics form long-tailed distributions: their head typically contains low-information functional words, while the less frequent tail contains higher-information words.The model’s attributions are compared with these statistics; the reported observation is that models rely heavily on high-LMI head features and usually ignore more complex tail features.Source: §2.1 · pp. 2–3

2.2 Shortcuts Samples are Learned First

• Shortcut features at the head are learned very early and rapidly reduce training loss, while tail features are learned later.The section compares an early snapshot, taken after the first epoch, with the final converged model; matching predictions are assessed through cosine similarity of their Integrated Gradient explanations.Source: §2.2 · p. 3

2.3 Shortcut Degree Measurement

• The proposed shortcut degree combines a feature-based measure and a learning-dynamics measure, then normalizes their sum to [0, 1].The feature-based measure marks samples whose highest- or second-highest-attribution word is in the distribution’s top 5%; the dynamics measure compares early and final snapshots.Source: §2.3 · p. 3, §2.1 · p. 3, §2.2 · p. 3

3 Proposed Mitigation Framework

• LTGR smooths the teacher’s probability distribution according to each training sample’s shortcut degree, aiming to suppress overconfident predictions on shortcut-heavy samples.The stated rationale is that models over-associate shortcut features with class labels; smoothing is intended to down-weight reliance on those features and encourage attention to task-relevant ones.Source: §3 · p. 4
• At shortcut degree 0, the teacher probabilities remain unchanged; at 1, they become equal across labels, with intermediate values producing greater smoothing as the degree increases.The method uses the total number of labels when smoothing the distribution.Source: §3 · p. 4
• A student is trained with a loss balancing smoothed teacher probabilities and ground-truth labels; the fixed teacher is discarded and the student makes predictions.The teacher and student use the same architecture, and the procedure first trains and fixes the teacher before training the student.Source: §3 · p. 4

4.1 Experimental Setup

• MNLI, FEVER, and MNLI-backdoor were evaluated with adversarial or OOD sets, including HANS, MNLI hard, Sym1/Sym2, and a hard-backdoor set with an intentionally added trigger.The MNLI-backdoor training set adds the rare quotation-mark trigger to 10% of entailment samples; all hard-backdoor test samples receive it.Source: §4.1 · pp. 4–5

4.2 Shortcut Behaviour Analysis

• The top integrated-gradient word fell in the 5% distribution head for 25.3% of MNLI samples, and the largest-attribution word was in one input branch for 75%-99% of samples across labels.The head commonly contains function words, punctuation, numbers, and contradiction markers; Table 1 reports head-word ratios, while Table 2 concerns branch preference.Source: §4.2 · p. 6, Table 1 · p. 6, Table 2 · p. 6
• In first-epoch tests, easy or shortcut samples reduced validation loss and increased accuracy, whereas hard samples could increase loss and reduce accuracy; the learning curves support that shortcut samples are learned first.Figure 3 compares easy-first and hard-first orders across 10 checkpoints in the first epoch.Source: §4.2 · pp. 6–7, Figure 3 · p. 7

4.3 Mitigation Performance Analysis

• LTGR increased OOD accuracy by 0.68% to 5.86% on MNLI and 1.54% to 3.87% on FEVER while maintaining in-distribution accuracy.The comparison is reported for FEVER and MNLI in Tables 3 and 4; on MNLI-backdoor, LTGR also improved predictions for contradiction and neutral samples from 0.0% to 30.45% and 17.53%, respectively, while MNLI validation accuracy was 81.96% versus 82.37%.Source: §4.3 · pp. 7–8, Table 3 · p. 7, Table 4 · p. 7
• After mitigation, the model attended to both premise and hypothesis, but it still attended strongly to shortcuts for some samples.Figure 4 illustrates the attention shift; the authors identify stronger architectural inductive bias or domain knowledge as possible future directions.Source: §4.3 · p. 8, Figure 4 · p. 8

4.4 Ablation and Hyperparameters Analysis

• Ablations using only head preference or learning dynamics generalized worse than LTGR, while randomly assigning shortcut labels could lower hard-validation accuracy by 1.72%.The authors conclude that the two measurements provide complementary information; stronger α improved MNLI hard-validation generalization, but α = 0.9 could sacrifice accuracy.Source: §4.4 · pp. 8–9

• Prior analyses find that neural models capture superficial patterns rather than task-relevant features across tasks including NLI, question answering, reading comprehension, and VQA.These analyses commonly use adversarial test sets and model explainability; this work focuses on lexical bias.Source: §5 · p. 9
• Many mitigation approaches combine expert knowledge with data-driven training, using bias-only teachers, reweighting, augmentation, explanation regularization, or adversarial training.Examples include hypothesis-only or bag-of-words models for NLI and question-only models for VQA.Source: §5 · p. 9
• Unlike methods that require prior knowledge of the bias type, LTGR is motivated by the long-tailed phenomenon and does not rely on a bias-only network.The paper relates its account to reported high-frequency preference in CNNs, including texture bias.Source: §5 · p. 9

• LTGR improves OOD generalization on several NLU benchmarks without sacrificing in-distribution accuracy.The paper also reports that long-tailed distributions explain shortcut behavior and that shortcuts are learned early in training.Source: §6 · p. 9
• LTGR suppresses overconfident predictions on samples with large shortcut degree, but models still rely on shortcut features to some extent.The authors suggest that architectural inductive bias or domain knowledge could further alleviate this reliance.Source: §6 · p. 9

• BERT-base is a lower-cased English model with 12 layers, 110M parameters, and 768-dimensional contextual word representations.The authors use bert-base-uncased as the tokenizer.Source: A Model architectures · p. 13
• DistilBERT is described as a smaller, faster BERT variant trained by distillation from BERT-base.It has 40% fewer parameters, runs 60% faster, and preserves over 95% of BERT's GLUE performance, as reported in the passage.Source: A Model architectures · p. 13
• The classification head adds a bidirectional LSTM of hidden size 150, followed by max pooling and fully connected layers of sizes 100 and 3.The three output classes correspond to MNLI, FEVER, and MNLI-backdoor.Source: A Model architectures · p. 13

• Shortcut words such as stop words, negations, punctuation, and numbers tend to be frequent and low-information, placing them at the distribution head; task-relevant words tend toward the tail.The account describes MNLI and FEVER inputs as covering diverse topics and semantics.Source: B More on Long-tailed Distribution · p. 13
• For FEVER, the claim-only model is 17.9% lower than the full model, while the evidence-only model falls below the 33.33% random-guess accuracy.The passage attributes long-tailed artifacts partly to annotation and says the phenomenon may be less significant in tasks not labeled by crowd workers.Source: B More on Long-tailed Distribution · p. 13, Table 7 · p. 13
• The analysis uses a word-level rather than word-and-phrase distribution because Integrated Gradient results indicate that most examples focus on a single shortcut word.Source: B More on Long-tailed Distribution · p. 13
• Switching the optimizer from Adam to SGD still leaves observable early learning of shortcut instances.The switch used the same momentum and learning rates.Source: B More on Long-tailed Distribution · p. 13

C More Analysis on FEVER & MNLI

• In FEVER and MNLI, models focus mainly on the claim and hypothesis branches, respectively, and attend to functional-word shortcuts such as “only,” “and,” and “yet.”These words also correspond to high-LMI words in the long-tailed distribution.Source: C More Analysis on FEVER & MNLI · p. 13
• The word “only” can express negation in both datasets, appearing with refutes in FEVER and contradiction in MNLI.The authors identify this overlap as a shared annotation artifact.Source: C More Analysis on FEVER & MNLI · p. 13
• Figure 6 presents FEVER and MNLI examples with the model prediction, confidence score, and integrated-gradient explanation visualization.The section uses these visualizations to examine shortcut learning behavior.Source: C More Analysis on FEVER & MNLI · p. 13, Figure 6 · p. 14

D More Analysis on MNLI-backdoor

• With the quotation-mark trigger, the model predicts entailment with high confidence regardless of the ground truth, and the visualization shows highest attention to the trigger.The trigger’s entailment LMI is higher than that of other shortcut features, and the authors report high confidence when strong shortcuts are present.Source: D More Analysis on MNLI-backdoor · p. 15
• Poisoning only 10% of entailment training samples with the trigger is sufficient for the model to capture the spurious trigger–label correlation.The authors also note that unintentionally inserted annotation artifacts can be captured even without the trigger’s high LMI.Source: D More Analysis on MNLI-backdoor · p. 15
• The authors report that the backdoor behavior meets both stated requirements: normal behavior without the trigger and designer-specified predictions when it is present.They refer to the first row of Table 5 and Figure 7 as evidence for these requirements.Source: D More Analysis on MNLI-backdoor · p. 15
• The quotation-mark trigger can also serve as a DNN watermark, and the authors suggest a rarer trigger such as a stakeholder’s name for this purpose.They describe watermarking as a way to claim model ownership and protect intellectual property.Source: D More Analysis on MNLI-backdoor · p. 15

E Generalizable New Knowledge Beyond the Narrow World of NLU tasks?

• The authors suggest their findings may extend beyond BERT-based NLU to question answering, reading comprehension, and visual question answering.They connect this possibility to annotation and collection artifacts in training data.Source: E Generalizable New Knowledge Beyond… · p. 15
• Crowdsourcing can produce annotation and collection artifacts that skew feature and label distributions, yielding long-tailed training data.The authors say such artifacts also occur in tasks beyond NLU, particularly those with heavy crowdsourcing.Source: E Generalizable New Knowledge Beyond… · p. 15
• The authors argue that models fit to skewed data may replicate or amplify its biases, favor head features, and rely on superficial correlations as shortcuts.They propose that their mitigation framework may be adapted to improve generalization in other tasks.Source: E Generalizable New Knowledge Beyond… · p. 15