Source-linked AI summary
Navigating the Shortcut Maze: A Comprehensive Analysis of Shortcut Learning in Text Classification by Language Models
Yuqing Zhou, Ruixiang Tang, Ziyu Yao, Ziwei Zhu
TL;DR
Language models may rely on spurious correlations, while the effects of subtler, more complex shortcuts remain underexplored. This paper proposes a benchmark organized around occurrence, style, and concept shortcuts and evaluates traditional, large, and robust models. It finds that none of the evaluated models is universally robust to these shortcuts.
Problem
Prior shortcut studies often examined explicit, simple cues, leaving the effects of subtler and more realistic shortcuts on language models largely unexplored.
Source: §1 · p. 1Method
The paper proposes a three-class, seven-type shortcut framework and benchmark, then evaluates traditional, large, and state-of-the-art robust models.
Source: §1 · p. 2Results
Across the evaluated model types, none is universally robust to the shortcut types; BERT is vulnerable to all types, and larger models are not necessarily more robust.
Source: §1 · p. 2Takeaways & Limitations
The results point to a need for more sophisticated methods to counteract subtle and intricate shortcuts.
Source: §1 · p. 2The benchmark is limited to sentiment analysis and emotion prediction within text classification, without extending to other language-model capabilities.
Source: §6 · p. 9
Abstract
from arXiv · showhide
Language models (LMs), despite their advances, often depend on spurious correlations, undermining their accuracy and generalizability. This study addresses the overlooked impact of subtler, more complex shortcuts that compromise model reliability beyond oversimplified shortcuts. We introduce a comprehensive benchmark that categorizes shortcuts into occurrence, style, and concept, aiming to explore the nuanced ways in which these shortcuts influence the performance of LMs. Through extensive experiments across traditional LMs, large language models, and state-of-the-art robust models, our research systematically investigates models' resilience and susceptibilities to sophisticated shortcuts. Our benchmark and code can be found at: https://github.com/yuqing-zhou/shortcut-learning-in-text-classification.
1 Introduction
Language models can learn spurious correlations rather than causally relevant features, threatening out-of-distribution generalization. This paper introduces a framework and benchmark for studying subtler shortcuts, then finds vulnerabilities across model types.
1 Introduction
- Spurious correlations can lead language models to rely on coincidental label-linked features instead of causally relevant ones, undermining out-of-distribution generalizability.The introduction illustrates this with review style: training links casual language to high ratings and formal language to low ratings.Source: §1 · p. 1
- Prior work often studied explicit shortcuts such as appended letters, punctuation, or words, leaving subtler, more realistic shortcuts largely unexplored.The resilience of large language models and robust models to these more intricate shortcuts remains an unresolved challenge.Source: §1 · p. 1
- The paper proposes a framework with occurrence, style, and concept classes, comprising seven shortcut types, and exemplifies it with a benchmark based on three public datasets.The framework includes types such as synonym occurrence and concept occurrence that the introduction says had not been investigated in prior research.Source: §1 · p. 2
- Across BERT, Llama models, and state-of-the-art robust models, no model is universally robust to the shortcut types examined.BERT is vulnerable across all types, model size alone does not ensure better robustness, and robust models sometimes outperform LLMs.Source: §1 · p. 2
2 Shortcut Framework and Benchmark
The paper's main contribution is a framework that organizes shortcuts into three classes and seven types, with a three-dataset benchmark to exemplify it and support empirical analysis.
2 Shortcut Framework and Benchmark
- The framework organizes shortcuts into occurrence, style, and concept classes, with seven specific types, and the benchmark exemplifies it for empirical analysis.The methodology can be used to create additional benchmarks with other datasets, classification tasks, and shortcut-construction approaches.Source: §2 · p. 2, Figure 2 · p. 3
2.1 Datasets
- The benchmark uses Yelp reviews, Go Emotions, and Beer palate ratings; Beer aroma and appearance reviews serve as distractors for palate evaluation.Go Emotions is curated to four emotional states, while the Beer analysis uses ratings 0.4, 0.6, 0.8, and 1.0.Source: §2.1 · p. 2
2.2 Shortcut Definition and Construction
- Shortcut strength is controlled by λ, with larger values indicating stronger shortcuts; occurrence and style are illustrated with Yelp and Go Emotions, and concept shortcuts with Beer.The framework defines occurrence, style, and concept shortcut classes and describes their construction using the adopted datasets.Source: §2.2 · pp. 2–3
2.2.1 Occurrence
- Occurrence shortcuts link a term, synonym, or category cue to a label; the benchmark uses “honestly,” 15 synonyms, and country or city mentions.Anti-test sets reverse the shortcut's base-probability distribution, while shortcut strength is varied through λ.Source: §2.2.1 · pp. 3–4
2.2.2 Style
- Style shortcuts associate labels with either formal versus casual register or Shakespeare versus Hemingway writing styles.The styles are generated by rewriting original samples with Llama2-70b; register shortcuts contrast complex, professional phrasing with simple, casual phrasing.Source: §2.2.2 · p. 4
2.2.3 Concept
- Concept shortcuts test whether irrelevant aroma or appearance content affects palate ratings, and whether correlated aroma sentiment acts as a shortcut for palate prediction.For concept occurrence, aroma or appearance reviews are paired with palate reviews; for concept correlation, palate and aroma reviews are paired with matching ratings in the training and normal-test data.Source: §2.2.3 · pp. 4–5
3 Empirical Analyses
The experiments compare shortcut vulnerability across BERT, Llama models, and robust-learning methods, then use SHAP to examine how shortcut tokens influence predictions. All three model groups retain vulnerabilities, although robustness varies by model and shortcut type.
3.1.1 Experiment Settings
- BERT is evaluated on normal and anti-test sets using accuracy and macro F1, with each setting averaged over five runs.The reported Δ compares normal-test and anti-test performance; values nearer 0 indicate greater shortcut robustness.Source: §3.1.1 · p. 5, §3.1.2 · p. 5, Table 1 · p. 5
3.1.2 Experiment Results
- On anti-test datasets, BERT’s prediction accuracy and macro F1 are both more than 10% lower than on test datasets, with statistically significant drops (p < 0.001).As λ increases, the macro F1 difference between test datasets increases in most cases.Source: §3.1.2 · p. 6
3.2.2 Experiment Results
- Llama models have smaller macro F1 drops than BERT but remain vulnerable; larger model size does not consistently improve performance or robustness.Llama2-13b is less robust than Llama2-7b on Yelp style shortcuts, and Llama3-8b shows mixed robustness across shortcut types.Source: §3.2.2 · p. 6, Table 2 · p. 8
3.1.2 Experiment Results; 3.2.2 Experiment Results; 3.3.2 Experiment Results
- BERT, Llama models, and robust methods all show performance drops on anti-test data, and no robust method resists every shortcut type.Some robust methods outperform BERT or LLMs in particular cases, but their advantages are not universal.Source: §3.2.2 · p. 6, §3.1.2 · p. 6, §3.3.2 · p. 7
3.3.1 Experiment Settings; 3.3.2 Experiment Results
- A2R, CR, and AFR each improve resistance over BERT in some cases, but all decline on anti-tests and none is robust across all shortcuts.A2R performs particularly well on category shortcuts but worse than BERT on Yelp style shortcuts; AFR reweights minority groups before retraining the last layer.Source: §3.3.2 · p. 7, §3.3.1 · p. 7, Table 3 · p. 8
3.4 Model Analysis via Explainability
- SHAP analysis links shortcut tokens to label predictions: single terms and category words align with training correlations, while AFR and BERT show different sensitivities.For category shortcuts, city terms favor label 0 and country terms favor label 4; “Austria” has a smaller SHAP value in AFR than in BERT.Source: §3.4 · pp. 7–8, Table 4 · p. 8
4 Related Work
Prior work identifies shortcut learning as reliance on spurious correlations rather than task-relevant features, which can impair out-of-distribution generalization. Earlier text-classification studies often used simple, mechanically inserted patterns, while this paper defines and integrates shortcuts that preserve original text meaning.
4 Related Work
- Studies using adversarial test sets and explainability techniques reveal that DNNs exploit spurious correlations instead of higher-level, task-relevant training features, which can impair generalization to out-of-distribution samples (Jia and Liang, 2017; Du et al., 2019; Wang et al., 2020; Deng et al., 2021).Source: §4 · p. 9
- Earlier shortcut studies used simple patterns or mechanically inserted correlations without preserving the original text semantics, whereas this work defines triggers and integrates them into datasets while retaining meaning.Prior work includes lexical bias in NLU [Du et al., 2021] and prompted shortcuts in LLMs [Tang et al., 2023].Source: §4 · p. 9
5 Conclusion
The paper defines a benchmark for shortcut learning in text classification and evaluates susceptibility to occurrence, style, and concept shortcuts. Its reported experiments find susceptibility across BERT, Llama, and three robust models.
5 Conclusion
- The paper defines text shortcuts, introduces a robustness-assessment benchmark, and reports susceptibility to occurrence, style, and concept shortcuts across BERT, Llama, and three state-of-the-art robust models.The benchmark is intended to detect shortcut learning in text classification.Source: §5 · p. 9
6 Limitations
The benchmark is limited to sentiment analysis and emotion prediction within text classification. The authors also identify tradeoffs in LLM rewriting and limitations in their dataset-quality checks and label-preservation assumptions.
6 Limitations
- The benchmark covers sentiment analysis and emotion prediction, focusing on text classification rather than other LM capabilities.Source: §6 · p. 9
- LLM rewriting is time-consuming, and output-length limits trade truncation of longer texts against irrelevant additions to shorter inputs.Source: §6 · p. 9
- Dataset-quality checks are basic, and the authors lack a reliable method to assess style matching; rewrites may shift medium-review labels under an assumption of faithful instruction following.The authors state that possible label shifts would not affect their conclusions.Source: §6 · p. 9
Ethics Statement
The statement says modified-dataset content is fictional and disclaims any representation of the authors’ views.
Ethics Statement
- Modified-dataset content is fictional, coincidental resemblance is unintended, and the content does not represent the authors’ views.Source: Ethics Statement · p. 9
A.1 Datasets
The appendix describes the datasets and the sample selection used to create the base and modified evaluation sets. It reports dataset sizes and, for Beer, the aspects and rating values retained.
A.1 Datasets
- Yelp’s base split contains 9,500 training, 500 validation, and 10,000 test samples; the style-shortcut split uses 4,500, 500, and 5,000, respectively.The style-shortcut split is sampled at 1,000 examples per label for training and test, and 100 per label for validation.Source: §A.1 · p. 11
- Go Emotions is reduced from 28 emotions to neutral, amusement, joy, and excitement, with 1,619 training, 100 validation, and 680 test samples.Source: §A.1 · p. 11
- The Beer dataset retains rationale-containing sentences for aroma, palate, and appearance, sampling ratings of 0.4, 0.6, 0.8, and 1.0.The final base set contains 2,000 samples per aspect for each of the training, test, and validation datasets.Source: §A.1 · p. 12
A.2.1 Occurrence
The appendix documents shortcut-generation materials, dataset-quality checks, and model settings and evaluation materials. It also reports a statistically significant accuracy decrease for all three robust models under shortcuts.
A.2.1 Occurrence
- Occurrence-shortcut materials list 15 honesty-related phrases and provide country and city name lists for category shortcuts.The country list is collected from a State Department source; the city list is primarily provided by ChatGPT-3.5, with manual additions or deletions.Source: §A.2.1 · p. 12
A.2.2 Style
- Style shortcuts use prompts to transfer texts into different register and author-writing styles, and GPT-4 ratings indicate high quality and faithfulness to the originals.Quality is assessed across four aspects; results are reported for the Yelp and Go Emotions training datasets.Source: §A.2.2 · p. 12, Figure 5 · p. 12, Figure 6 · p. 12
A.3.1 BERT
- BERT uses a learning rate of 2e−5, selected as best from four candidates, with batch size 16 and 15 epochs for Yelp and Go Emotions or 20 for Beer.Overall BERT results are presented in Tables 9 and 10.Source: §A.3.1 · p. 12, Table 9 · p. 16, Table 10 · p. 17
A.3.2 LLMs
- Llama2-13b results are reported in Tables 13 and 14; its QLoRA settings are searched across learning rate, rank, and alpha, and its evaluation prompts are shown in Figure 8.Inference uses temperature 0.0000, five maximum new tokens, and a repetition penalty of 1.1; the rating is extracted from the generated string.Source: §A.3.2 · pp. 12–14
A.3.5 AFR
- AFR’s Yelp and Beer figures chart Macro F1 drop as λ changes, while its appendix also points to overall performance tables.The Yelp figure labels synonym and category shortcuts; the supplied Beer figure caption identifies the same λ-dependent Macro F1 drop measure.Source: Figure 9 · p. 15, Figure 10 · p. 15, §A.3.5 · p. 15
- All robust models show a statistically significant accuracy decrease, with p < 0.05.Source: Table 8 · p. 16