Source-linked AI summary
Learning the Difference that Makes a Difference with Counterfactually-Augmented Data
Divyansh Kaushik, Eduard Hovy, Zachary C. Lipton
TL;DR
The paper addresses the lack of a coherent statistical distinction between spurious and non-spurious associations by using human counterfactual document revisions. It finds that models trained on original data fail on revised data and vice versa, while combined datasets improve performance across sentiment analysis and NLI.
Problem
Standard statistical learning lacks a conceptual distinction between spurious and non-spurious associations, limiting its account of model reliance on spurious patterns.
Method
Humans revise documents to fit counterfactual target labels while preserving coherence and avoiding unnecessary changes, producing augmented sentiment-analysis and NLI datasets.
Results
Models trained on original data fail on counterfactually revised data and vice versa, while combined datasets perform well across sentiment analysis and NLI.
Takeaways & Limitations
Human counterfactual edits can expose spurious associations and support classifiers less dependent on them when associations do not transport across domains.
Abstract
from arXiv · showhide
Despite alarm over the reliance of machine learning systems on so-called spurious patterns, the term lacks coherent meaning in standard statistical frameworks. However, the language of causality offers clarity: spurious associations are due to confounding (e.g., a common cause), but not direct or indirect causal effects. In this paper, we focus on natural language processing, introducing methods and resources for training models less sensitive to spurious patterns. Given documents and their initial labels, we task humans with revising each document so that it (i) accords with a counterfactual target label; (ii) retains internal coherence; and (iii) avoids unnecessary changes. Interestingly, on sentiment analysis and natural language inference tasks, classifiers trained on original data fail on their counterfactually-revised counterparts and vice versa. Classifiers trained on combined datasets perform remarkably well, just shy of those specialized to either domain. While classifiers trained on either original or manipulated data alone are sensitive to spurious features (e.g., mentions of genre), models trained on the combined data are less sensitive to this signal. Both datasets are publicly available.
1 INTRODUCTION
The paper argues that standard associative learning cannot distinguish spurious from non-spurious associations, and draws on causality to motivate human counterfactual revisions. These revisions expose classifier brittleness across original and altered data while combined training improves robustness.
- Standard statistical learning defines relevance through predictive association but lacks a conceptual distinction between spurious and non-spurious associations.
- Causality characterizes spurious associations as confounded relationships without a direct or indirect causal path.
- The authors use human editors to revise documents toward counterfactual labels while preserving coherence and avoiding gratuitous changes.
- Classifiers trained on original IMDb reviews fail on revised reviews and vice versa, whereas combined training raises accuracy to 81.7% on original and 92.0% on revised data.
- On NLI, BERT trained on original SNLI pairs suffers more than a 30 point accuracy drop on revised pairs, while combined data improves performance across datasets.
2 RELATED WORK
Related work includes robustness tests using synthetic text transformations, rationale-based feedback, and automated counterfactual edits for reducing gender bias. The paper distinguishes its human document-editing approach from these alternatives.
- Prior studies show NLP systems can fail under distractor phrases, paraphrases, template modifications, or label-preserving synonym and hypernym substitutions.
- Rationale-based methods ask annotators to highlight label-indicative spans and sometimes remove them to create contrast documents.
- Automated gender-bias methods modify gendered words, names, or grammatical forms while attempting to preserve sentence structure.
3 DATA COLLECTION
The authors collect counterfactually revised sentiment and NLI examples through crowdsourced editing, verification, and quality control. Workers alter the text toward target labels while retaining the relevant unchanged context.
- Editors were recruited through Mechanical Turk with requirements of at least 500 completed HITs and over 97% approval, while pilots refined compensation and instructions.
- Sentiment Analysis: Sentiment revisions used sampled, class-balanced IMDb reviews, with two workers editing each review toward a counterfactual label.
- Sentiment Analysis: Roughly 2% of revised reviews were rejected after manual inspection, and one approved revision was randomly selected for each original review.
- Natural Language Inference: For NLI, workers revised either the hypothesis or premise while leaving the other sentence intact, targeting entailment, contradiction, or neutral relations.
- Natural Language Inference: Three workers verified each revised premise-hypothesis label, using majority decisions and additional manual verification when reviewers disagreed.
4 MODELS
The experiments use five model families, with standard classifiers trained on TF-IDF bag-of-words representations and neural models including Bi-LSTMs, ELMo-LSTM, and BERT.
- Models: The study evaluates SVMs, Naïve Bayes, Bi-LSTMs, ELMo models with LSTM, and fine-tuned BERT models.These five model families are used across the experiments.
- Models: SVMs and Naïve Bayes classifiers for sentiment analysis use TF-IDF bag-of-words feature representations.Their parameters are selected by grid search on the validation set.
- Models: The Bi-LSTM sentiment model uses a 20k-token vocabulary, 50-dimensional embeddings, recurrent dropout, and global max-pooling.Inputs are limited to 300 tokens and shorter reviews are padded.
- Models: ELMo-LSTM represents each word with a 128-dimensional embedding concatenated with a 1024-dimensional ELMo representation.This produces a 1152-dimensional hidden representation before the LSTM.
- Models: BERT uses an off-the-shelf uncased BERT Base model fine-tuned separately for sentiment analysis and NLI.Maximum lengths are 350 tokens for sentiment analysis and 50 for NLI.
5 EXPERIMENTAL RESULTS
Across sentiment analysis and NLI, models trained on original data often fail on revised data and vice versa, while combined training improves cross-distribution performance and out-of-domain generalization.
- Sentiment Analysis: 80% accuracy on original reviews fell to 51% on revised reviews for linear models trained on the original 1.7k reviews.Models trained on revised reviews reached 91% on revised reviews but only 58.3% on the original test set; combined training performed well on both.
- Sentiment Analysis: Counterfactually-augmented IMDb training generally improved performance on Amazon, Twitter, and Yelp sentiment datasets compared with comparable quantities of original data.These evaluations test generalization to new domains.
- Sentiment Analysis: Linear models trained on original or revised reviews learned seemingly irrelevant genre-associated words as high-weight sentiment predictors.Combining original and revised datasets made these terms cease to be predictive of sentiment.
- Natural Language Inference: 72.2% accuracy on SNLI fell to 39.7% on revised-premise pairs for BERT fine-tuned on 1.67k original sentence pairs.Fine-tuning on revised-premise pairs raised revised-set accuracy to 66.3% but caused an approximately 20-point drop on SNLI.
- Natural Language Inference: Combining revised-premise and revised-hypothesis data produced consistent BERT performance across datasets because each hypothesis appears with different premises and labels.Adding original sentences improved these results further, while the combined data outperformed a similarly sized SNLI sample on both revised sets.
- Natural Language Inference: A Bi-LSTM trained on SNLI hypotheses alone achieved 69% on SNLI but dropped to 44% when trained on original, revised-premise, and revised-hypothesis data.The combined dataset gives identical hypotheses conflicting feedback because they occur with different premises and truth values.
- Dataset Distinguishability: BERT identified original versus revised sentiment reviews with 77.3% accuracy, showing that the two data sources remain distinguishable.The experiment tests whether models become hypersensitive to dataset domain.
6 CONCLUSION
The paper uses human intervention on text, rather than labels alone, to expose label-relevant differences and train models less dependent on spurious associations across sentiment analysis and NLI.
- Conclusion: Human revisions that make documents accord with target labels can elucidate the difference that makes a difference.Workers intervene on data by revising documents rather than only providing labels.
- Conclusion: Counterfactually augmented data can train classifiers that hold up better when spurious associations do not transport out of domain.The paper reports this promise for both sentiment analysis and NLI.
- Conclusion: The method is applied to contrasting tasks: sentiment analysis emphasizes expressions of opinion, whereas NLI emphasizes stated facts.SNLI additionally involves three classes and two input sentences.
- Conclusion: Future work proposes extending the techniques to question answering and summarization.
APPENDIX
The appendix includes annotation analyses, feature visualizations, and Mechanical Turk instructions describing how workers revise passages while preserving coherence and matching counterfactual labels.
- Annotation Analyses: The appendix analyzes frequent human insertions and deletions for sentiment analysis and SNLI.Separate tables summarize the most frequent edits in each task.
- Data Collection Instructions: The appendix includes separate panels for models trained on the original and revised datasets.
- Feature Visualizations: Feature visualizations show the most important TF-IDF bag-of-words features learned by SVM classifiers.The appendix includes separate visualizations for the listed feature sets.
- Data Collection Instructions: Mechanical Turk instructions ask workers to make a small number of changes so the document remains coherent and the new label accurately describes it.Workers are also told not to change portions unnecessarily.