Source-linked AI summary
Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases
Christopher Clark, Mark Yatskar, Luke Zettlemoyer
TL;DR
Models often rely on superficial dataset patterns that fail in out-of-domain or adversarial settings. The paper trains a bias-only model and ensembles it with a robust model, reporting improved out-of-domain performance across all evaluated settings, including a 12-point VQA-CP gain.
Problem
Superficial dataset patterns can produce strong in-domain performance while making models brittle under out-of-domain or adversarial evaluation.
Method
The method trains a bias-only model first, then trains a robust model in an ensemble with it to encourage an alternative strategy, using the robust model alone at test time.
Results
The method improves out-of-domain performance in all settings, including a 12-point gain on VQA-CP and 6- and 9-point gains on two QA datasets.
Takeaways & Limitations
Known dataset biases can be explicitly modeled and factored out through ensemble-based training to improve robustness to domain shift.
Takeaways & Limitations
The method can reduce the effective training data by reducing the need to solve examples handled well by the bias-only model.
Abstract
from arXiv · showhide
State-of-the-art models often make use of superficial patterns in the data that do not generalize well to out-of-domain or adversarial settings. For example, textual entailment models often learn that particular key words imply entailment, irrespective of context, and visual question answering models learn to predict prototypical answers, without considering evidence in the image. In this paper, we show that if we have prior knowledge of such biases, we can train a model to be more robust to domain shift. Our method has two stages: we (1) train a naive model that makes predictions exclusively based on dataset biases, and (2) train a robust model as part of an ensemble with the naive one in order to encourage it to focus on other patterns in the data that are more likely to generalize. Experiments on five datasets with out-of-domain test sets show significantly improved robustness in all settings, including a 12 point gain on a changing priors visual question answering dataset and a 9 point gain on an adversarial question answering test set.
1 Introduction
Models can exploit dataset biases that work in-domain but fail under domain shift. The paper proposes identifying such biases with naive models and using ensemble training to improve out-of-domain robustness.
- Motivation: Dataset-specific shortcuts, such as keyword-based entailment predictions and context-insensitive answer selection, often fail out of domain or under adversarial evaluation.These superficial patterns are called bias and can make models brittle and easy to fool.
- Method: The method first trains a bias-only model, then trains a second model in an ensemble with it so the second model learns an alternative strategy.The robust model is used alone at test time, and several product-of-experts-style ensembling methods are explored.
- Evaluation: The evaluation spans synthetic MNLI datasets and challenge datasets covering textual entailment, reading comprehension, VQA, and changing-prior question answering.The challenge set includes TriviaQA-CP, which tests generalization to held-out entity types.
- Results: 12 points is the gain on the VQA challenge set, compared with a 3 point gain from prior work.The paper also reports 6- and 9-point gains on two QA datasets.
2 Related Work
Prior work documents dataset biases across language and vision tasks and develops ways to reduce reliance on them. This paper positions its approach as a cross-dataset evaluation of methods for preventing models from using known biases.
- Known biases: Bias can let models solve multimodal or multi-hop tasks while ignoring part of the input or using a simpler strategy than the task requires.Examples include ignoring visual information and solving multi-hop questions with single-hop models.
- Dataset design: Dataset construction methods reduce bias by limiting annotation shortcuts, question-only prediction, or examples solvable by simple baselines.Examples include CoQA, QuAC, VQA 2.0, and filtering protocols for other datasets.
- Model-based approaches: Existing model-based solutions include generative objectives, architecture changes, and adversarial removal of class-indicative features from internal representations.The paper contrasts these approaches with its consideration of biases beyond partial-input cases.
- Fairness-related work: Fairness research often removes information about target features such as gender or ethnicity, whereas the biases considered here concern features essential to the overall task.This distinguishes the paper’s setting from a common adversarial-representation-learning formulation.
- Evaluation setting: The paper studies larger input-distribution changes designed to expose higher-level model flaws, rather than only minor perturbations of existing examples.This separates its evaluation setting from much adversarial-robustness work focused on small perturbations.
3 Methods
The method first models a known dataset bias with a bias-only predictor, then trains a robust model in an ensemble with it to encourage an alternative strategy. Several ensemble variants address conditional-independence and bias-trust limitations, including learned weighting and entropy regularization.
- Method overview: The two-stage procedure builds a bias-only model, then uses it in ensemble training so a second model learns a strategy that avoids the captured bias.The robust model is used alone at test time.
- Training a bias-only model: Bias-only models use features correlated with training labels but expected to be uncorrelated or anticorrelated under the out-of-domain shift.Examples include question type for VQA-CP, TF-IDF views for SQuAD, and a pretrained NER tagger for changing-prior TriviaQA.
- Ensemble training: The ensemble combines the main model’s distribution with the bias-only distribution during training, backpropagating through the main model while evaluating it alone.The framework assumes labeled examples, a pretrained bias predictor, and a second predictor whose objective is to avoid the bias strategy.
- Bias Product: Bias Product multiplies the two predictive distributions elementwise, corresponding to a product-of-experts ensemble under a conditional-independence justification.The class-prior factor is omitted for dataset-specific reasons, including uniform MNLI classes and desired prior removal in VQA.
- Limitations and regularization: The conditional-independence assumption can be too strong, because selective compensation for bias-model inaccuracies may cause out-of-domain errors.An entropy penalty encourages the bias component to remain non-uniform and therefore influence the ensemble more strongly.
- Learned-Mixin: Learned-Mixin lets the model determine how much to trust the bias for each input, using a nonnegative learned function that reduces to Bias Product when g(x_i)=1.Without additional constraints, the model may absorb the bias into its own prediction and set g(x_i)=0.
4 Evaluation Methodology
The evaluation tests bias-aware training on multiple out-of-domain settings using a train set, shifted test set, bias-only model, and main model. It compares the approach with unmodified training, reweighting, and selected adversarial VQA baselines while noting a hyperparameter-selection caveat.
- Evaluation setup: The evaluation covers tasks with challenging train–test domain shifts, including HANS, Adversarial SQuAD, and other challenge datasets.Table 1 summarizes each evaluation’s domain shift and bias-only model construction.
- Evaluation setup: Each evaluation trains a bias-only model on the training set, trains the main model with an ensemble method, and evaluates the main model on the out-of-domain test set.In-domain performance is also reported when available.
- Baselines: The comparisons include unmodified main-model training and a reweighting baseline that emphasizes examples the bias-only model classifies incorrectly.On VQA, the study also compares against two adversarial methods; those methods do not directly apply to other considered biases.
- Hyperparameters: One Learned-Mixin +H method requires hyperparameter tuning, but the datasets lack development sets with related domain shifts.The authors therefore select models on the test set, identifying this as an important caveat.
5 Experiments
Experiments across synthetic and real domain-shift benchmarks show that bias-aware ensembles improve out-of-domain robustness, while performance depends on the ensembling method and regularization. The learned-mixin approach dynamically controls reliance on the bias-only model, often outperforming alternatives but sometimes trading away in-domain accuracy.
- Synthetic data: Synthetic MNLI results show all ensembling methods work well for Indicator bias, but reweighting performs poorly for Excluder bias.The authors identify the lack of bias-free examples for learning as a general weakness of reweighting methods.
- Synthetic data: On Dependent bias, bias product assigns 22.5% probability to the bias-indicated class, below the 33% expected from an unbiased model.The model becomes anticorrelated with the bias because it learns to subtract the bias-only model’s contribution.
- Synthetic data: The learned-mixin model sets g(x_i) to 0.00 ± 0.0001 when the indicator is off and 1.91 ± 0.285 when it is on.This behavior lets the ensemble turn off the bias-only component as needed; the entropy regularizer is unnecessary on this dataset.
- VQA-CP: 12 points: learned-mixin improves VQA-CP performance by about 9 points, with entropy regularization adding another 3 points.The learned-mixin gate correlates with the bias model’s expected accuracy, with Spearman correlation 0.77 on test data.
- Adversarial SQuAD: 9 points: learned-mixin +H improves Adversarial SQuAD, compared with up to 3 points for bias product.Without the entropy penalty, the model drives g(x_i) near zero and effectively ignores the bias-only model.
- Overall results: Across five domains, the method improves out-of-domain performance in all settings, with learned-mixin generally outperforming bias product when paired with entropy regularization.The reweight baseline improves HANS but is relatively ineffective elsewhere.
- Discussion: Improved out-of-domain performance usually comes at the cost of lower in-domain performance, although TriviaQA-CP shows minimal trade-off.The authors also note that relying on the ensemble may reduce the effective amount of training data for examples already solved by the bias-only model.
6 Conclusion
The paper uses human knowledge of non-generalizing methods to improve robustness to domain shift by training a robust model alongside a pre-trained naive model. Experiments show gains on adversarial and changing-prior datasets, including 12 points on VQA-CP.
- The method uses human knowledge about methods unlikely to generalize to improve model robustness to domain shift.
- The robust model is trained in an ensemble with a pre-trained naive model and then used alone at test time.
- Experiments show the method works well on two adversarial datasets and two changing-prior datasets.
- 12 points of gain are reported on VQA-CP.
- Future work includes automatically detecting dataset bias to reduce reliance on specific prior knowledge.
A Entropy Penalty Weights
This section reports entropy penalty weights for the learned-mixin +H ensemble across the experiments.
- Table 7 reports the entropy penalty weight for the learned-mixin +H ensemble across all experiments.
B Co-Attention NLI Model
The NLI model uses an ESIM-based architecture that embeds premise and hypothesis words, computes co-attention, pools the representations, and predicts among three classes.
- The NLI model is based on ESIM.
- Embed: Words are embedded with a character CNN and fastText crawl embeddings, followed by a shared BiLSTM.
- Co-Attention: Co-attention produces context vectors and augmented premise and hypothesis representations using embeddings, context, and elementwise products.
- Pool: A second shared BiLSTM and max pooling feed a fully connected layer and a softmax with three output classes.
- Training uses variational dropout at rate 0.2, Adam for 30 epochs, batch size 32, and a learning-rate decay of 0.999 every 100 steps.
C Fine-Grained HANS Results
Fine-grained HANS results show that the methods reduce reliance on heuristic entailment guesses, while learned-mixin preserves entailment-example performance better than the other methods for BERT.
- The methods reduce naive entailment guessing across all HANS heuristics.
- For BERT, reweight and bias product significantly degrade performance on entailment examples, whereas learned-mixin largely maintains it.
D Modified BiDAF QA Model
The modified BiDAF QA model embeds question and passage text, applies bi-directional attention, and predicts answer span boundaries through layered recurrent and linear components.
- The model embeds words with a character CNN and fastText vectors, then applies a BiLSTM for context-aware question and passage representations.
- Bi-directional attention produces question-aware passage embeddings for downstream answer prediction.
- A fully connected layer, two BiLSTM layers, and a two-dimensional linear layer produce start and end scores for each token.
- Variational dropout is applied between all layers at a rate of 0.2, with Adam optimization and learning-rate decay by 0.999 every 100 steps.
- SQuAD uses a 200-dimensional fully connected layer and 100-dimensional LSTMs, while TriviaQA uses 256-dimensional and 128-dimensional counterparts with additional connections and dropout.
E TriviaQA-CP
TriviaQA-CP evaluates question answering under changing topic distributions by constructing person- and location-focused splits and selecting answer-containing passages. Its question-type labels combine distant supervision, manual annotation, and classifier recalibration.
- Dataset construction: TriviaQA-CP contains person and location changing-priors versions created by removing one topic from training and evaluating on that topic’s development questions.
- Distantly Supervised Classification: The dataset construction begins with a preliminary question-type classifier trained using noisy pattern-based and Yago-derived labels.
- Distantly Supervised Classification: A recurrent classifier embeds questions with fastText vectors, applies a 100-dimensional BiLSTM and max-pooling, and predicts three classes with a softmax layer.
- Supervised Classification: A second linear classifier recalibrates recurrent predictions and integrates them with manually labeled questions and Yago-based answer-category features.
- Supervised Classification: The final question classifier achieves about 95% accuracy; approximately 25% of questions concern people and 20% concern locations.
- Paragraph Selection: Each TriviaQA question is paired with one highest-ranked answer-containing paragraph after documents are split into passages of at most 400 tokens.