PaperlayerView exact revision on arXiv

Source-linked AI summary

Adversarial Filters of Dataset Biases

Ronan Le Bras, Swabha Swayamdipta, Chandra Bhagavatula, Rowan Zellers, Matthew E. Peters, Ashish Sabharwal, Yejin Choi

arXiv:2002.04108v3cs.LGcs.AIcs.CLstat.ML

TL;DR

The paper asks whether high benchmark performance reflects task learning or exploitation of spurious dataset biases. It theoretically analyzes and empirically evaluates AFLITE, finding better out-of-distribution generalization but substantially lower filtered-benchmark performance, motivating upgraded robustness benchmarks.

  • Problem

    Large neural models often degrade on adversarial or out-of-distribution samples, raising whether benchmark success reflects dataset-specific learning driven by spurious biases.

  • Method

    The paper theoretically situates AFLITE within optimum bias reduction and empirically tests its bias filtering across synthetic and real NLP and vision datasets.

  • Results

    Models trained on AFLITE-filtered data generalize better out of distribution, while the strongest filtered-benchmark performance drops by 30 points for SNLI and 20 points for ImageNet.

  • Takeaways & Limitations

    AFLITE-filtered datasets can serve as upgraded benchmarks posing new research challenges for robust generalization.

  • Takeaways & Limitations

    Filtering can remove genuinely easy examples, lowering human performance somewhat; finding subsets that generalize well is considerably harder for vision tasks such as ImageNet-A.

Abstract

from arXiv · show

Large neural models have demonstrated human-level performance on language and vision benchmarks, while their performance degrades considerably on adversarial or out-of-distribution samples. This raises the question of whether these models have learned to solve a dataset rather than the underlying task by overfitting to spurious dataset biases. We investigate one recently proposed approach, AFLite, which adversarially filters such dataset biases, as a means to mitigate the prevalent overestimation of machine performance. We provide a theoretical understanding for AFLite, by situating it in the generalized framework for optimum bias reduction. We present extensive supporting evidence that AFLite is broadly applicable for reduction of measurable dataset biases, and that models trained on the filtered datasets yield better generalization to out-of-distribution tasks. Finally, filtering results in a large drop in model performance (e.g., from 92% to 62% for SNLI), while human performance still remains high. Our work thus shows that such filtered datasets can pose new research challenges for robust generalization by serving as upgraded benchmarks.

1. Introduction

The paper argues that benchmark performance can be inflated by spurious dataset biases and investigates AFLITE as a systematic way to reduce them. It develops theory and broad experiments showing improved out-of-distribution generalization, but substantially lower performance on filtered benchmarks.

  • Motivation: Models often perform strongly on standard benchmarks yet degrade on out-of-distribution or adversarial samples, suggesting dataset-specific rather than task-level learning.Spurious input-output correlations can bias models and inflate reported capabilities.
  • Approach: AFLITE is a bottom-up, model-based approach that systematically discovers and filters dataset artifacts beyond those humans can intuitively recognize.It was proposed as an alternative to domain-specific, intuition-guided bias-reduction methods.
  • Contributions: The paper provides a theoretical framework for optimum bias reduction and shows that AFLITE practically approximates the ideal but computationally intractable AFOPT method.The framework is paired with extensive experiments on synthetic data and benchmarks spanning NLP and vision.
  • Results: Models trained on AFLITE-filtered data generalize substantially better to out-of-domain samples than models trained on the original biased datasets.The findings indicate that spurious biases make benchmarks artificially easier and encourage reliance on less transferable features.
  • Results: 63% accuracy is achieved by the best model on SNLI-AFLITE, a 30% drop from its accuracy on original SNLI, while human performance remains high.AFLITE preserves an identical train-test distribution and retains a sizable training set.
  • Implications: AFLITE-filtered datasets can serve as upgraded benchmarks that pose new research challenges for robust generalization.The paper presents this as the practical implication of its filtering and evaluation results.

2. AFLITE

AFLITE formalizes bias reduction as selecting a sufficiently large subset whose representations are difficult for a model family to exploit, then approximates this objective by iteratively removing highly predictable instances.

  • Formalization: AFLITE seeks a subset of at least n examples that minimizes representation bias for a feature representation Φ and model family M.Representation bias is the best achievable out-of-sample classification accuracy using Φ(X).
  • Formalization: Representation bias is defined as expected classification accuracy when models train on S \ T and are evaluated on randomly selected held-out subsets T.The expectation averages over the subset-selection distribution q.
  • Predictability scores: Predictability score p(i) measures how reliably a model predicts instance i across training subsets that exclude it, identifying exploitable representations.High-scoring instances are undesirable because their features support confident correct prediction.
  • Predictability scores: Uniform fixed-size subset sampling reduces the representation-bias expectation to a linear sum over instance-level predictability scores.This refactoring replaces an exponential summation over subsets with |S| terms.
  • Algorithm: Because optimum bias reduction remains intractable, AFLITE greedily removes slices of the most predictable instances and stops early when all scores fall below τ.The procedure begins with S = D and repeatedly recomputes predictability over the current candidate set.
  • Algorithm: AFLITE takes a dataset, precomputed representation, model family, target size, partition count, training size, slice size, and stopping threshold as inputs.Its implementation typically trains linear classifiers on random partitions and collects out-of-sample predictions.

3. Synthetic Data Experiments

Synthetic experiments test whether AFLITE removes artificially constructed spurious correlations and make the resulting data harder for linear models while retaining nonlinear task structure.

  • Out-of-distribution evaluation: Table 1 compares zero-shot SNLI performance across HANS, NLI-Diagnostics, and Stress for original, AFLITE-filtered, and equal-sized random-subset training data.Results are averaged across 5 random seeds, with standard deviations reported.
  • Adversarial evaluation: Table 2 evaluates SNLI accuracy on Adversarial NLI after separate in-distribution finetuning across rounds Rd1, Rd2, and Rd3.It compares RoBERTa-large models pretrained on original versus AFLITE-filtered SNLI data.
  • Synthetic setup: Synthetic datasets contain concentric-circle classes at four separation levels, with class-specific Gaussian bias features added to 75% of each class.The bias features make the task solvable by a linear function, although the original geometry requires a nonlinear RBF model.
  • Filtering effects: AFLITE considerably reduces biased samples, making the filtered task hard for linear models while remaining solvable for nonlinear models.The filtered datasets are shown in the lower half of Figure 2.
  • Filtering effects: AFLITE removes examples with spurious correlations across all separation levels and removes most flipped examples from the highest-separation dataset.The experiments use linear and RBF-kernel SVM performance as diagnostics.

4. NLP Experiments

The NLP experiments test whether AFLITE-filtered datasets improve out-of-distribution generalization and provide stricter estimates of benchmark performance. Across SNLI, MultiNLI, and QNLI, filtering improves diagnostic transfer while substantially reducing in-domain accuracy, beyond effects of dataset size alone.

  • Out-of-distribution Generalization: AFLITE-filtered SNLI models consistently match or exceed full-data models on HANS, NLI-Diagnostics, and Stress, and surpass them across all three Adversarial NLI settings.The filtered samples outperform a same-sized random subsample, suggesting they are more informative for generalization.
  • In-distribution Benchmark Re-estimation: AFLITE lowers SNLI model accuracy by typically 15–35% across feature representations, with the largest drops when RoBERTa learns the filtering representation.A weaker ESIM+GloVe adversary still causes a 13.7% drop for RoBERTa evaluation.
  • In-distribution Benchmark Re-estimation: Randomly reducing SNLI to approximately 92k examples preserves nearly full-dataset performance, whereas AFLITE substantially reduces it, indicating redundancy does not explain the filtering effect.The comparison controls for the reduction in training-set size.
  • In-distribution Benchmark Re-estimation: AFLITE reduces performance more than PMI and hypothesis-only baselines on SNLI, dropping accuracy by about 30% versus nearly 13% and almost 24%, respectively.Because AFLITE is agnostic to known bias types, the result indicates coverage of a broader class of spurious biases.
  • In-distribution Benchmark Re-estimation: A single-model, single-iteration filter yields 72.1% RoBERTa dev accuracy, compared with 62.6% for iterative AFLITE, supporting the need for iterative multi-partition filtering.Both subsets have the same size.
  • In-distribution Benchmark Re-estimation: RoBERTa accuracy falls from over 90% on original MultiNLI and QNLI to 66.2% and 77.7% after AFLITE, even after controlling for training-set size.Partial-input baselines also decrease substantially on both filtered datasets.

5. Vision Experiments

AFLITE-filtered ImageNet data improves out-of-distribution performance while making in-distribution evaluation substantially harder. The strongest model’s ImageNet accuracy falls sharply after filtering, indicating that standard ImageNet performance relies partly on dataset biases.

  • Adversarial Image Classification: All EfficientNet models struggle on ImageNet-A, including models trained on the entire ImageNet dataset.ImageNet-A is an out-of-distribution evaluation set for image classification.
  • Adversarial Image Classification: 2% improvement in EfficientNet-B7 top-1 accuracy on ImageNet-A follows training on AFLITE-filtered data.The comparison is against training on randomly sampled ImageNet data of the same size.
  • In-distribution Image Classification: 84.4% to 63.5%: EfficientNet-B7 top-1 accuracy drops after AFLITE filtering on ImageNet validation.A randomly filtered subsample of the same size causes a much smaller reduction, likely because it reduces training data.
  • In-distribution Image Classification: Image classification remains far from solved even within a subset of ImageNet’s closed world.The authors connect this difficulty to biases toward canonical poses and texture rather than shape.

6. Related Work

Related work addresses dataset and representation bias through adversarial filtering, dataset resampling, and debiasing objectives. AFLITE differs by filtering existing data with a lightweight, broadly applicable approach rather than generating instances or retraining at every iteration.

  • Adversarial Filtering: AFLITE is more broadly applicable and lightweight because it does not require overgeneration or model retraining at each filtering iteration.Retraining at every iteration is especially expensive with large adversary models such as BERT.
  • Adversarial Filtering: AFLITE addresses biases in existing datasets, unlike adversarial filtering methods that perturb or overgenerate data instances.Prior adversarial filtering variants were used to create datasets such as HellaSwag and Abductive NLI.
  • Data Selection for Debiased Representations: REPAIR favors instances that are hard for a representation, whereas AFLITE uses out-of-sample generalization accuracy rather than in-training classification loss.The comparison distinguishes their data-selection criteria and implementation.
  • Data Selection for Debiased Representations: AFLITE’s filtering approach aligns with work advocating subsampling majority groups to reduce worst-group error rather than upweighting minority groups.The cited connection also matches the paper’s robustness and out-of-distribution findings.
  • Learning Objectives for Debiasing: Invariant Risk Minimization promotes representations stable across environments, while AFLITE removes instances exhibiting dataset artifacts.The methods target different stages of debiasing: representation learning versus dataset filtering.

7. Conclusion

The paper develops and evaluates AFLITE as an iterative method for removing spurious dataset biases. Across four datasets, filtered subsets improve out-of-distribution performance while sharply reducing strongest-model benchmark scores.

  • Conclusion: AFLITE is an iterative greedy algorithm that adversarially filters spurious biases for more accurate benchmark estimation.The paper provides a theoretical framework and evaluates the approach on synthetic and real datasets.
  • Conclusion: Models trained on AFLITE-filtered subsets perform better on out-of-distribution and adversarial test sets across the evaluated benchmarks.The conclusion interprets this pattern as evidence of higher generalization abilities.
  • Conclusion: 30 points for SNLI and 20 points for ImageNet: strongest-model performance drops significantly on the resulting filtered datasets.The filtered datasets are presented as upgraded benchmarks for robust generalization research.
  • Conclusion: The authors propose that dataset creators use AFLITE to identify unknown artifacts before releasing challenge datasets.The stated goal is more reliable estimates of task progress on future AI benchmarks.

A.1. Filtering Heuristics

The paper describes three heuristic approximations to optimum bias reduction: single-instance greedy removal, batch greedy slicing, and probabilistic slice sampling. AFLITE implements the greedy slicing strategy because it is scalable and practical.

  • Filtering Heuristics: The simple greedy heuristic repeatedly removes the instance with the highest predictability score until the target dataset size is reached.It starts from the full dataset S = D and removes instances one at a time.
  • Filtering Heuristics: Greedy slicing removes the k instances with the highest predictability scores at each iteration.The process repeats in batches until the target size is reached.
  • Filtering Heuristics: Slice sampling randomly selects k instances with probabilities proportional to their predictability scores.Unlike greedy slicing, it does not deterministically choose the top-k instances.
  • Filtering Heuristics: AFLITE implements greedy slicing, providing a scalable and practical approximation without look-ahead retraining.The authors found the lighter heuristics effective despite omitting the extra computation.

A.2. Slice Sampling Details

AFLITE’s slice sampling can be implemented through Gumbel perturbations, while synthetic-data results compare filtered and unfiltered performance across model types.

  • A.2. Slice Sampling Details: Gumbel perturbations convert sampling from a probability distribution into an optimization problem by selecting the maximizer of perturbed log-predictability scores.Selecting the k highest perturbed scores samples k items without replacement.
  • A.2. Slice Sampling Details: The Gumbel method samples items by adding independent standard-Gumbel noise to log-predictability scores and choosing the largest resulting values.The selected maximizer is an exact sample from the distribution defined by the scores.
  • A.2. Slice Sampling Details: Table 7 reports mean development accuracy before and after AFLITE for an RBF-kernel SVM and a logistic-regression classifier across four synthetic datasets.The datasets differ in the degree of separation between their two classes, with standard deviations reported across 10 random-seed runs.

A.3. Results on Synthetic Data Experiments

Synthetic experiments examine AFLITE on concentric-circle datasets with varying class separation and compare an RBF-kernel SVM with logistic regression. The SVM is more robust to artifacts, motivating task-specific reasoning models that avoid spurious-artifact dependence.

  • A.3. Results on Synthetic Data Experiments: Figure 2 evaluates AFLITE on four synthetic concentric-circle datasets spanning four degrees of class separation.Table 7 provides accuracies for the RBF-kernel SVM and logistic regression.
  • A.3. Results on Synthetic Data Experiments: The SVM is more robust to artifacts than the linear classifier, supporting models designed to reason about the specific task rather than exploit spurious artifacts.The passage presents this as the implication for real datasets.

A.4. NLI Out-of-distribution Benchmarks

The NLI out-of-distribution evaluation uses several targeted benchmarks, while AFLITE experiments document sampling, training, and filtering choices. Retained SNLI examples are described as more challenging and semantically nuanced than removed examples.

  • A.4. NLI Out-of-distribution Benchmarks: HANS tests structural heuristics, NLI Diagnostics probes fine-grained semantic categories, and Stress Tests target competence, distraction, and noise.These benchmarks assess robustness beyond standard NLI evaluation.
  • A.4. NLI Out-of-distribution Benchmarks: AFLITE’s filtering hyperparameters include support size m, classifier training size t, slice size k, and early-stopping threshold τ.For ImageNet, n = 640K is fixed and τ is not controlled; other settings use τ instead of controlling n.
  • A.4. NLI Out-of-distribution Benchmarks: Adversarial NLI uses human-generated hypotheses and premises from Wikipedia and news corpora across challenge tiers created with humans and models in the loop.It is one of the four described NLI out-of-distribution benchmarks.
  • A.4. NLI Out-of-distribution Benchmarks: Out-of-distribution results report means and variances across 5 random seeds, except for Adversarial NLI because it requires model finetuning.The seed-based variance reporting addresses observed variation on out-of-distribution test sets.
  • A.4. NLI Out-of-distribution Benchmarks: For SNLI out-of-distribution experiments, AFLITE uses n = 182K, while ImageNet filtering uses n = 640K.The implementation explicitly controls dataset size in these out-of-distribution experiments.
  • A.4. NLI Out-of-distribution Benchmarks: The NLP experiments use Adam with learning rate 1e-5, epsilon 1e-8, 3 epochs, and batch size 92 across NLI tasks.These hyperparameters were selected through grid search, with other settings retained from the original setup.
  • A.4. NLI Out-of-distribution Benchmarks: AFLITE-retained SNLI examples are more challenging and semantically nuanced, whereas removed examples show greater word overlap and other artifacts.The filtered dataset also contains more neutral examples than contradiction examples in the illustrated set.
Loading 2002.04108v3…