PaperlayerView exact revision on arXiv

Source-linked AI summary

On Feature Learning in the Presence of Spurious Correlations

Pavel Izmailov, Polina Kirichenko, Nate Gruver, Andrew Gordon Wilson

arXiv:2210.11369v1cs.LGcs.CVstat.ML

TL;DR

Spurious correlations can make classifiers perform well on average while failing on subgroups, raising the question of whether robust training learns better core-feature representations. The paper evaluates representations by retraining their last layer with DFR and finds that ERM features are broadly competitive, while architecture and pretraining matter more than strong regularization. Simple ERM plus DFR also improves several benchmark results.

  • Problem

    The paper asks how much core-feature information is encoded by ERM and group-robustness representations when classifiers can exploit spurious correlations.

  • Method

    The paper uses DFR to discard the original classification head and retrain a linear classifier on held-out data where spurious correlations are broken.

  • Results

    ERM features are highly competitive with group robustness features after DFR, while model architecture and pretraining strongly affect representation quality and strong regularization usually does not.

  • Takeaways & Limitations

    For many problems, retraining the last layer can provide the practical benefits of robustness without significant representation gains from group-robust feature-extractor training.

  • Takeaways & Limitations

    The paper leaves open whether methods can significantly improve learned representations beyond standard ERM for a fixed base model.

Abstract

from arXiv · show

Deep classifiers are known to rely on spurious features $\unicode{x2013}$ patterns which are correlated with the target on the training data but not inherently relevant to the learning problem, such as the image backgrounds when classifying the foregrounds. In this paper we evaluate the amount of information about the core (non-spurious) features that can be decoded from the representations learned by standard empirical risk minimization (ERM) and specialized group robustness training. Following recent work on Deep Feature Reweighting (DFR), we evaluate the feature representations by re-training the last layer of the model on a held-out set where the spurious correlation is broken. On multiple vision and NLP problems, we show that the features learned by simple ERM are highly competitive with the features learned by specialized group robustness methods targeted at reducing the effect of spurious correlations. Moreover, we show that the quality of learned feature representations is greatly affected by the design decisions beyond the training method, such as the model architecture and pre-training strategy. On the other hand, we find that strong regularization is not necessary for learning high quality feature representations. Finally, using insights from our analysis, we significantly improve upon the best results reported in the literature on the popular Waterbirds, CelebA hair color prediction and WILDS-FMOW problems, achieving 97%, 92% and 50% worst-group accuracies, respectively.

1 Introduction

The paper studies whether ERM learns useful core-feature representations despite spurious correlations, evaluating them through last-layer retraining. It finds ERM features competitive with group robustness methods and improves benchmark worst-group results through architecture, pretraining, and DFR choices.

  • Motivation: Spurious features predict labels without causal relevance, enabling strong average performance but often poor accuracy when correlations differ across subgroups.Examples include image backgrounds, texture, words, and syntactic heuristics.
  • Approach: The study separates feature extraction from linear classification and uses DFR to evaluate how well core features are encoded in learned representations.DFR retrains the last layer on held-out data where the spurious correlation is broken.
  • Findings: ERM learns core-feature representations that are highly competitive with those learned by specialized group robustness methods, including group DRO.After DFR, ERM and group DRO achieve similar performance across spurious-correlation problems.
  • Findings: Feature quality depends strongly on model architecture and pretraining strategy, whereas strong regularization is not necessary for high-quality representations.The paper reports that pretraining has a very significant effect, while strong regularization usually does not significantly improve representations.
  • Findings: Simple ERM followed by DFR significantly outperforms prior results on Waterbirds, CelebA hair color, and WILDS-FMOW benchmarks.The paper attributes these improvements to finetuning a pretrained state-of-the-art ConvNext model.

2 Related Work

Prior work documents spurious correlations across vision, NLP, and high-risk applications, motivating group robustness methods and feature-learning analyses. This paper extends representation-focused evaluation from synthetic settings to large-scale natural datasets and complements concurrent last-layer-retraining work.

  • Spurious correlations: Neural networks can exploit backgrounds, secondary objects, textures, and other irrelevant features, with especially consequential examples reported in medical imaging.NLP models likewise use specific words and syntactic heuristics as spurious signals.
  • Robustness methods: Group robustness methods target worst-group performance through objectives such as group DRO, minority-group identification, diverse classifiers, partial group labels, and group subsampling.Group DRO minimizes worst-group loss rather than average loss.
  • Feature learning: Feature-learning studies have examined synthetic datasets, texture bias, data augmentation, and feature extractability after NLP pretraining and fine-tuning.Counterfactually augmented datasets can reduce reliance on spurious correlations on corresponding benchmarks.
  • Positioning: Building on Kirichenko et al., this work studies core-feature decodability on large-scale natural datasets using DFR.Related observations also appear in domain generalization and long-tail classification.
  • Positioning: A concurrent study also evaluates out-of-distribution generalization through last-layer retraining while comparing supervised, self-supervised, and unsupervised training.The paper characterizes those observations as complementary.

3 Background

The paper models group imbalance and distribution shifts as sources of gaps between average and worst-group performance. It evaluates feature representations by discarding the original head and retraining a classifier on group-balanced held-out data.

  • Problem setup: Groups are unequally represented in training data, and their proportions may change between training and test distributions.This imbalance can produce a gap between average and worst-group test performance under ERM.
  • Problem setup: The experiments assume access to spurious attributes for baseline training and feature-quality analysis, but ERM itself does not use those attributes.The authors state that their ERM representation results therefore also apply when spurious attributes are unknown.
  • Deep Feature Reweighting: DFR decomposes a model into a feature extractor and classification head, discards the head, and computes embeddings from the extractor.The resulting embeddings are used to train a new classifier.
  • Deep Feature Reweighting: The reweighting dataset is group-balanced held-out data where the spurious correlation does not hold, and the final classifier is trained by logistic regression on its embeddings.The paper uses a balanced validation subset as DFRVal.
  • Deep Feature Reweighting: For stability, logistic regression is trained ten times on different group-balanced subsets and the learned weights are averaged.

4 Experimental Setup and Evaluation Procedure

The evaluation spans four image and two text classification problems with distinct spurious attributes, using pretrained models and DFR-based measures of core- and spurious-feature decodability. It also reports base-model worst-group and mean accuracy without reweighting.

  • Datasets: The benchmark suite contains four image-classification and two text-classification problems.
  • Datasets: Waterbirds uses bird type as the class and background as the spurious feature, with landbirds usually on land and waterbirds usually over water.
  • Datasets: CelebA hair-color prediction uses blond classification with gender as the spurious feature, since 94% of blond-labeled images depict females.
  • Datasets: WILDS-FMOW predicts 62 land-use or building types while using geographic region as the spurious attribute, and includes domain shift between validation and test data.
  • Datasets: CXR-14 predicts pneumothorax, where chest drains form a hidden non-causal stratification; evaluation follows prior work by reporting worst-group AUC.
  • Datasets: MultiNLI uses negation words as a spurious correlate of contradiction, while CivilComments uses mentions of eight demographic or religious categories as its spurious attribute.
  • Datasets: FMOW and CXR-14 are challenging real-world datasets whose inputs differ from natural ImageNet images, limiting reliance on feature transfer.
  • Models and evaluation: The study uses pretrained ResNet-50, BERT, and DenseNet-121 models, and evaluates representations by DFR-based core-feature decoding and optional s-DFR spurious-attribute decoding.It also reports base WGA and base accuracy without DFR.

5 ERM vs Group Robustness Training

DFR reveals that ERM learns feature representations comparable to specialized group robustness methods, despite weaker base worst-group performance. The results indicate that robustness methods mainly improve last-layer feature weighting, while early stopping and training duration have limited effects on representation quality.

  • Results: ERM achieves 68.8% WGA on Waterbirds versus 90.6% for group DRO with early stopping before DFR.This illustrates the substantial base-model advantage of group robustness training on some datasets.
  • Results: After DFR, ERM and group DRO reach similar performance on Waterbirds: 91.1% for ERM versus 90% for group DRO.Similar patterns hold across the evaluated datasets.
  • Results: After DFR, no considered group robustness method improves worst-group accuracy over ERM by more than 1–2% across the datasets.The result applies to ERM comparisons with group DRO, RWG, and RWY.
  • Interpretation: These findings suggest that group robustness methods primarily improve the weighting of learned features in the final linear layer rather than the underlying feature representations.If they learned substantially better core-feature representations, DFR would be expected to produce larger gains over ERM.
  • Early stopping: Early stopping is crucial for strong base performance with RWY, RWG, and group DRO but generally does not significantly improve representation quality after DFR.The findings suggest that later training may increase spurious-feature weights while preserving core-feature information in the representations.
  • Group DRO analysis: For the best group DRO runs, DFR does not improve performance, indicating that group DRO already learns a nearly optimal last layer.DFR can nevertheless improve poorly performing final checkpoints, removing the need for early stopping in those cases.

6 Effect of the Base Model

Architecture and pretraining substantially shape learned feature quality: stronger in-distribution models generally yield better DFR worst-group accuracy, while large-scale pretraining can deliver strong performance before target-data training.

  • Architecture and pretraining: 78 Waterbirds, 78 CelebA, 40 FMOW and 40 CXR models were evaluated across architectures and pretraining strategies.The study compares base and DFR performance across a wide model set.
  • Architecture and pretraining: DFR worst-group accuracy is highly linearly correlated with in-distribution accuracy on every dataset except CXR.This relationship includes Waterbirds and CelebA, where base worst-group accuracy is not fully linear with mean accuracy.
  • Architecture and pretraining: 97.2% DFR WGA on Waterbirds and 92.2% on CelebA were achieved with ConvNext Large, while ConvNext Small reached 50.6% on FMOW.These results significantly improve upon previously reported results according to the authors.
  • Architecture and pretraining: Models with stronger in-distribution generalization generally provide better worst-group performance and core-feature representations across CNN and ViT architectures.The authors report no qualitative robustness difference between vision transformers and CNNs in their experiments.
  • Training on target data: On Waterbirds, ImageNet1k-pretrained ResNet-50 achieved 88.2% DFR WGA without target-data training and 92.9% after finetuning.On other datasets, DFR WGA improved after training; on CelebA, the reported values were 88.3% without and 92.2% with target-data training.
  • Training on target data: 94% DFR WGA on Waterbirds was achieved without target-data feature-extractor training using ConvNext Large pretrained on ImageNet22k.After target-data training, performance was 94.3%, indicating little change in this setting.
  • Training on target data: On CelebA, the spurious gender feature became less predictable during training for many models, while Waterbirds and FMOW showed no consistent trend.This pattern is measured using DFR s-WGA.

7 Effect of Regularization

The study finds that strong regularization is generally unnecessary for high-quality feature representations, although particular augmentation and weight-decay choices can modestly affect performance.

  • Weight decay: Zero weight decay can provide near-optimal DFR WGA despite underperforming in base WGA on Waterbirds and CelebA.On CelebA, no weight decay had the worst base WGA but the best DFR WGA.
  • Weight decay: Strong weight decay can improve base worst-group accuracy by reducing spurious-feature reliance in the last layer without improving feature representations.The authors report similar or worse DFR WGA under strong weight decay.
  • Weight decay: FMOW was the only considered dataset where non-zero weight decay appeared helpful for learning high-quality core-feature representations.On other datasets, zero weight decay remained competitive.
  • Data augmentation: AugMix provided the best performance on each dataset, while random erasing hurt Waterbirds and MixUp hurt FMOW.Random erasing may cover the relatively small bird region more often than the background, encouraging spurious-feature reliance; its DFR s-WGA was 91.8±0.1%.
  • Overview: Strong regularization is not necessary for robustness to spurious correlations, though appropriate augmentation and weight decay can provide small improvements.This conclusion covers both regularization techniques studied.

8 Discussion

Worst-group performance reflects both core-feature representation quality and last-layer feature weighting. The paper argues that architecture and pretraining matter more for representation quality than many robustness-oriented training or regularization choices.

  • Discussion: Worst-group performance depends on core-feature representation quality and the weight assigned to core features in the last classification layer.The discussion separates feature extraction from final-layer weighting.
  • Discussion: Many group robustness methods primarily improve worst-group accuracy through better last-layer weighting rather than better core-feature representations.The same distinction applies to early stopping and strong weight decay.
  • Discussion: Base architecture and pretraining strategy have a major effect on learned feature representations, whereas regularization does not consistently improve them.This conclusion is supported across the paper’s experiments.
  • Open question: The paper leaves open whether standard ERM can be significantly improved in representation quality for a fixed base model.Future directions include richer feature construction, gradient starvation, ensembling, and feature-diversity methods.

A.1 Datasets

The study evaluates four image and two text classification datasets with spurious correlations, documenting their group structures, attributes, labels, and preprocessing choices.

  • Datasets: The experiments cover four image datasets and two text datasets with spurious attributes correlated with target labels.The image datasets are Waterbirds, CelebA hair color, WILDS-FMOW, and CXR; the text datasets are Civil Comments and MultiNLI.
  • Image datasets: Waterbirds uses bird type as the core feature and background type as the spurious attribute, with four groups defined by (y, s).Waterbird images combine CUB birds with Places backgrounds, whose water-versus-land association is spuriously correlated with bird type.
  • Image datasets: CelebA predicts blond versus non-blond hair color while gender serves as the spurious attribute, producing four (y, s) groups.Blond people are predominantly female in the dataset.
  • Image datasets: FMOW predicts 62 building or land-use classes across regions, whereas CXR predicts pneumothorax and includes a chest-drain shortcut with three available groups.FMOW regions are unevenly represented, and CXR spurious labels are available only for validation and test images showing sick patients.
  • Text datasets: Civil Comments uses protected-attribute mentions as the spurious feature, while MultiNLI uses negation words; both features are correlated with class labels.Civil Comments classifies comments as toxic or non-toxic, and MultiNLI classifies sentence pairs as contradiction, entailment, or neutral.

B.1 Methods and hyper-parameters

The paper compares ERM, reweighting methods, and Group DRO, then evaluates their learned representations by retraining a regularized linear classifier with DFR.

  • Training methods: Group DRO minimizes worst-group loss, while ERM, RWY, and RWG provide comparison training procedures across the datasets.Group DRO uses a neural model, cross-entropy loss, group set G, group sizes n_g, and adjustment hyper-parameter C.
  • Dataset-specific procedure: For CXR, DFR trains on the full validation set without group balancing and selects regularization using worst AUC rather than worst-group accuracy.This adapts DFR to CXR’s available labels and evaluation setup.
  • Representation evaluation: DFR s-WGA applies the same DFR procedure to predict the spurious attribute instead of the class label.This measures how decodable the spurious feature remains in the learned representation.
  • Results: After DFR, methods achieve similar performance, although Group DRO with early stopping typically retains a small improvement.The improvement is much smaller than the gain from using a better base model.

C Details: Effect of the Base Model

The base-model experiments examine training length, augmentation, weight decay, and architectures, finding that representation quality is influenced more by model choices than by prolonged training or strong regularization.

  • Model comparison: Some ViT models perform poorly because the default hyper-parameters are suboptimal, and further tuning may improve results, especially for ViT-based models.This limits direct comparisons among architectures under the fixed default settings.
  • Regularization and augmentation: The study varies weight decay and image augmentation while using default models and hyper-parameters to assess their effects on learned features.The augmentation comparison includes the policies described in the implementation section.
  • Training length: Five epochs generally suffice for near-optimal DFR WGA under both ERM and Group DRO.For ERM, longer training neither helps nor hurts DFR WGA even when it hurts base-model WGA; Group DRO DFR WGA deteriorates over time on Waterbirds.
  • Training length: Longer ERM training does not improve DFR WGA, which remains roughly constant after rapid convergence across datasets.This contrasts with possible deterioration in base-model performance during continued training.

E Additional results on MultiNLI

Additional MultiNLI experiments compare pretrained model families, pretraining strategies, and target-data training, showing that architecture and pretraining materially affect feature quality.

  • Experimental setup: The MultiNLI experiments train models for five epochs with learning rate 10^-5 and zero weight decay.Weight decay is separately ablated for BERT-Base using AdamW.
  • Effect of the base model: DeBERTa-Large provides the best base performance and DFR WGA among the evaluated BERT and DeBERTa models.The comparison includes BERT-Base, BERT-Large, DeBERTa-Base, and DeBERTa-Large.
  • Effect of pretraining: Pretraining is necessary for strong MultiNLI performance, while different pretraining datasets produce competitive results.The comparison uses BERT-Base models with different pretraining configurations.
  • Effect of target-data training: Training on MultiNLI makes both core and spurious features significantly more decodable for the evaluated pretrained BERT and DeBERTa models.This differs from Waterbirds, where training did not significantly improve DFR WGA.

F Broader Impact and Limitations

The paper identifies limits in the factors and procedures studied, while emphasizing that ERM does not eliminate the practical risks of spurious correlations. It also reports implementation details and selected experimental observations about pretraining and weight decay.

  • Limitations: The study does not cover every factor affecting feature learning, including regularization methods beyond weight decay and early stopping and diverse feature-learning methods.The authors specifically mention DivDis and Teney et al.’s method as examples for future study.
  • Experimental Context: Pretraining is required for strong MultiNLI performance, while multilingual pretraining is competitive but inferior to English pretraining.
  • Limitations: DFR performs well experimentally but is not guaranteed to learn an optimal linear classifier, so improved last-layer training could refine the analysis.
  • Broader Impact: ERM still performs suboptimally because it does not provide a correct weighting of features in the final classification layer, so spurious correlations remain a practical concern.
  • Experimental Context: Weight decay 0 provides competitive MultiNLI performance, while the best DFR worst-group accuracy is achieved with weight decay 1.
Loading 2210.11369v1…