PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning De-biased Representations with Biased Representations

Hyojin Bahng, Sanghyuk Chun, Sangdoo Yun, Jaegul Choo, Seong Joon Oh

arXiv:1910.02806v3cs.CVcs.LGstat.ML

TL;DR

In-distribution evaluation can hide models’ reliance on dataset biases, leaving cross-bias generalisation inadequately assessed and making conventional de-biasing costly or difficult. ReBias encourages task representations to be statistically independent of representations from models biased by design, using HSIC. Across synthetic and real-world biases, it reduces shortcut dependence and improves generalisation when bias shifts or is removed.

  • Problem

    Random train-test splits can conceal reliance on dataset biases, while augmentation, re-sampling, and explicit bias quantification may be costly or difficult.

  • Method

    ReBias encourages a task representation to be statistically independent of representations from a set of models designed to capture the bias, using HSIC.

  • Results

    ReBias reduces shortcut dependence and improves test accuracy when bias is shifted or removed across synthetic and real-world settings.

  • Takeaways & Limitations

    Defining biased representations by design can support de-biased learning when directly defining or quantifying the bias is difficult.

  • Takeaways & Limitations

    Realistic cross-bias generalisability is difficult to evaluate directly, so the experiments use surrogate measures.

Abstract

from arXiv · show

Many machine learning algorithms are trained and evaluated by splitting data from a single source into training and test sets. While such focus on in-distribution learning scenarios has led to interesting advancement, it has not been able to tell if models are relying on dataset biases as shortcuts for successful prediction (e.g., using snow cues for recognising snowmobiles), resulting in biased models that fail to generalise when the bias shifts to a different class. The cross-bias generalisation problem has been addressed by de-biasing training data through augmentation or re-sampling, which are often prohibitive due to the data collection cost (e.g., collecting images of a snowmobile on a desert) and the difficulty of quantifying or expressing biases in the first place. In this work, we propose a novel framework to train a de-biased representation by encouraging it to be different from a set of representations that are biased by design. This tactic is feasible in many scenarios where it is much easier to define a set of biased representations than to define and quantify bias. We demonstrate the efficacy of our method across a variety of synthetic and real-world biases; our experiments show that the method discourages models from taking bias shortcuts, resulting in improved generalisation. Source code is available at https://github.com/clovaai/rebias.

1. Introduction

Standard random train-test splits can conceal reliance on dataset biases, producing models that fail when those biases shift. The paper proposes ReBias, which encourages task representations to differ from intentionally biased representations, improving shifted-bias accuracy.

  • Randomly splitting one data source for training and testing is blind to whether models exploit dataset bias as a shortcut.
  • Bias shortcuts can yield high in-distribution accuracy while causing poor generalisation when the bias changes.Examples include swamp backgrounds for frogs, local texture for images, and static cues for videos.
  • Existing de-biasing methods augment data, define biases explicitly, or reweight and ensemble logits, but real-world biases are often difficult to define or quantify.
  • ReBias trains a de-biased representation by encouraging statistical independence from representations produced by models biased by design.The framework uses the Hilbert-Schmidt Independence Criterion to formulate independence.
  • ReBias reduces dependence on training-data shortcuts and improves accuracy when test-time bias is shifted or removed.

2. Problem Definition

The paper distinguishes in-distribution, cross-domain, and cross-bias shifts by how signal, bias, and labels relate across training and test data. It motivates representing bias through deliberately biased model classes when direct bias labels, disentanglement, or new data are impractical.

  • Signals are cues essential for predicting the target, whereas biases are nonessential cues correlated with the target and invariant to interventions on the bias.A frog’s shape is a signal; its swamp background is a bias because changing the scene should not change the label.
  • The framework separates learning into in-distribution, cross-domain, and cross-bias generalisation according to changes in p(S, B, Y) between training and test.
  • In-distribution learning preserves the joint distribution, while cross-domain learning changes the marginal bias distribution between training and test.
  • Cross-bias generalisation preserves p(B) but changes the dependence between bias and label, producing unusual bias-label combinations at test time.
  • Prior strategies require disentanglement, additional or generated data, or ground-truth or predictive knowledge of p(B|X), assumptions that are often unrealistic for texture bias.Texture and shape may be inseparable, data collection or generation may be expensive, and enumerating all textures is infeasible.
  • A bias-characterising model class G approximates bias by design; small-receptive-field CNNs, for example, capture texture patterns and are more liable to overfit to texture.
  • ReBias assumes G is known and regularises task model f to be independent of predictions from models in G, while allowing G to have imperfect recall or precision.

3. Proposed Method

ReBias trains a task representation to be independent of biased representations produced by a designed model class, using HSIC-based regularisation. Its analysis connects this independence to differing invariance types, while toy experiments show improved de-biased accuracy.

  • ReBias: Removing bias with bias: ReBias trains f for p(Y|X) while regularising independence between f(X) and predictions from biased models g ∈ G.The biased model class G is designed to capture a known bias, and ReBias encourages f to differ from multiple biased predictions.
  • Independence criterion: HSIC measures dependence between high-dimensional representations, with zero HSIC under RBF kernels iff the variables are statistically independent.HSIC uses the Hilbert-Schmidt norm of a cross-covariance operator in reproducing kernel Hilbert spaces.
  • Why and how does it work?: In the linear Gaussian case, statistical independence of affine representations is equivalent to covariance-weighted orthogonality of their null spaces.For one-dimensional outputs, this becomes orthogonality of weight vectors and decision boundaries.
  • Why and how does it work?: For general nonlinear models, the paper interprets HSIC minimisation as metric learning that encourages f and g to learn different invariance types.The analysis notes that a complete theory for arbitrary nonlinear function pairs remains an open question.
  • Effect of HSIC regularisation on toy data: 89% accuracy on de-biased toy test data was achieved with HSIC regularisation, versus 55% for an MLP trained only on perfectly biased data.The bias-characterising class consisted of MLPs using only the bias dimension, and the regularised model learned de-biased decision boundaries.

4. Experiments

Experiments evaluate ReBias on synthetic colour bias and realistic texture and static biases, using biased-model families to encourage representations that capture cues beyond those biases. Across Biased MNIST, ImageNet, and action recognition, ReBias improves cross-bias generalisation while generally preserving in-distribution performance.

  • Experimental setup: ReBias is evaluated on synthetic Biased MNIST and realistic ImageNet classification and action recognition biases.Biased MNIST controls colour bias; ImageNet uses clustering-based proxy ground truths for texture bias, and action recognition uses the Mimetics dataset.
  • Biased MNIST: At ρ = 0.999, ReBias raises Biased MNIST unbiased accuracy from 10.4% to 22.7%, and at ρ = 0.997 from 33.4% to 64.2%.The gains occur when colour cues are highly correlated with digit labels during training.
  • Biased MNIST: On Biased MNIST, ReBias uniformly improves off-diagonal accuracies without sacrificing diagonal accuracies, unlike baselines with limited or uneven improvements.Diagonals represent biased accuracy, while the overall average represents unbiased accuracy.
  • Learning curves: During training with ρ = 0.997, unbiased accuracy rises from 33.4% to above 60% as HSIC between F and G is minimized.The learning curves show a strong correlation between HSIC values and unbiased accuracies.
  • ImageNet: On ImageNet, ReBias improves ResNet18 texture-unbiased accuracy from 88.8% to 90.5% and ImageNet-A accuracy from 24.9% to 29.6%.ReBias also improves biased, unbiased, ImageNet-A, and ImageNet-C accuracies overall; its ImageNet-C accuracy is 57.5%.
  • Action recognition: For action recognition, ReBias improves Kinetics accuracy from 54.5% to 55.8% and Mimetics accuracy from 18.9% to 22.4%.The comparison uses 3D-ResNet18 as F and 2D-ResNet18 as the static-biased model G.

5. Conclusion

The paper identifies cross-bias generalisation as a practical problem: models exploit sufficient bias shortcuts instead of extracting non-bias signals such as global shapes or temporal actions.

  • Models exploit bias shortcuts when those cues provide sufficient evidence for recognising the target.

A. Statistical Independence is Equivalent to Functional Orthogonality for Linear Maps

The appendix connects statistical independence of affine representations to a covariance and functional-orthogonality condition, then describes the minimax training algorithm using HSIC.

  • For affine mappings of normally distributed inputs, representation independence is equivalent to a covariance condition.The appendix states f(X) ⊥⊥ g(X) if and only if ker(A) is orthogonal to ker(B) under the covariance matrix.
  • The covariance of affine representations is computed as AΣB^T, whose vanishing is equivalent to covariance-weighted functional orthogonality.
  • Algorithm 1 solves the minimax objective by alternating updates of f and g over mini-batches until convergence.
  • The training objective combines the original task loss with an HSIC term measuring independence between f(X) and g(X).HSIC is estimated with an unbiased finite-sample estimator within each mini-batch.

C. Implementation Details

The implementation uses alternating ADAM optimisation, dataset-specific training schedules, and architecture details tailored to the Biased MNIST, ImageNet, and action-recognition experiments.

  • The appendix reports an unbiased HSIC estimator and uses alternating stochastic gradient descent with ADAM to solve the minimax problem.The regularisation parameters λ and λg are set to 1.0 in all experiments.
  • Batch sizes are 256, 128, and 128 for Biased MNIST, ImageNet, and action recognition, respectively.Learning-rate schedules differ across the three datasets.
  • AdamP improves performance in Biased MNIST and ImageNet experiments and is recommended for better optimisation stability.
  • Table A1 reports AdamP experiments on the Biased MNIST and 9-Class ImageNet benchmarks.
  • Comparison methods use method-specific training procedures, including pre-training G for LearnedMixin and simultaneous F,G updates for RUBi.

D. Standard Errors in Experimental Results

The appendix reports standard errors for the main results across Biased MNIST, ImageNet, and action recognition.

  • Standard errors are reported for the Biased MNIST, ImageNet, and action-recognition results in Tables A3, A4, and A5.

E. Decision Boundary Visualisation for Toy Experiment

The toy experiment’s decision boundaries are shown in Figure A1, while the supplied tables describe architectures and evaluation summaries for the broader experiments.

  • Figure A1 visualises the toy experiment’s decision boundaries.
  • Table A2 describes the action-recogniser architecture using temporal, width, height, and channel kernel dimensions.
  • Table A3 reports unbiased accuracy on Biased MNIST across varying train correlations ρ.
  • Table A4 reports ImageNet results for ResNet18 and BagNet18, including ImageNet-A.
  • Means and standard errors are computed over three independent runs.
  • Table A5 reports top-1 action-recognition accuracies for 3D-ResNet18 and 2D-ResNet18 with baseline comparisons.

F. Texture Clustering on ImageNet

The ImageNet analysis approximates texture ground-truth labels by clustering low-level texture features, revealing strong correlations between texture clusters and semantic classes.

  • Texture features are extracted from relu1_2 of ImageNet-pretrained VGG16 using Gram matrices of low-layer feature maps.The features are intended to capture edge and colour cues.
  • Mini-batch k-means with k = 9 and batch size 1024 clusters the texture features of 9-Class ImageNet data.Three initialisations are used because k-means is non-convex, and performances are averaged across trials.
  • Texture clusters capture similar texture patterns and exhibit strong correlations with semantic classes.For example, the “water” texture is strongly associated with turtle, fish, and bird classes.
  • Figure A2 shows example images from texture clusters, while Figure A3 visualises each cluster’s top-3 correlated classes.

G. Further Analysis on Texture Bias of ImageNet-Trained Models

The ImageNet bias analysis compares texture-class-wise accuracies and finds that ReBias narrows the biased-versus-unbiased accuracy gap while improving reported accuracies.

  • The analysis compares texture-class-wise accuracies for vanilla-trained and ReBias-trained ResNet18 models.Dominant texture clusters are defined as the largest cluster containing more than 30% of a class’s samples.
  • 90.6% versus 86.3%: vanilla ResNet18 is more accurate on biased classes than less biased classes.The difference is presented as signifying a bias towards texture.
  • 94.8% versus 90.4%: ReBias reports similar accuracies on biased and unbiased classes.The passage states that ReBias overcomes the bias and enhances generalisation across distributions.
  • Figure A4 visualises accuracy for every texture-class pair and masks cells with population below 10 in gray.
  • Figure A5 contrasts Kinetics examples, which have certain scene-context biases, with relatively context-unbiased Mimetics examples.

H. Action Recognition Datasets

The action-recognition evaluation uses paired Kinetics and Mimetics data to examine generalisation when static scene and object correlations differ.

  • Kinetics contains 300K videos across 400 action classes, whereas Mimetics contains 713 videos across 50 classes.Mimetics classes are a subset of Kinetics classes.
  • The investigation subsamples 10 action classes shared by Kinetics and Mimetics.
  • Kinetics samples are biased toward static scene and object cues, while Mimetics is relatively free from such correlations.The passage identifies Mimetics as a suitable benchmark for cross-bias generalisation.
Loading 1910.02806v3…