Source-linked AI summary
REPAIR: Removing Representation Bias by Dataset Resampling
Yi Li, Nuno Vasconcelos
TL;DR
Representation bias lets models exploit feature-specific shortcuts rather than learning the underlying task, motivating methods to reduce bias in datasets. The paper introduces REPAIR, an optimization-based resampling procedure with an evaluation framework, and reports substantial bias reduction plus improved generalization on synthetic and action-recognition data.
Problem
Representation bias allows algorithms to achieve high dataset performance through favored representations without learning the underlying task.
Method
REPAIR formulates bias minimization as a minimax optimization that alternates classifier-parameter and dataset-weight updates using stochastic gradient descent.
Results
REPAIR significantly reduces representation bias and improves generalization across controlled Colored MNIST and large-scale action-recognition experiments.
Takeaways & Limitations
The proposed evaluation setup connects dataset representation bias with model generalization and the fairness of algorithm comparisons.
Takeaways & Limitations
Cross-dataset generalization assumes the compared datasets do not share exactly the same representation bias.
Abstract
from arXiv · showhide
Modern machine learning datasets can have biases for certain representations that are leveraged by algorithms to achieve high performance without learning to solve the underlying task. This problem is referred to as "representation bias". The question of how to reduce the representation biases of a dataset is investigated and a new dataset REPresentAtion bIas Removal (REPAIR) procedure is proposed. This formulates bias minimization as an optimization problem, seeking a weight distribution that penalizes examples easy for a classifier built on a given feature representation. Bias reduction is then equated to maximizing the ratio between the classification loss on the reweighted dataset and the uncertainty of the ground-truth class labels. This is a minimax problem that REPAIR solves by alternatingly updating classifier parameters and dataset resampling weights, using stochastic gradient descent. An experimental set-up is also introduced to measure the bias of any dataset for a given representation, and the impact of this bias on the performance of recognition models. Experiments with synthetic and action recognition data show that dataset REPAIR can significantly reduce representation bias, and lead to improved generalization of models trained on REPAIRed datasets. The tools used for characterizing representation bias, and the proposed dataset REPAIR algorithm, are available at https://github.com/JerryYLi/Dataset-REPAIR/.
1. Introduction
The paper frames representation bias as a dataset property that can let models exploit shortcuts instead of learning the underlying task. It introduces REPAIR, an optimization-based resampling procedure, and an evaluation setup showing reduced bias and improved model generalization.
- Representation bias occurs when datasets favor particular feature representations, allowing models to solve them without learning the intended task.
- Action-recognition datasets may support competing cues, including single frames, long-term dependencies, human pose, and context.
- REPAIR assigns example-level weights that penalize examples easy for a classifier built on a selected feature representation.
- The method formulates bias reduction as maximizing classifier loss relative to ground-truth label uncertainty, producing a minimax optimization problem.Classifier parameters and resampling weights are alternately updated with stochastic gradient descent.
- REPAIR substantially reduces representation bias in controlled grayscale-image experiments and action-recognition datasets, unlike random subsampling.The evaluation setup also examines how bias affects model performance, cross-dataset generalization, and fairness of algorithm comparisons.
2. Related Work
Related work distinguishes dataset bias from representation bias and places both within broader concerns about fairness and generalization. It also situates the paper among action-recognition methods and dataset-resampling approaches.
- Fairness research has primarily examined bias in learned models, while bias in the data itself has received less attention.
- Dataset bias is distribution mismatch caused by data-collection procedures, including mismatch between datasets and real-world observations.Prior work linked dataset bias to negative cross-dataset model generalization.
- Representation bias describes how a feature representation can solve a dataset and enable shortcuts that bypass the underlying task.Examples include contextual cues and model preferences for shape or other representations.
- Action-recognition research spans hand-crafted descriptors, two-stream and 3D convolutional networks, recurrent networks, and methods modeling long-term temporal structure.
- Dataset resampling changes example frequencies, commonly balancing classes through minority oversampling and majority undersampling.It is also used to train models that do not discriminate against minority classes.
3. Minimum-bias Dataset Resampling
The paper defines representation bias through the predictive information of a fixed feature representation and minimizes it by reweighting dataset examples. REPAIR alternates classifier and weight optimization, then resamples examples to reduce bias.
- 3.1. Representation Bias: Representation bias measures how strongly a dataset favors a feature representation φ through its best achievable classification performance, normalized by chance level.The representation φ remains fixed while a classifier is trained on its features.
- 3.1. Representation Bias: Bias is quantified from cross-entropy risk and class-label entropy, with lower risk indicating a more informative representation and larger bias.The normalized bias lies in [0, 1] and characterizes uncertainty reduction about Y when Z is observed.
- 3.1. Representation Bias: The bias estimator is a differentiable linear classifier whose loss and class frequencies determine the bias measure and enable its optimization.Differentiability with respect to classifier parameters distinguishes this formulation and supports bias minimization.
- 3.2. Adversarial Example Reweighting: REPAIR assigns example-level resampling weights to create a dataset D′ that is less biased toward the selected representation.The weights encode each example’s probability of being selected during non-uniform resampling.
- 3.2. Adversarial Example Reweighting: The method solves a minimax problem by alternating classifier optimization with weight optimization, assigning larger weights to examples misclassified by the bias estimator.Sigmoid parameterization keeps each weight in (0, 1), while classifier updates improve bias estimates.
- 3.3. Mini-batch Optimization: Mini-batch estimates enable stochastic-gradient optimization of the weights, and REPAIR combines training and test splits so their distributions remain matched after resampling.The risk and entropy are estimated by replacing dataset size with the mini-batch size.
4. Case studies
The case studies show how representation bias can undermine generalization and model evaluation, while REPAIR resampling reduces bias and improves cross-dataset or task-relevant performance. Controlled Colored MNIST experiments and action-recognition experiments provide complementary evidence for this effect.
- 4.1. Colored MNIST: Colored MNIST creates controllable color bias, allowing representation bias and its effect on digit-recognition generalization to be measured directly.Color distributions are class-dependent, so color can reveal the digit without modeling digit shape.
- 4.1. Colored MNIST: 10% accuracy: under the strongest color bias, CNN performance on grayscale MNIST approaches chance despite deceptively high performance on Colored MNIST.The CNN can overfit to color without learning properties of digits or digit recognition.
- 4.1. Colored MNIST: All four weight-based resampling strategies significantly reduce color bias relative to both no resampling and uniform resampling, improving MNIST test generalization.Thresholding and ranking are more effective for large biases, corresponding to small color-distribution variances.
- 4.2. Scenario II: Action Recognition: REPAIR targets static bias in action-recognition datasets by assigning lower weights to videos with abundant static cues and higher weights to videos lacking significant static cues.Examples include pool tables in billiards and parallel lines in playing harp versus complex human interactions in push.
- 4.2. Scenario II: Action Recognition: 50% resampling reverses the UCF101 ranking of C2D and TSN, showing that original-dataset rankings can reflect reliance on representation bias rather than architecture quality alone.All evaluated networks perform worse on REPAIRed datasets, and the accuracy drop measures dependence on static features.
- 4.2. Scenario II: Action Recognition: ∼2% mAP: resampling increases mean average precision across 12 Kinetics-to-HMDB51 classes, while temporal-structure scores also increase as biased videos are removed.Seven of 12 classes show negative bias-generalization correlations, and random removal does not produce the same temporal-structure improvement.
5. Conclusion
REPAIR is presented as an effective dataset-resampling procedure for minimizing representation bias across controlled Colored MNIST and large-scale action-recognition settings. The paper also introduces experiments connecting bias removal with recognition-model generalization and evaluation fairness.
- REPAIR minimizes dataset representation bias through a novel resampling procedure.
- The procedure is effective in both controlled Colored MNIST experiments and large-scale modern action-recognition datasets.
- The evaluation relates representation bias to recognition-model generalization capability and the fairness of model evaluation.