PaperlayerView exact revision on arXiv

Source-linked AI summary

Fairness in Machine Learning: A Survey

Simon Caton, Christian Haas

arXiv:2010.04053v1cs.LGstat.ML

TL;DR

As ML increasingly affects socially consequential decisions, researchers and practitioners need ways to address bias, but fairness research remains complex and difficult to navigate. This survey synthesizes fairness concepts and technical approaches through a pre-processing, in-processing, and post-processing framework, alongside broader method areas, tools, and open dilemmas. It concludes by organizing unresolved challenges around incompatible fairness goals, context and policy, data, and practical intervention constraints.

  • Problem

    ML systems increasingly affect citizens through socially consequential applications, while fairness research is complex and difficult for newcomers to navigate.

  • Method

    The survey provides an entry-level overview, standardizes fairness-measurement notation, develops a two-dimensional taxonomy, reviews toolkits, and covers intervention stages and broader fairness settings.

  • Results

    The paper synthesizes fairness concepts, mitigation approaches, available toolkits, and open research dilemmas across the fairness in ML literature.

  • Takeaways & Limitations

    Fairness work must account for protected and proxy variables, competing fairness criteria, contextual factors, and constraints associated with different intervention stages.

  • Takeaways & Limitations

    The literature lacks consensus on fairness, and fairness criteria can be incompatible with one another or with social, economic, and legal understandings.

Abstract

from arXiv · show

As Machine Learning technologies become increasingly used in contexts that affect citizens, companies as well as researchers need to be confident that their application of these methods will not have unexpected social implications, such as bias towards gender, ethnicity, and/or people with disabilities. There is significant literature on approaches to mitigate bias and promote fairness, yet the area is complex and hard to penetrate for newcomers to the domain. This article seeks to provide an overview of the different schools of thought and approaches to mitigating (social) biases and increase fairness in the Machine Learning literature. It organises approaches into the widely accepted framework of pre-processing, in-processing, and post-processing methods, subcategorizing into a further 11 method areas. Although much of the literature emphasizes binary classification, a discussion of fairness in regression, recommender systems, unsupervised learning, and natural language processing is also provided along with a selection of currently available open source libraries. The article concludes by summarising open challenges articulated as four dilemmas for fairness research.

1 Introduction

Fairness in ML has become important as automated systems increasingly affect socially consequential decisions, while the field remains difficult to navigate and lacks consensus on fairness itself. This survey provides an entry-level synthesis of concepts, methods, tools, and open dilemmas.

  • ML systems increasingly influence socially consequential contexts, including re-offence assessment, automated support, and job-application screening.
  • Historical fairness research supplies concepts that remain relevant to ML, including protected variables, group and individual fairness, incompatibility conditions, and fairness metrics.
  • No general consensus emerged on what fairness means or whether it should be quantified, partly because fairness notions and measurements can conflict.
  • Bias can arise from intrinsic data biases, inappropriate data use, poor quality, context shifts, and subjectivity filters, leaving ML researchers with many challenges.
  • The survey offers an entry-level overview, standardizes fairness-measurement notation, builds a two-dimensional taxonomy, reviews toolkits, and outlines future research dilemmas.

2 Fairness in Machine Learning: key methodological components

The survey frames technical fairness interventions around where they act in the ML pipeline and around the variables, relationships, metrics, and trade-offs that shape fairness judgments. It covers pre-processing, in-processing, and post-processing while emphasizing unresolved limits involving proxies, incompatible criteria, and deployment constraints.

  • Technical fairness interventions are organized by pipeline location: pre-processing before modeling, in-processing during modeling, and post-processing after modeling.
  • 2.1 Sensitive and Protected Variables and (Un)privileged Groups: Protected variables define the sensitive aspects used to identify protected and unprivileged groups, but fairness analysis must also consider correlated proxy variables.
  • 2.1 Sensitive and Protected Variables and (Un)privileged Groups: Overlooked variable relationships can enable deanonymization, while fairness methods often assume prior knowledge of quasi-identifiers or discover them case by case.
  • Fairness measurement is difficult because metrics may prioritize individual or group fairness, and mathematical definitions may not match social, economic, or legal meanings.
  • Pre-processing repairs training data, in-processing incorporates fairness metrics into model optimization, and post-processing transforms predictions using predictions and sensitive attributes.
  • Pre- and post-processing preserve unchanged ML methods and support library reuse, but data or output modifications can reduce interpretability and create legal concerns.

3 Measuring Fairness and Bias

Fairness measurement in ML comprises diverse metrics and criteria that capture different aspects of group or individual outcomes. Because these notions can be difficult or impossible to combine, the survey presents their definitions, notation, trade-offs, and limitations rather than identifying one universally best measure.

  • Overview: Fairness metrics lack consistent naming conventions and notation, while different interpretations of algorithmic fairness motivate broader taxonomies and summaries.The survey extends prior overviews with additional perspectives on bias types and a larger set of metrics and definitions.
  • Trade-offs and Limitations: No universal fairness measure or clear guideline identifies which metric is best, because each notion emphasizes different aspects of fairness and many are difficult or impossible to combine.The survey therefore offers category overviews and lay interpretations to inform decision making.
  • Abstract Fairness Criteria: Binary-classification fairness measures commonly use a sensitive variable S, target Y, and score R, with metrics organized around Independence, Separation, and Sufficiency.The notation framework treats S as defining groups, Y as the binary target, and R as the predicted score in [0, 1].
  • Trade-offs and Limitations: Fairness criteria can be mutually incompatible: when S and Y are not independent, Independence and Sufficiency cannot both hold.More broadly, the literature reports trade-offs between fairness measures and model accuracy, as well as disagreements over individual versus group fairness.
  • Group-based Metrics: Group-based metrics compare outcomes across groups, including statistical parity, disparate impact, confusion-matrix measures, calibration-based measures, and related criteria.Parity metrics compare predicted positive rates; disparate impact uses the ratio between unprivileged and privileged groups, sometimes applying an 80% rule.
  • Confusion Matrix-based Metrics: Equal Opportunity requires equal true-positive rates across groups, whereas Equalized Odds additionally requires equal false-positive rates.These metrics use confusion-matrix quantities to account for underlying group differences that parity-based approaches may omit.

4 Binary Classification Approaches

The survey organizes binary-classification fairness interventions by their location in the ML framework: pre-processing, in-processing, and post-processing. Binary classification dominates because many motivating decisions are binary and fairness quantification is mathematically more convenient.

  • Binary classification dominates fairness-mitigation research because many contentious applications involve binary decisions such as hiring, lending, and re-offending.
  • Fairness quantities are mathematically more convenient for binary dependent variables than for multi-class problems.Extending fairness quantities to multi-class settings would add terms to the formulation.
  • The taxonomy arranges mitigation methods by intervention stage: pre-processing, in-processing, and post-processing.Some method families, including adversarial-learning approaches, can span multiple stages.

4.1 Blinding

Blinding seeks to make classifiers immune to sensitive variables, but simply omitting those variables can overlook proxies and joint characteristics that sustain or conceal discrimination.

  • Blinding makes a classifier immune to one or more sensitive variables, whereas omission removes sensitive variables from the training data.The survey distinguishes immunity from omission rather than treating them as identical approaches.
  • Omitting sensitive variables can reduce model accuracy without improving discrimination.The survey cites prior work reporting both effects.
  • Sensitive-variable omission and immunity can overlook proxy variables and joint characteristics, potentially increasing or concealing discrimination.

4.2 Causal Methods

Causal methods model dependencies between sensitive and non-sensitive variables to identify discrimination and proxies, and can use those dependencies to repair training data under fairness constraints.

  • Causal methods seek to uncover causal relationships and dependencies between sensitive and non-sensitive variables in training data.
  • These methods are suited to identifying proxies of sensitive variables and analyzing which subgroups are most or least fair.
  • Data-repair extensions insert, modify, or remove training samples to satisfy fairness-specific constraints and conditional-independence properties.
  • Initial data-repair results produced debiased classifiers robust to unseen test data but requiring significant computational resources.

4.3 Sampling and Subgroup Analysis

Sampling and subgroup-analysis methods either correct training data for robust learning or evaluate how classifiers affect disadvantaged groups. Related transformation methods seek fair representations while preserving task fidelity, subject to utility and data-type constraints.

  • Sampling and Subgroup Analysis: Sampling methods create training samples to correct bias or identify disadvantaged groups for model evaluation.
  • Sampling and Subgroup Analysis: Decoupled classifiers split data into sensitive-variable subgroups, while multitask learning can learn such groupings during preprocessing.These approaches seek accurate models for particular subgroups.
  • Sampling and Subgroup Analysis: Selecting subgroup definitions requires balancing atomicity against rarity and robustness to gerrymandering.Recursive partitioning and clustering are common group-formation approaches.
  • Sampling and Subgroup Analysis: Subgroup analysis uses statistical quantities and hypothesis testing to compare model fairness across observed groups.
  • Transformation: Transformation methods learn mappings or projections that seek fairness while preserving ML-task fidelity, but current approaches mainly support numeric data.
  • Transformation: Transformation involves a trade-off between fairness repair and classifier performance, and excessive alteration can reduce utility or incur data loss.

4.5 Relabelling and Perturbation

Relabelling and perturbation modify training data to address fairness, while sensitivity analysis examines how feature uncertainty affects outcomes without directly improving fairness.

  • Relabelling and Perturbation: Relabelling flips or modifies training labels, whereas perturbation changes the distribution of one or more training variables.These approaches are transformation methods, with relabelling also described as data-massaging.
  • Relabelling and Perturbation: Perturbation-based data repair has been reported not to significantly affect accuracy.
  • Sensitivity Analysis: Sensitivity analysis explores how feature-vector aspects affect outcomes and helps clarify uncertainty related to fairness.It is relatively under-addressed in fairness research.
  • Sensitivity Analysis: Sensitivity analysis is not itself a fairness-improvement method, so it is omitted from the survey’s fairness-method figures.
  • Limitations: Data modification may be legally impermissible, should be minimized, and can affect classifiers differently, requiring continuous reassessment of fairness and methodological choices.
  • Reweighing: Reweighing leaves the data unchanged while assigning instance weights to represent frequencies, emphasize sensitive samples, or improve classifier stability.
  • Reweighing: Reweighing can maintain higher accuracy with appropriate sampling, but may reduce stability and transparency by changing effective data composition.

4.7 Regularization and Constraint Optimisation

Regularization and constraint optimization incorporate fairness considerations into model training, while adversarial learning uses feedback to improve fairness; all face stability and trade-off challenges.

  • Regularization: Fairness regularization adds penalty terms to the classifier loss to penalize discriminatory practices according to selected fairness notions.
  • Regularization: Fair-ML approaches can be unstable, with small training-data changes substantially affecting performance; regularization has been proposed to address this.
  • Constraint Optimisation: Constraint optimization incorporates fairness notions into the classifier loss during training, often operating on the confusion matrix.
  • Challenges: Regularization involves non-convexity or sacrifices probabilistic interpretation, and penalty strength does not affect all fairness measures equally.
  • Challenges: Different penalties produce different accuracy–fairness trade-offs across datasets, while conflicting constraints can make training difficult or unstable.
  • Adversarial Learning: Adversarial learning trains an adversary to detect unfairness in the training process and uses its feedback to improve the model.
  • Adversarial Learning: Adversarial methods can handle multiple fairness constraints and black-box models, but often lack stability and are difficult to train reliably.

4.9 Bandits

Bandit-based fairness methods target online decision-making when decision-makers may recognize unfairness without being able to define fairness formally; the surrounding post-processing literature exposes competing objectives.

  • Bandits: Fairness applications of bandits are nascent, with existing papers providing proofs but lacking general evaluation on specific datasets.
  • Bandits: Bandit approaches are motivated by online decisions and settings where decision-makers may recognize unfairness without defining it formally.
  • Calibration: Calibration requires subgroup prediction rates to match subgroup positive-example rates, but multiple protected groups or fairness criteria can make this impossible.
  • Post-processing: Randomized post-processing can balance accuracy and fairness, but may harm overall accuracy and does not necessarily benefit randomized individuals.
  • Post-processing: Cost-based and multi-calibration approaches have been proposed to address the conflict between calibration, error parity, and individual–group fairness.
  • Thresholding: Thresholding searches classifier probability regions near decision boundaries to classify favored and protected groups positively or negatively.
  • Thresholding: Thresholds may be selected manually or estimated statistically, including with logistic regression, fairness frontiers, or uncertainty-region ensembles.

5 Beyond Binary Classification

Fairness research extends beyond binary classification to regression, rankings, recommender systems, unsupervised learning, transfer learning, and NLP, each requiring domain-specific formulations and methods.

  • Beyond Binary Classification: The survey notes that binary classification dominates fairness research, while other machine-learning problems remain important areas of study.
  • Regression: Fair regression minimizes prediction loss while imposing fairness constraints on continuous actual and predicted values.
  • Regression: Regression work includes impartial parameter estimates, constrained least squares, independence criteria, projection methods, and regularization.
  • Rankings and Recommenders: Ranking research addresses group fairness in top-k results, individual fairness, equality of opportunity, and fairness over time.
  • Rankings and Recommenders: Recommender-system fairness can be multi-sided, concerning both users and items through consumer-oriented C-fairness and producer-oriented P-fairness.
  • Unsupervised Methods: Unsupervised fairness work covers fair clustering, discrimination detection in association-rule mining, and transfer learning.
  • Unsupervised Methods: Fair clustering balances protected-group representation across clusters but carries significant computational costs, addressed partly through coresets and approximate decomposition.
  • Natural Language Processing: NLP exhibits gender- and race-related biases from embeddings, labeling, language use, data preparation, and model architecture; mitigation often occurs during preprocessing.

6 Current Platforms

The paper surveys readily available projects that address fairness in machine learning, emphasizing tools useful for researchers and practitioners.

  • The survey emphasizes readily available tools while noting that proprietary software also exists.
  • Table 4 overviews projects addressing fairness in Machine Learning.

7 Concluding Remarks: The Fairness Dilemmas

The paper frames fairness in machine learning as four interconnected dilemmas involving performance, incompatible fairness notions, sociocultural context and policy, and democratization alongside a skills gap. It calls for better comparative methods, realistic data, stakeholder engagement, and accessible, repeatable frameworks.

  • Dilemma 1: Fairness vs. Model Performance: Fairness interventions can trade off against model performance, and performance measures may themselves harbor ethical concerns.
  • Dilemma 2: (Dis)agreement and Incompatibility of “Fairness”: There is no consensus on prioritizing individual or group fairness because metrics generally cannot combine both.
  • Dilemma 2: (Dis)agreement and Incompatibility of “Fairness”: The community should develop ways to combine incompatible fairness metrics or systematically review their ideological trade-offs and preferences.
  • Dilemma 3: Tensions with Context and Policy: Fairness optimization often lacks transparency about the roots of unfairness and may not reflect contextual, sociocultural, or policy challenges.
  • Dilemma 3: Tensions with Context and Policy: Researchers need more realistic datasets, stronger industry engagement, and proactive debate with policymakers on privacy, policy, and standardization.
  • Dilemma 4: Democratisation of ML vs the Fairness Skills Gap: Democratizing machine learning may increase socially insensitive uses because fairness methods accommodate neither varying technical proficiency nor multi-stage model pipelines.
  • Open methodological challenges: The literature needs diversification beyond supervised binary classification and greater care because fixing one bias can inflate another.
  • Intervention-specific challenges: Pre-processing can conflict with the right to an explanation and introduce subjectivity biases, while in-processing faces integration and porting challenges.
Loading 2010.04053v1…