Source-linked AI summary
The Frontiers of Fairness in Machine Learning
Alexandra Chouldechova, Aaron Roth
TL;DR
Algorithmic fairness research is expanding as machine learning increasingly informs consequential decisions, but its fundamental science remains nascent. This report synthesizes a 2018 expert workshop’s assessment of theoretical foundations, unanswered questions, and promising research directions.
Problem
Machine learning systems can encode existing biases or introduce new ones, while fairness theory remains limited for complex, dynamically changing systems.
Method
A 2018 workshop of about fifty experts assessed the fundamentals of fairness in machine learning and synthesized theoretical research directions.
Results
The surveyed theory identifies unresolved challenges involving fairness definitions, biased data, representation learning, system composition, and sequential decision-making.
Takeaways & Limitations
Progress requires deeper theoretical understanding of fairness in complex systems and continued work connecting formal definitions with practical settings.
Takeaways & Limitations
Fairness corrections require assumptions about unobservable constructs when the data-biasing mechanism cannot be inferred empirically.
Abstract
from arXiv · showhide
The last few years have seen an explosion of academic and popular interest in algorithmic fairness. Despite this interest and the volume and velocity of work that has been produced recently, the fundamental science of fairness in machine learning is still in a nascent state. In March 2018, we convened a group of experts as part of a CCC visioning workshop to assess the state of the field, and distill the most promising research directions going forward. This report summarizes the findings of that workshop. Along the way, it surveys recent theoretical work in the field and points towards promising directions for research.
1 Introduction
Machine learning’s expanding use in consequential decisions has heightened concern that data-driven systems can encode or introduce discrimination. Although fairness research has rapidly grown into a major subfield, foundational understanding remains nascent, motivating an expert workshop focused on theoretical research directions.
- 1 Introduction: Machine learning now informs consequential decisions such as loan screening, policing, and bail or parole, increasing concern about discriminatory and unfair outcomes.Empirical findings show that data-driven methods can encode existing human biases and introduce new ones.
- 1 Introduction: Fairness and transparency research has rapidly expanded from a niche topic into a major machine-learning subfield with a dedicated archival conference.
- 1 Introduction: Despite the field’s publication volume and speed, fundamental questions about fairness in machine learning remain insufficiently understood.
- 1 Introduction: A March 2018 workshop convened about fifty experts from academia, industry, and government to assess foundational understanding and identify pressing unanswered questions.
- 1 Introduction: The report emphasizes theoretical work that could provide a scientific foundation for understanding algorithmic bias rather than comprehensively covering the largely empirical field.
2 What We Know
The field has identified recurring sources of unfairness and developed statistical and individual fairness definitions, but each approach has important limitations. Data can encode prior bias, average-error objectives can disadvantage minorities, exploration creates ethical tensions, and individual guarantees require difficult assumptions.
- 2 What We Know: Off-the-shelf machine-learning systems can produce intuitive unfairness through biased data, majority-focused error minimization, and the need to explore.
- 2.1 Causes of Unfairness: Arrest records used for recidivism prediction may reflect unequal policing rather than crime occurrence, especially for drug crimes and minority populations.
- 2.1 Causes of Unfairness: Group-blind classifiers minimizing overall error can fit majority populations better than minority populations, producing higher error distributions for minorities.
- 2.1 Causes of Unfairness: Exploration in recidivism prediction and drug trials requires actions believed to be sub-optimal to gather counterfactual data, raising ethical questions about individual harm and learning.
- 2.2.1 Statistical Definitions of Fairness: Statistical fairness definitions seek approximate parity of measures such as positive rates or error rates across protected groups, offering simple, assumption-light, verifiable constraints.
- 2.2.2 Individual Definitions of Fairness: Individual fairness definitions constrain treatment of specific pairs, but practical use remains unclear because they require assumptions such as agreed similarity metrics or functional relationships between features and labels.
- 2.2.2 Individual Definitions of Fairness: Causal notions of fairness were identified as important but intentionally excluded because they were reserved for a separate CCC visioning workshop.
3 Questions at the Research Frontier
Research at the fairness frontier addresses how to define meaningful yet implementable guarantees, correct biased data, and understand fairness in dynamic, multi-component systems. Existing results reveal important risks—including recoverable representations, feedback effects, poor composition, and brittle long-term predictions—while identifying directions for broader theory.
- Between Statistical and Individual Fairness: Fairness definitions seek a balance between implementability and meaningful individual guarantees, with group-based approaches extended to rich classes of intersecting groups.These approaches retain statistical fairness’s limited assumptions while better addressing intersectionality, but raise questions about reasonable function classes and protected features.
- Data Evolution and Dynamics of Fairness: Fairness research must move beyond one-shot classification to systems whose components interact and whose environments change over time.The report highlights composition, feedback, and socio-technical dynamics as central unresolved aspects of fairness.
- Data Evolution and Dynamics of Fairness: Fair components often compose poorly, while models of long-term fairness remain brittle because their predictions depend on specific modeling assumptions.Special cases such as filtering pipelines admit well-behaved multiplicative approximations, but broader guidance remains limited.
- Modeling and Correcting Bias in the Data: Fairness correction depends on assumptions about how observed data relates to an unobservable construct space, and these assumptions may be difficult to infer empirically.Correction can visibly reduce prediction accuracy, while its benefits depend on judgments about the world that should be represented.
- Fair Representations: Fair representations can remove sensitive-attribute information while retaining task-relevant information, but stronger downstream models may recover the protected information.The effectiveness of this approach therefore depends critically on how thoroughly associations with protected characteristics are removed.
- Beyond Classification: In bandit settings, exploration can sacrifice an individual’s welfare, and its costs may fall disproportionately on structured sub-populations.Some results identify conditions allowing linear learners to avoid exploration, while other work finds that adding a majority population can increase subgroup exploration costs.