Source-linked AI summary
On Formalizing Fairness in Prediction with Machine Learning
Pratik Gajane, Mykola Pechenizkiy
TL;DR
Machine-learning prediction increasingly informs critical decisions, creating a need to formalize fairness against discrimination based on protected attributes. This article surveys fairness formalizations and their distributive-justice counterparts, critiques them theoretically and empirically, and proposes two prospective notions addressing some critiques. It concludes that selecting fairness formalizations should be guided by social-science critiques and empirical studies, while recognizing unequal resource access and social conditioning.
Problem
Machine-learning prediction is increasingly used in high-impact domains, raising the need for algorithms that avoid discrimination based on legally protected attributes.
Method
The article surveys machine-learning fairness formalizations, relates them to distributive-justice theories, and examines those theories through theoretical and empirical critiques.
Results
The article proposes two prospective fairness notions and discusses avenues for future formalizations while acknowledging that they do not resolve all critiques.
Takeaways & Limitations
Choosing fairness formalizations for particular domains should draw on social-science critiques and empirical studies of their use in practice.
Takeaways & Limitations
Individual fairness depends substantially on the distance metric, which may itself use protected attributes directly or indirectly and thereby permit discrimination.
Abstract
from arXiv · showhide
Machine learning algorithms for prediction are increasingly being used in critical decisions affecting human lives. Various fairness formalizations, with no firm consensus yet, are employed to prevent such algorithms from systematically discriminating against people based on certain attributes protected by law. The aim of this article is to survey how fairness is formalized in the machine learning literature for the task of prediction and present these formalizations with their corresponding notions of distributive justice from the social sciences literature. We provide theoretical as well as empirical critiques of these notions from the social sciences literature and explain how these critiques limit the suitability of the corresponding fairness formalizations to certain domains. We also suggest two notions of distributive justice which address some of these critiques and discuss avenues for prospective fairness formalizations.
1. Introduction
The article frames fair prediction as avoiding discrimination in increasingly high-impact machine-learning decisions, then surveys fairness formalizations alongside their social-science foundations and critiques.
- Machine-learning predictions increasingly affect credit, employment, education, and criminal-justice decisions prone to discrimination.
- Fairness in prediction aims to design algorithms whose predictions are devoid of discrimination against groups distinguished by legally protected attributes.
- The article surveys machine-learning fairness formalizations and presents their corresponding notions of distributive justice from the social sciences.
- It compares these formalizations through theoretical and empirical critiques to assess their suitability for particular domains and guide newer formulations.
- The prediction setup represents individuals, protected attributes, remaining attributes, outcomes, and possibly randomized mappings from individuals to predicted outcomes.
2. What is fair? (Formalizations of fairness in prediction with machine learning)
The survey organizes fairness formalizations around parity versus preferences and treatment versus impact, then connects each approach to distributive-justice theories and domain-specific critiques. It covers unawareness, counterfactual, group, individual, equality-of-opportunity, and preference-based formulations.
- 2. What is fair? (Formalizations of fairness in prediction with machine learning): Fairness formalizations differ over whether fairness requires parity or preferences, and whether it concerns treatment or impacts.Table 1 summarizes how surveyed approaches answer these questions.
- 2.1. Fairness through unawareness: Fairness through unawareness excludes protected attributes from the prediction process, but background knowledge and unrealistic assumptions can still permit discrimination.Its suitability is therefore challenged when protected attributes are inferable from non-protected attributes or structural barriers are present.
- 2.1. Fairness through unawareness: The corresponding distributive-justice approach is being “blind” to counter discrimination, although documented discrimination and comparative evidence challenge race-blind approaches in several domains.Some studies nevertheless report that a blind approach can work for specific tasks.
- 2.2. Counterfactual measures: Counterfactual fairness requires a predictor’s output to remain unchanged when an individual’s protected attribute is flipped to a counterfactual value.The approach compares each individual with a different version of themselves and is linked to causal-interference tools.
- 2.2. Counterfactual measures: Counterfactual measures may be unsuitable in healthcare and judicial domains because hindsight and outcome biases are frequently observed there, and counterfactual reasoning may affect causal identification.These critiques are presented through the corresponding social-science theory of counterfactual reasoning.
- 2.3. Group fairness (Statistical/demographic parity): Group fairness imposes near-equal outcome probabilities across groups and, unlike some alternatives, does not depend on ground-truth labels.This can be useful where reliable ground truth is unavailable, including employment, housing, credit, and criminal justice.
- 2.3. Group fairness (Statistical/demographic parity): Group fairness corresponds to collectivist egalitarianism and is implemented through affirmative action, while objections about merit and efficiency are weakened by evidence summarized in the survey.The paper notes that deficiencies nevertheless limit its applicability.
- 2.4. Individual fairness: Individual fairness requires similar outputs for similar individuals, but its anti-discrimination power depends largely on whether the distance metric is reliable and non-discriminating.The associated (D, d)-Lipschitz property is also shown to imply statistical parity with a certain bias.
3. Prospective notions of fairness
The paper proposes equality of resources and equality of capability of functioning as prospective fairness notions that address unequal allocation caused by unchosen attributes. These notions differ in whether they compensate for natural and social endowments, while capability theory also faces identification and information challenges.
- Equality of resources: Equality of resources treats unequal social benefits as fair only when they result from individuals’ intentional decisions and actions.It is ambition-sensitive: benefits reflect individuals’ ambitions and choices.
- Equality of resources: Equality of resources is endowment-insensitive, requiring unchosen circumstances to be offset.Unlike equality of opportunity, it does not treat differences in natural endowments such as sex as facts that need not be adjusted.
- Equality of capability of functioning: Equality of capability compensates people for unequal powers to convert opportunities into functionings when protected or personal attributes create difficulties.Functionings are states of being and doing, including activities an individual can undertake.
- Equality of capability of functioning: Equality of capability addresses inequalities from both social endowments such as gender and natural endowments such as sex, unlike equality of resources.Sen’s account extends responsibility limits to personal attributes that affect the development of functionings.
- Critiques: Capability theory is flexible and influential but criticized for failing to identify valuable capabilities and requiring substantial information.The United Nations used this notion in the foundations of the human development paradigm.
4. Discussion and further directions
The article compares machine-learning fairness formalizations with distributive-justice theories and uses social-science critiques and empirical studies to assess their domain suitability. It also proposes two prospective notions that account for unequal resources and social conditioning, while acknowledging that these factors are difficult to formalize.
- Discussion and further directions: The paper juxtaposes fairness notions from machine learning with corresponding theories of distributive justice in the social sciences.It analyzes these notions using theoretical critiques from the social-science literature.
- Discussion and further directions: Theoretical critiques and experimental studies of large-scale use guide the choice of fairness formalizations for particular domains.The paper frames domain suitability as dependent on the implications of different formalizations.
- Discussion and further directions: The paper proposes two prospective fairness notions but does not claim they resolve all critiques of current notions.Their purpose is to initiate discussion about fairness formalizations in prediction with machine learning.
- Discussion and further directions: The proposed discussion recognizes unequal access to resources and social conditioning as social issues relevant to fair prediction.The paper states that these factors are difficult to quantify and formalize mathematically, but their impact should be acknowledged and incorporated where possible.