Source-linked AI summary
A Moral Framework for Understanding of Fair ML through Economic Models of Equality of Opportunity
Hoda Heidari, Michele Loi, Krishna P. Gummadi, Andreas Krause
TL;DR
Algorithmic fairness definitions can be understood through economic models of equality of opportunity, which distinguish accountability features from arbitrary ones. The paper maps existing definitions to those models and introduces a broader family of fairness measures. It reports that using a measure whose moral assumptions are unmet can harm disadvantaged-group welfare.
Problem
The paper asks how recently proposed algorithmic-fairness notions relate to equality of opportunity, given that automated decisions increasingly affect domains where that fairness ideal applies.
Source: §1 · p. 2Method
The authors map predictive models to policies, irrelevant features to circumstances, and effort-based utilities to effort, then propose Roemer-inspired measures for supervised learning beyond binary classification.
Source: §1 · pp. 2–3Results
Predictive value parity and equality of odds can be interpreted as special cases of equality of opportunity, and using a measure with unmet moral assumptions can harm disadvantaged-group welfare.
Source: §1 · pp. 2–3Takeaways & Limitations
Because each fairness notion carries distinct moral assumptions, practitioners should choose one suited to the specific application context.
Source: §1 · p. 3The framework assumes the partition between morally accountable and arbitrary features has already been identified, leaving that determination to domain experts and stakeholders.
Source: §1 · p. 2
Abstract
from arXiv · showhide
We map the recently proposed notions of algorithmic fairness to economic models of Equality of opportunity (EOP)---an extensively studied ideal of fairness in political philosophy. We formally show that through our conceptual mapping, many existing definition of algorithmic fairness, such as predictive value parity and equality of odds, can be interpreted as special cases of EOP. In this respect, our work serves as a unifying moral framework for understanding existing notions of algorithmic fairness. Most importantly, this framework allows us to explicitly spell out the moral assumptions underlying each notion of fairness, and interpret recent fairness impossibility results in a new light. Last but not least and inspired by luck egalitarian models of EOP, we propose a new family of measures for algorithmic fairness. We illustrate our proposal empirically and show that employing a measure of algorithmic (un)fairness when its underlying moral assumptions are not satisfied, can have devastating consequences for the disadvantaged group's welfare.
1 Introduction
The paper connects algorithmic fairness to equality of opportunity (EOP), a framework that distinguishes morally accountable effort from arbitrary circumstances. It maps fairness definitions to different EOP assumptions and proposes a new fairness-measure family for broader supervised-learning tasks.
1.1 Equality of Opportunity: An Overview
- EOP seeks a level playing field before people compete for positions, allowing outcome inequalities that reflect merit or deservingness.Its distinction between morally acceptable and arbitrary sources of inequality is central to the framework.Source: §1 · p. 1
1 Introduction
- The proposed mapping treats predictive models as policies, irrelevant features as circumstances, and effort-based utilities as effort, interpreting predictive value parity and equality of odds as EOP cases.Equality of odds corresponds to Rawlsian EOP under the assumption that people with the same true label have equal effort-based utility; predictive value parity corresponds to luck egalitarian EOP if predicted label or risk reflects that utility.Source: §1 · p. 2
- Inspired by Roemer’s egalitarian EOP, the authors propose fairness measures for supervised-learning tasks beyond binary classification and report that using a measure with unmet moral assumptions can harm disadvantaged-group welfare.They illustrate the proposal on a real-world regression dataset and compare it with existing regression fairness notions.Source: §1 · p. 3
- The framework makes each fairness notion’s moral assumptions explicit, rather than advocating one universal definition, and guides practitioners toward context-specific choices.The partition between accountable and arbitrary features is assumed to be identified in advance, with its resolution left to domain experts and stakeholders.Source: §1 · pp. 2–3
1.1 Equality of Opportunity: An Overview
- The paper focuses on substantive EOP, especially Rawlsian and luck egalitarian refinements, rather than libertarian or formal EOP.Formal EOP bars direct discrimination by arbitrary features but can permit those circumstances to affect prospects indirectly; substantive EOP also concerns fair access to qualifications.Source: §1.1 · pp. 3–4
- Rawlsian EOP equalizes utility distributions across circumstances at fixed effort, whereas luck egalitarian EOP compares people by their rank within their circumstance-specific effort distributions.The paper’s economic setup treats circumstance as morally irrelevant and effort as encompassing factors for which a person can be held accountable.Source: §1 · pp. 1–2, §1.1 · p. 4
2 Setting
The setting treats a predictive model as a salary policy, with observable features divided into morally irrelevant and accountable attributes. It defines utility advantage as the gap between actual and effort-based utility, then uses that quantity to frame equality-of-opportunity fairness.
2 Setting
- The running example uses an automated model to predict salary intended to improve employee performance, although that target need not match the salary an employee is morally qualified for.Source: §2 · p. 5
- The learner fits h from labeled training examples to predict labels for new individuals.Source: §2 · p. 5
- Observable features x are split into z, morally irrelevant attributes such as race or zip code, and w, accountable attributes such as education and experience.Source: §2 · p. 5
- Effort-based utility d is due solely to accountability factors and is estimated from x, y, and h by g; actual utility a is modeled from those inputs by f.Source: §2 · p. 5
- Predictive advantage u—the discrepancy between actual and effort-based utility—is the quantity to equalize among people alike in accountable respects, regardless of irrelevant attributes.For simplicity, higher values of both a and d are treated as more desirable.Source: §2 · p. 6
- The paper maps predictive model h to a policy, arbitrary features z to circumstance, and effort-based utility d to effort.Source: §2 · p. 6
3 EOP for Supervised Learning
Section 3 connects established algorithmic fairness criteria to economic models of equality of opportunity. It shows that these criteria rest on different assumptions about which factors make people accountable for outcomes.
3 EOP for Supervised Learning
- The section casts statistical parity, equality of odds, equality of accuracy, and predictive value parity as special cases of economic equality of opportunity.The mapping is used to make each criterion’s moral assumptions explicit.Source: §3 · p. 6
3.1 Statistical Parity, Equality of Odds and Accuracy as Rawlsian EOP
- Equality of odds is Rawlsian EOP when a person’s true label represents their effort-based utility, an assumption the authors say must be checked in context.True labels may reflect the current state of affairs, which may itself be tainted by past injustice.Source: §3.1 · pp. 7–8
- Statistical parity is Rawlsian EOP when everyone is assumed to have the same effort-based utility.The equivalence is stated for binary classification.Source: §3.1 · p. 8
- Equality of accuracy is Rawlsian EOP when everyone’s effort-based utility is assumed to be zero, but this treats beneficial and harmful errors alike.The authors illustrate the distinction with salaries above versus below what a person deserves.Source: §3.1 · p. 9
3.2 Predictive Value Parity as Egalitarian EOP
- Egalitarian EOP compares people at the same quantile of their type’s effort-based utility distribution and equalizes their indirect utility distributions across types.The comparison is made at fixed effort quantiles.Source: §3.2 · p. 10
- Predictive value parity is egalitarian EOP when predicted labels represent effort-based utility and true labels represent actual utility; it is not Rawlsian EOP because effort then depends on the model.The authors note that both assumptions require contextual evaluation.Source: §3.2 · pp. 9–10
- Recent incompatibility results can be read as conflicts between criteria that assume different bases for accountability.Equality of odds uses true labels, while predictive value parity uses predicted labels; unless Y = h(X), satisfying both is often unnecessary on moral grounds.Source: §3.2 · p. 11
4 Egalitarian Measures of Fairness
The paper proposes egalitarian measures of algorithmic fairness inspired by Equality of Opportunity, then illustrates how the choice of fairness measure can affect disadvantaged-group welfare. Its approach extends beyond binary classification by balancing utility across groups and effort ranks.
4 Egalitarian Measures of Fairness
- The new family of egalitarian measures applies beyond binary classification and utility functions beyond the paper’s simple linear form.The proposal is inspired by Roemer’s egalitarian model of Equality of Opportunity.Source: §4 · p. 11, §4.1 · p. 13
4.2 Illustration
- The proposed method improves the disadvantaged group’s average utility as the error bound ϵ rises, while the regression baseline quickly drives residual differences to zero but performs very poorly on that utility.The figure compares the proposal and baseline using residual differences and disadvantaged-group utility as ϵ, the upper bound on mean squared error, varies.Source: Figure 2 · p. 12, §4.2 · p. 14
4.1 A New Family of Measures
- Because no predictive model is generally optimal at every effort rank, the paper defines an e-EOP model as a compromise that maximizes expected utility for the worst-off group across ranks.The method evaluates the measure on the data using an in-sample analogue; its utility calculation uses group membership and individual utility under the deployed model.Source: §4.1 · p. 12
4.2 Illustration
- The illustration uses a linear regression model on 1,994 U.S. communities to predict violent crime, with illustrative group-specific utility assumptions that the authors say reflect no deep knowledge of neighborhood well-being.The proposed in-processing method maximizes the worst-off group’s expected utility subject to an error upper bound; it is convex under the stated convex-loss and concave-objective conditions.Source: §4.1 · p. 12, §4.2 · p. 13
5 Conclusion
The paper offers a unifying moral framework for understanding algorithmic fairness through philosophical interpretations and economic models of Equality of Opportunity. It maps fairness notions to moral assumptions, while emphasizing that selecting among them depends on the application context and appropriate stakeholder input.
5 Conclusion
- Statistical parity, equality of odds, and predictive value parity map to specific assumptions about what decision subjects morally deserve.The paper does not advocate for one fairness notion; it argues that each is suitable only in certain application domains.Source: §5 · pp. 14–15
- Determining which features count as morally accountable effort calls for stakeholder and domain-expert input, and reasonable people may disagree about the suitable fairness notion.The authors caution that disagreement does not make all existing notions equally acceptable in a given context.Source: §5 · p. 15