Source-linked AI summary
Algorithmic Fairness from a Non-ideal Perspective
Sina Fazelpour, Zachary C. Lipton
TL;DR
Algorithmic-fairness research lacks a natural way to formulate justice within prediction, despite the use of parity metrics and algorithms to guide consequential decisions. The paper analyzes these methods through ideal and non-ideal political philosophy, arguing that their shortcomings reflect broader problems with ideal theorizing and can lead to misguided interventions.
Problem
Predictive models do not naturally formulate justice, while fairness metrics and algorithms offer competing ways to assess and mitigate disparities without resolving how decision-makers should act justly.
Source: §1 · pp. 1–2, §3.2 · p. 5Method
The paper connects technical fairness methods to ideal and non-ideal approaches in political philosophy and examines their assumptions, limitations, and consequences.
Source: §1 · p. 2, §3.3 · p. 5Results
Current fairness methods resemble ideal theorizing, and ignoring disparity causes and intervention consequences can produce misguided mitigation that risks amplifying injustice.
Source: §3.3 · p. 5, §4.3 · p. 8Takeaways & Limitations
A non-ideal approach should examine the causal mechanisms behind actual injustices and interventions within the wider human-centered socio-technological system.
Source: §5.3 · p. 10, §6 · p. 11Current algorithmic-fairness approaches are limited to modifying ML algorithms and omit deployment context, decision consequences, and agents’ responsibilities.
Source: §3.3 · p. 5
Abstract
from arXiv · showhide
Inspired by recent breakthroughs in predictive modeling, practitioners in both industry and government have turned to machine learning with hopes of operationalizing predictions to drive automated decisions. Unfortunately, many social desiderata concerning consequential decisions, such as justice or fairness, have no natural formulation within a purely predictive framework. In efforts to mitigate these problems, researchers have proposed a variety of metrics for quantifying deviations from various statistical parities that we might expect to observe in a fair world and offered a variety of algorithms in attempts to satisfy subsets of these parities or to trade off the degree to which they are satisfied against utility. In this paper, we connect this approach to \emph{fair machine learning} to the literature on ideal and non-ideal methodological approaches in political philosophy. The ideal approach requires positing the principles according to which a just world would operate. In the most straightforward application of ideal theory, one supports a proposed policy by arguing that it closes a discrepancy between the real and the perfectly just world. However, by failing to account for the mechanisms by which our non-ideal world arose, the responsibilities of various decision-makers, and the impacts of proposed policies, naive applications of ideal thinking can lead to misguided interventions. In this paper, we demonstrate a connection between the fair machine learning literature and the ideal approach in political philosophy, and argue that the increasingly apparent shortcomings of proposed fair machine learning algorithms reflect broader troubles faced by the ideal approach. We conclude with a critical discussion of the harms of misguided solutions, a reinterpretation of impossibility results, and directions for future research.
1 Introduction
The paper frames algorithmic fairness as a methodological problem: how to give decision-makers normative guidance about just action amid systemic injustice. It connects technical fairness research to ideal and non-ideal approaches in political philosophy and previews critiques of current methods.
1 Introduction
- Machine-learning systems increasingly allocate social benefits and burdens in sensitive domains, raising concerns that they may harm vulnerable communities.The paper notes applications including hiring, social services, and criminal justice [18].Source: §1 · p. 1
- Fairness research formalizes metrics for assessing whether algorithmic allocations are just and modifies data processing, objectives, or learning algorithms to mitigate measured unfairness.Researchers disagree about how fairness should be conceptualized and formalized [30] [5].Source: §1 · pp. 1–2
- The central methodological question is how to provide normative guidance for agents acting amid systemic injustice, given that justice concerns apportioning benefits and burdens according to rights and deserts.Source: §1 · p. 2
- The paper distinguishes ideal from non-ideal approaches, characterizes current technical fairness methods as small-scale ideal theorizing, and identifies shared challenges.It then discusses harms from misguided solutions and offers recommendations for future research.Source: §1 · p. 2
2 Two Methodologies: Ideal vs. Non-Ideal
Ideal theorizing assesses actual practices against a conception of a perfectly just world, but that approach alone can distort diagnosis, provide inadequate mitigation guidance, and obscure responsibility. Non-ideal theorizing starts from actual injustices and evaluates proposed ideals as hypotheses against their practical efficacy.
2 Two Methodologies: Ideal vs. Non-Ideal
- Ideal standards can neglect overlooked injustices and obscure the historical origins and social forces sustaining current ones.They diagnose injustice by measuring deviations from a static ideal target.Source: §2 · p. 2
- An ideal-state discrepancy alone cannot determine what to mitigate: its causes vary, all discrepancies may not be eliminable, and gap-closing policies can worsen injustice.The paper gives race-blind policies opposing affirmative action as an example of this risk.Source: §2 · p. 3
- Ideal theorizing often assumes strict compliance, leaving unclear what real decision-makers should do when others neglect or avoid their responsibilities.Source: §2 · p. 3
- Non-ideal theory begins with actual injustices and affected people’s reasonable complaints, then investigates causes, remedies, and which agents should act.Within this approach, ideals function as revisable hypotheses about solutions, assessed by their efficacy in practice.Source: §2 · pp. 3–4
3 Work on Algorithmic Fairness as Small-scale Ideal Theorizing
The paper characterizes dominant algorithmic-fairness methods as small-scale ideal theorizing: they posit fairness ideals, translate them into metrics, and modify algorithms to reduce measured deviations. These methods focus on the model rather than the broader context, mechanisms, consequences, and responsibilities surrounding its deployment.
3 Work on Algorithmic Fairness as Small-scale Ideal Theorizing
- The paper applies the ideal/non-ideal distinction to algorithmic fairness at the scale of individual decision-makers, characterizing dominant approaches as small-scale ideal theorizing.Source: §3 · p. 4
3.1 Developing a Fairness Ideal
3.2 Specifying a Fairness Metric
- Fairness metrics quantify how far protected groups are from parity, treating measured disparity as divergence from the ideal the metric represents.Source: §3.2 · p. 5
- Metrics require additional value judgments: statistical and causal formulations of an ideal can differ in verdicts, and causal metrics require a faithful, identifiable causal model.Source: §3.2 · p. 5
3.3 Promoting Justice by Minimizing Deviations from the Ideal
- Current methods modify models to maximize utility subject to parity constraints, but omit deployment context, disparity mechanisms, decision consequences, and agents’ responsibilities.Interventions may occur in preprocessing, learning, or postprocessing; their scope remains limited to the ML algorithm.Source: §3.3 · p. 5
4 Troubles with Ideal Fairness Metrics
The paper identifies three recurring problems with treating parity ideals as guides for algorithmic fairness: they can miss or distort injustice, provide inadequate mitigation guidance, and leave decision-maker responsibility unresolved.
4.1 Systematic Neglects of Rights
- Parity-centered fairness ideals can miss non-comparative rights violations and injustices involving relevant comparison groups outside designated protected classes.Treating all defendants equally can still violate rights, while job appraisals may rely on irrelevant characteristics such as height or weight.Source: §4.1 · pp. 6–7
4.2 Distortion of the Harms of Discrimination
- Parity metrics disregard why disparities arise, although distinct historical mechanisms can produce similar representation gaps and call for different interpretations.Demographic membership alone does not capture the societal mechanisms that give it significance in context.Source: §4.2 · p. 7
- Causal fairness methods explain how protected traits affect predictions but do not establish whose discrimination a disparity reflects or when decision-makers should intervene.They also lack an account of how a proposed intervention may affect the social system.Source: §4.2 · p. 7
4.3 Insufficient Insights and Misguided Mitigation
- A DLP can achieve demographic parity by favoring applicants associated with women-dominated fields, while penalizing women in fields that most need greater diversity.The authors warn that such mitigation can reinforce stereotypes and risk amplifying the injustices it is intended to address.Source: §4.3 · p. 8
4.4 Lack of Practical Guidance
- Statistical disparity measures leave unanswered who discriminated and what responsibility belongs to the current decision-maker.When education influences employment, confusion matrices do not resolve how responsibility should be divided between the education system and employer.Source: §4.4 · p. 8
5 Discussion
The discussion argues that parity-based fair ML can offer an appealing appearance of fairness without practical guidance on justice. It reframes impossibility results and calls for fact-sensitive, system-wide evaluation of interventions.
5.1 If not Solutions, then Solutionism?
- Parity metrics can serve as a marketplace-facing certificate of fairness, while loose definitions and blind optimization risk leaving problems unaddressed or causing harm.The authors note that these metrics can still be useful diagnostics that flag disparities for further investigation.Source: §5.1 · p. 9
5.2 Re-interpreting Impossibility Results
- Impossibility results show that a decision-maker cannot eliminate every disparity through predictions alone, and may reduce one gap only at the expense of widening another.The paper interprets these results as evidence that an ideal approach needs a basis for choosing which competing discrepancy to address.Source: §5.2 · pp. 9–10
- Fairness metrics can describe disparities and trade-offs, but cannot determine which metric should be optimized in a given situation.The paper frames this prioritization as the pressing question for deciding how to act justly.Source: §5.2 · p. 10
Empirical understanding of the problem:
- Non-ideal prescriptions require fact-sensitive accounts of the causal mechanisms that produce present injustices and shape proposed interventions’ impacts.Building such accounts requires domain knowledge and value judgments about which abstractions and assumptions are appropriate.Source: §5.3 · p. 10, Empirical understanding of the problem · p. 10
Empirically-informed choice of treatment:
- Current fair-ML evaluations are typically local and static, whereas non-ideal guidance should assess interventions’ situated, system-wide, and longer-term impacts.The paper notes that some dynamic evaluations have found proposed mitigation strategies counterindicated.Source: Empirically-informed choice of treatment · pp. 10–11
- Choosing how to intervene requires a coherent framework for the decision-maker’s role and human judgment about social context, beyond the reductive variables used by most statistical fair ML.The relevant value judgments cannot be made from the model’s statistical description alone.Source: Empirically-informed choice of treatment · p. 11
6 Conclusion
The conclusion calls for moving beyond parity-constrained predictive models to consider the broader socio-technological system. It argues that this human-centered perspective can support novel mitigation strategies.
6 Conclusion
- Addressing algorithmic harms requires considering human users, decision contexts, and aims beyond parity-constrained predictive models, potentially enabling mitigation strategies beyond algorithm modifications.Source: §6 · p. 11