Source-linked AI summary
The Measure and Mismeasure of Fairness
Sam Corbett-Davies, Johann D. Gaebler, Hamed Nilforoshan, Ravi Shroff, Sharad Goel
TL;DR
Formal fairness criteria are widely used to guide consequential algorithmic decisions, but their policy consequences remain contested. This paper categorizes these criteria and shows that they can produce Pareto-dominated policies, motivating context-specific evaluation of algorithmic fairness.
Problem
Formal fairness criteria have proliferated for consequential algorithmic decisions, creating a need to assess their statistical and policy consequences.
Source: §1 · p. 4, §6 · p. 43Method
The paper surveys and categorizes fairness definitions, then analyzes them mathematically and empirically through college admissions and a consequentialist framework.
Source: §1 · p. 5, §2.1 · p. 6, §4 · p. 23Results
Popular fairness conceptions can produce strongly Pareto-dominated policies, including admissions classes with lower diversity and weaker academic preparation.
Source: §1 · p. 5Takeaways & Limitations
Equitable algorithm design should evaluate context-specific real-world consequences and treat fairness as a policy objective rather than solely an axiomatic criterion.
Source: §1 · p. 5, §6 · p. 43Theorem 11’s conditions do not hold in every setting, including cases where intervening on race shifts test scores by a constant amount.
Source: §3.3.2 · p. 22
Abstract
from arXiv · showhide
The field of fair machine learning aims to ensure that decisions guided by algorithms are equitable. Over the last decade, several formal, mathematical definitions of fairness have gained prominence. Here we first assemble and categorize these definitions into two broad families: (1) those that constrain the effects of decisions on disparities; and (2) those that constrain the effects of legally protected characteristics, like race and gender, on decisions. We then show, analytically and empirically, that both families of definitions typically result in strongly Pareto dominated decision policies. For example, in the case of college admissions, adhering to popular formal conceptions of fairness would simultaneously result in lower student-body diversity and a less academically prepared class, relative to what one could achieve by explicitly tailoring admissions policies to achieve desired outcomes. In this sense, requiring that these fairness definitions hold can, perversely, harm the very groups they were designed to protect. In contrast to axiomatic notions of fairness, we argue that the equitable design of algorithms requires grappling with their context-specific consequences, akin to the equitable design of policy. We conclude by listing several open challenges in fair machine learning and offering strategies to ensure algorithms are better aligned with policy goals.
1. Introduction
The paper categorizes formal fairness definitions into two families and argues, using theory and empirical evidence, that enforcing them can cause more harm than good. It advocates consequentialist algorithm design that treats fairness as a policy objective and confronts substantive trade-offs.
- Formal fairness definitions divide into criteria constraining decision effects on disparities and criteria limiting protected characteristics’ direct or indirect effects on decisions.Examples include equal performance across demographic groups and robustness to race-related causal pathways.Source: §1 · p. 4
- Common fairness criteria almost always produce strongly Pareto dominated policies for natural utility families valuing both academic preparedness and student-body diversity.A policy is strongly Pareto dominated when another feasible policy is preferred under every utility function in the family.Source: §1 · p. 4
- In college admissions, several popular fairness criteria would yield both lower diversity and weaker academic preparation than policies explicitly tailored to desired outcomes.Under one prominent definition, the induced policy admits all applicants with equal probability regardless of qualifications or group membership, potentially harming the groups fairness aims to protect.Source: §1 · p. 5
- Equitable algorithm design must acknowledge and directly address substantive trade-offs because formal fairness axioms are poor tools for balancing competing policy priorities.The authors argue that algorithms encode policy choices and should therefore be tailored to their context and goals.Source: §1 · p. 5
- The paper’s consequentialist framework treats algorithmic fairness as a policy objective, exposing the statistical and normative limitations of axiomatic definitions while developing a positive vision for equitable design.Its contributions include surveying the literature, organizing definitions into a two-part taxonomy, and applying the framework to algorithm design.Source: §1 · p. 5
2. Mathematical Definitions of Fairness
The section develops a taxonomy of fairness definitions that either constrain disparities produced by decisions or constrain protected attributes’ effects on decisions. It grounds these criteria in formal decision settings and distinguishes noncausal, causal, blinding, counterfactual, and path-specific formulations.
2. Mathematical Definitions of Fairness
- Fairness definitions are organized into criteria limiting decisions’ effects on disparities and criteria limiting protected attributes’ effects on decisions.The framework introduces formal notation and uses diabetes screening and college admissions as running examples.Source: §2 · p. 6
2.1 Formal setting
- The formal setting models individuals with covariates X, protected attributes A, randomized binary decisions D under budget b, and binary outcomes Y or potential outcomes Y(0), Y(1).College admissions additionally models decisions affecting graduation outcomes and a limited admission budget, whereas diabetes screening assumes screening can be offered to everyone.Source: §2.1 · pp. 6–7
2.2 Limiting the Effect of Decisions on Disparities
- Decision-disparity criteria include classification parity, which equalizes confusion-matrix measures such as false-positive rates, false-negative rates, precision, recall, positive decisions, or AUC across protected groups.Causal analogues impose conditional-independence requirements involving potential outcomes, protected attributes, and decisions; the critique also applies to approximate rather than exact equality.Source: §2.2 · p. 8
2.3 Limiting the Effect of Attributes on Decisions
- Attribute-effect criteria range from blinding, which excludes protected attributes as explicit inputs, to calibration, counterfactual fairness, and path-specific fairness.Counterfactual fairness requires decisions to remain unchanged in expectation when protected characteristics are counterfactually altered, while path-specific fairness permits effects along selected causal paths but prohibits them along others.Source: §2.3 · pp. 9–11
- Causal attribute-based definitions require explicit assumptions about how protected characteristics are changed or perceived, because causal effects of immutable traits such as race or gender are conceptually difficult to define.Path-specific fairness therefore depends on a causal DAG and distinguishes potentially legitimate pathways from pathways reflecting unequal opportunities or biased tests.Source: §2.3 · pp. 10, 12, Figure 1 · p. 11
3. Equitable Decisions in the Absence of Externalities
This section examines equitable decision-making in settings without externalities, focusing on how utility-maximizing policies relate to parity, fairness-through-unawareness, and Π-fairness constraints. It characterizes when enforcing these criteria can limit the information used in decisions and conflict with optimal decision-making.
3.1 Utility, Risk, and Threshold Rules
- Threshold policies maximize individual and aggregate utility by acting when predicted risk exceeds the utility-derived threshold, without conflict between personal and population welfare.With no budget constraint or externalities, deviating from the optimal threshold only adds unnecessary costs or untreated illness without compensating benefits.Source: §3.1 · pp. 14–15
3.2 The Problem of Inframarginality
- At a 1.5% diabetes-risk threshold, 81% of Asian Americans versus 69% of White Americans would be screened, while equalizing screening would harm members of both groups.Equalizing false positive rates likewise requires screening relatively low-risk people and missing relatively high-risk people; the corresponding rates are 79% and 67%.Source: Figure 2 · p. 16, §3.2 · pp. 16–17
- For almost every collection of group-specific risk distributions, utility-maximizing policies violate demographic parity and equalized false positive rates because classification-parity constraints respond to inframarginal distribution differences.Theorem 9 establishes this incompatibility when the optimal threshold lies strictly between zero and one.Source: §3.2 · pp. 17–18
3.3.1 Blinding
- Blinding can nevertheless have legal, political, and social benefits, so risk tools must balance its decision-making costs against harms from race- or gender-specific assessments.The paper notes concerns about stereotyping, trust, and legal prohibitions on sensitive attributes, alongside possible harms from blinded tools.Source: §3.3 · p. 18, §3.3.1 · pp. 20–21
- Blinding protected attributes can miscalibrate risk estimates and reduce utility when those attributes add predictive value, causing decisions that miss higher-risk people and burden lower-risk people.Theorem 10 states that under positive conditional residual variation and regularity near the optimal threshold, no blind policy is utility-maximizing.Source: §3.3.1 · pp. 19–20
3.3.2 Counterfactual and Path-Specific Fairness
- With W = X, path-specific fairness can force policies to be constant across individuals sharing a reduced covariate value, and in common admissions settings may require equal admission probabilities regardless of preparation or group.Theorem 11 formalizes this restriction under counterfactual-support conditions, yielding d(X) = f(Z).Source: §3.3.2 · pp. 21–22
- Under relatively mild conditions, Theorem 11 implies that any Π-fair admissions policy admits every applicant with equal probability.This applies when, among students with a given test score, a positive fraction attain other test scores under counterfactual race changes, and holds regardless of the admissions budget.Source: §3.3.2 · p. 22
- Π-fairness can instead force admissions to depend only on age, ignoring test score and race, when age is causally unconnected to race.The resulting structural restriction may sometimes be desirable but appears largely unintended by the mathematical formalism.Source: §3.3.2 · p. 22
- Theorem 11 does not apply when race changes test scores by a constant amount or when W ≠ X in some path-specific fairness variants, though path-specific fairness typically remains incompatible with optimal decision-making under Theorem 17.With constant counterfactual effects, multiple-threshold policies can be Π-fair; deterministic counterfactuals are described as exceptional, while the path-specific result is established by Theorem 17.Source: §3.3.2 · p. 22
4. Equitable Decisions in the Presence of Externalities
With competing objectives such as academic preparation and diversity, existing fairness constraints typically exclude policies that improve both dimensions. The formal analysis shows that many constrained policies are strongly Pareto dominated, while less demanding criteria can sometimes—but not always—reach the Pareto frontier.
4.1 The Geometry of Fair Decision Making
- The admissions problem involves an unavoidable trade-off between academic preparation and diversity when the admission budget b < 1, and fairness constraints can harm the groups they aim to protect.The committee prefers more of both objectives, but limited seats prevent simultaneous maximization; the paper therefore treats equitable algorithm design as context-dependent rather than purely axiomatic.Source: §4.1 · pp. 23–24, §4 · p. 23
- In the college-admissions simulation, every fairness-constrained feasible region misses the Pareto frontier, so an unconstrained policy can achieve both greater diversity and a higher academic index.Counterfactual and path-specific fairness restrict policies to constant-probability admissions, while the other definitions also severely restrict the feasible policy space.Source: §4.1 · pp. 24–26
4.2 A Formal Theory of Fairness in the Presence of Externalities
- When utilities consistently rank applicants within groups, every Pareto-efficient decision policy has a group-specific multiple-threshold form.This formalizes the relevant structure under uncertainty about how to trade off academic index and diversity, subject to the admissions budget.Source: §4.2 · pp. 26–27
4.2.1 Fairness definitions with many constraints
- For almost every qualifying distribution, feasible policies satisfying counterfactual equalized odds, conditional principal fairness, or path-specific fairness are strongly Pareto dominated.Theorem 17 relies on U-fineness and consistency conditions; under these assumptions, the result follows because the fairness constraints outnumber the degrees of freedom of multiple-threshold policies.Source: §4.2.1 · pp. 27–28
4.2.2 Fairness definitions with few constraints
- Counterfactual predictive parity, equalized false positive rates, and demographic parity impose fewer constraints and can sometimes lie on the Pareto frontier, but not in all settings.Their single additional constraint can match the degrees of freedom in group-specific multiple-threshold policies; nevertheless, counterfactual predictive parity is Pareto dominated in common college-admissions cases where the target group has lower average graduation rates.Source: §4.2.2 · p. 30
5. A Path Forward
This section advances context-sensitive, outcome-oriented approaches to algorithmic fairness, organized around design principles for navigating trade-offs, calibration, prediction targets, and data collection. It illustrates these principles through medical resource allocation, highlighting considerations involving racial disparities, label bias, and causal treatment effects.
5. A Path Forward
- Formal fairness definitions can harm even intended beneficiaries, motivating consequence-sensitive algorithm design grounded in real-world costs and benefits.The authors argue that automated axiomatic constraints can obscure preferable unconstrained alternatives and produce less diverse, less academically prepared admissions classes.Source: §5 · p. 30, §5.1 · p. 33, §5.2 · p. 33
5.1 Competing Notions of Ethical Decision Making: Process vs. Outcomes
- The paper contrasts process-oriented, deontological fairness with outcome-oriented, consequentialist analysis and favors policies justified by their effects.Formal definitions often focus on protected attributes or error rates without directly addressing whether decisions improve outcomes for affected groups.Source: §5 · p. 30, §5.1 · pp. 32–33
5.2 Designing Equitable Algorithms
- Equitable algorithm design should explicitly weigh trade-offs, assess calibration, select prediction targets, and design representative data-collection strategies.The authors frame this work as context-specific policy design rather than automatic enforcement of axiomatic fairness criteria.Source: §5 · p. 31, §5.2 · p. 33
5.2.1 Contending with inherent trade-offs
- Decision makers should enumerate competing objectives and, when possible, choose policies on the Pareto frontier rather than blindly imposing formal fairness constraints.The admissions example highlights an inherent trade-off between academic preparedness and student-body diversity.Source: §5.2.1 · p. 33
5.2.2 Assessing Calibration
- Calibration is necessary but insufficient: inaccurate or intentionally discriminatory models can remain calibrated while producing inequitable decisions.A noisy diabetes model placed 97% of Asian patients above the 1.5% screening threshold versus 81% under more accurate estimates, despite remaining calibrated.Source: §5.2.2 · pp. 34–36
5.2.3 Selecting the target of prediction
- Prediction targets can encode label bias when observed outcomes poorly proxy the desired outcome or conceal counterfactuals, as in pretrial risk assessment.Arrests and convictions reflect unequal policing, while the desired quantity concerns what would happen if a defendant were released.Source: §5.2.3 · p. 36
5.2.4 Collecting training data
- Training data should represent deployment populations, because historical or imbalanced samples can worsen errors and reinforce existing disparities.Representative sampling is a heuristic rather than a universal solution: its value depends on statistical structure, protected-attribute predictiveness, and the costs of collecting better data.Source: §5.2.4 · pp. 37–38
5.3 Case Study: An Algorithm to Allocate Limited Medical Resources
- The medical-resource case study allocates a 2% enrollment capacity and evaluates a high-cost prediction model using data from approximately 43,000 White and 5,600 Black patients.The initial target is membership in the top decile of annual healthcare expenditures, while the authors note that healthcare cost is vulnerable to label bias.Source: §5.3 · pp. 38–39
- The race-blind healthcare-cost model was calibrated across race groups, and race appeared to provide little marginal predictive power in this example.Its risk estimates largely matched observed costs across groups despite omitting race.Source: §5.3 · p. 39
- In this example, the unconstrained rule enrolled 758 high-cost patients and 205 Black patients, while constraining the algorithm to satisfy formal fairness criteria provided fewer resources and worse care to Black patients.Source: §5.3 · p. 41
- The harm arose because Black patients had lower unconstrained false-negative rates, so equalization raised their enrollment threshold; in a younger female subgroup with higher Black error rates, fairness constraints instead admitted more Black patients.The authors argue that policies should confront these context-dependent trade-offs rather than privilege particular error-rate equalizations.Source: Figure 7 · p. 40, §5.3 · p. 41
- Cost-based targeting exhibited label bias: at similar predicted high-cost risk, Black patients were more likely than White patients to have complex medical needs, motivating prediction targets based on health status rather than cost.Medical cost is a poor proxy for medical need, partly because Black patients face worse healthcare access from socioeconomic factors and discrimination.Source: §5.3 · pp. 41–42, Figure 8 · p. 42
- Estimated treatment effects were only weakly correlated with current chronic-condition counts (r = 0.05), suggesting that prioritizing patients most likely to benefit could yield large health improvements.The proposed causal approach prioritizes estimated treatment effects, ˆY_i(1) − ˆY_i(0), rather than predicted health status alone.Source: §5.3 · pp. 42–43
6. Conclusion · A. Path-specific Counterfactuals
The conclusion argues that fairness should be evaluated through real-world consequences rather than dominant axiomatic criteria, while separating risk estimation from policy design. The appendix formalizes path-specific counterfactuals by computing variables under alternate protected-attribute settings along selected causal paths.
6. Conclusion
- Popular fairness measures have statistical limitations and can harm the groups they are intended to protect when used as algorithmic design principles.This concern motivates a consequentialist alternative to axiomatic fairness.Source: §6 · p. 43
- Fairness debates should focus on consequential quantities such as public safety, incarceration, due process, community development, and loan-program sustainability.The authors recommend grounding technical and policy discussions in real-world effects, assessed through experiments where possible and hypotheses otherwise.Source: §6 · p. 43
- Risk assessment should be decoupled from intervention design because predictions alone do not determine detention, rehabilitation, or appropriate group-specific policy thresholds.Constrained fairness algorithms may sacrifice the best available risk estimates by conflating statistical estimation with policy decisions.Source: §6 · pp. 43–44
- Fair machine learning still requires advances in measurement error, sample bias, externalities, equilibrium effects, and preference aggregation.Carefully designed and evaluated algorithms could nevertheless improve the efficacy and equity of consequential decisions.Source: §6 · p. 44
A. Path-specific Counterfactuals
- Path-specific fairness requires computing path-specific counterfactual values for variables in a directed acyclic graph.Algorithm 1 formally constructs Z_Π,a,a′ for an arbitrary variable or collection of variables.Source: A. Path-specific Counterfactuals · p. 45
- The counterfactual construction first samples exogenous variables, sets A to a, and computes the remaining nodes in topological order using structural equations.This produces the first set of node values under the initial protected-attribute setting.Source: A. Path-specific Counterfactuals · p. 45
- It then sets A to a′ and recomputes each node, using parent values according to whether their connecting paths belong to Π.The resulting values combine the two protected-attribute settings along the specified paths.Source: A. Path-specific Counterfactuals · p. 45
B. Constructing Causally Fair Policies · 24 end · C. A Stylized Example of College Admissions
The paper formulates causally fair policy design as utility maximization under outcome and budget constraints, solving the resulting problems with linear programs and characterizing attainable outcomes. A stylized college-admissions example then specifies a structural causal model, parameter values, and an admissions budget for applying these policies.
B. Constructing Causally Fair Policies
- Utility-maximizing policies under causal fairness, outcome, and budget constraints can be computed efficiently using one linear program or a series of linear programs.The single-LP cases are counterfactual equalized odds, conditional principal fairness, counterfactual fairness, and path-specific fairness; counterfactual predictive parity requires a series of LPs.Source: B. Constructing Causally Fair Policies · p. 45
- For counterfactual equalized odds, conditional principal fairness, path-specific fairness, and counterfactual fairness, the relevant utility-maximizing policy can be constructed with O(n) variables and constraints under finite-support distributions.The first two fairness classes use the distribution of (X, Y (0), Y (1)); path-specific and counterfactual fairness use the distribution of (X, DΠ,A,a).Source: B. Constructing Causally Fair Policies · p. 45
24 end
- Counterfactual predictive parity is handled by constructing a family of linear programs over the unit simplex, with at least one program yielding a utility-maximizing feasible policy.Each program has O(n) variables and constraints when the relevant finite-support distribution is known.Source: B. Constructing Causally Fair Policies · p. 45, §24 · pp. 46, 49
- The feasible outcome regions can be found through convex feasibility problems because the linear-program constraints are convex, followed by a grid search over target outcome points.In the admissions example, the constrained outcomes are the aggregate academic index and the number of admitted applicants from the target group, each within distance ϵ of a given point.Source: §24 · pp. 46–47
- The construction represents each policy by decision variables d_i in [0,1], while utility, budget, and admissions-outcome requirements become linear expressions in those variables.The objective and constraints are weighted by the covariate masses p_i = Pr(X = x_i), including the budget bound E[d(X)] ≤ b.Source: §24 · p. 47
C. A Stylized Example of College Admissions
- The college-admissions example defines endogenous variables through structural equations in a DAG with independently distributed exogenous variables and a logistic outcome model.The exogenous variables include U = {uA, uD, uE, uM, uT, uY}, with UE, UM, and UT normally distributed, and the outcome uses logit−1(βY,0 + m + βY,D · δ).Source: C. A Stylized Example of College… · p. 49
- The stylized admissions model fixes specific structural parameters and imposes an admissions budget of b = 1/4.The listed constants include µA = 1/3, βE,0 = 1, βE,A = −1, βM,0 = 0, βM,E = 1, βT,0 = 50, βT,E = 4, βT,M = 4, βT,u = 7, βT,B = 1, βY,0 = −1, and βY,D = 1/2.Source: C. A Stylized Example of College… · p. 49
D. Proof of Theorem 10
The proof uses a conditional-variance lemma for bounded random variables to show that no threshold event of the specified form is π-measurable. It establishes this by finding positive probability that the conditional threshold indicator is strictly between 0 and 1.
Proof of Theorem 10
- Lemma 22 bounds conditional moments of a non-negative variable X ≤ b, and Jensen’s inequality yields a conditional variance bound.The lemma is interpreted as showing that a bounded random variable’s minimum cannot be too close to its mean.Source: D. Proof of Theorem 10 · p. 50
- The proof reduces non-measurability to showing E[1_{r(X)>t} | π(X)] ∉ {0, 1} with positive probability.Because Pr(r(X)=t)=0, the indicators of r(X) ≥ t and r(X) > t agree almost surely.Source: D. Proof of Theorem 10 · p. 50
- For covariates with ρ(x) ∈ (t, t + ϵ), the proof applies Lemma 22 to r(X) − I(X) and obtains ρ(X) − I(X) > ϵ almost surely.Here I(x) is the essential infimum of the conditional distribution of r(X) given ρ(X)=ρ(x).Source: D. Proof of Theorem 10 · p. 50
- Both Pr(r(X) < t | Xu = xu) and Pr(r(X) > t | Xu = xu) are positive on a positive-probability set, proving the desired conditional indicator is nontrivial.The contradiction uses event (15), the tower law, and conditional expectations with respect to ρ(X).Source: D. Proof of Theorem 10 · p. 50
E. Proof of Proposition 15
The proof establishes that feasible policies outside the class of budget-exhausting multiple threshold policies with non-negative thresholds are strongly Pareto dominated. It also shows threshold representations are utility-invariant and characterizes utility-maximizing policies as budget-exhausting threshold policies.
Threshold invariance
- Under utilities consistent modulo α, whether a policy is a multiple threshold rule is independent of the chosen utility, including non-negative threshold representability.Thresholds transfer across utilities while preserving their signs.Source: E. Proof of Proposition 15 · pp. 51–52
Strengthening Proposition 15
- Any feasible decision policy that is not almost surely a multiple threshold policy with non-negative thresholds is strongly Pareto dominated.The proof improves utility by moving decision mass across a threshold, and handles non-negative-threshold failures by excluding decisions with negative utility.Source: E. Proof of Proposition 15 · pp. 52, 54
Budget exhaustion
- A feasible policy that is not a budget-exhausting multiple threshold policy with non-negative thresholds is strongly Pareto dominated.This follows by combining the strengthened dominance result with the consequences of failing to exhaust the available budget.Source: E. Proof of Proposition 15 · p. 54
Quantile-to-threshold mapping
- For any utility, group-specific quantiles map monotonically to thresholds that define a multiple threshold policy achieving those conditional decision rates.The threshold mapping is monotonically non-increasing in each quantile coordinate.Source: E. Proof of Proposition 15 · p. 55
Utility maximization
- A feasible policy is utility maximizing if and only if it is a budget-exhausting threshold policy, and at least one such maximizing policy exists.Existence follows from the quantile-to-threshold construction, so the maximum is attained.Source: E. Proof of Proposition 15 · p. 55
F. Prevalence and the Proof of Theorem 17 … F.3.1 Background and notation
This section develops the measure-theoretic framework, notation, and regularity conditions for analyzing Theorem 17, prevalence, and relative shyness. It establishes the properties of U-fine distributions and threshold policies needed for the subsequent proof.
F. Prevalence and the Proof of Theorem 17
- Typically, no Pareto-efficient policy satisfies all conditions of counterfactual equalized odds, conditional principal fairness, or counterfactual fairness, because such policies have at most |A| adjustable group thresholds while fairness imposes more constraints.Any coincidental satisfaction is not robust: a minor distribution shift can break the remaining constraints, so the distributions admitting a nondominated fair policy form a “small” set.Source: F. Prevalence and the Proof of Theorem… · p. 56
F.1 Shyness and Prevalence
- Prevalence generalizes full Lebesgue measure to infinite-dimensional spaces through shy complements, while preserving properties such as density, countable-intersection closure, and finite-dimensional agreement with Lebesgue null sets.No measure on an infinite-dimensional separable Banach space can retain Lebesgue measure’s local finiteness, strict positivity, and translation invariance; shy sets provide the relevant generalization (Ott and Yorke (2005)).Source: §F.1 · pp. 56–57
- Relative shyness extends prevalence to convex subsets, and k-dimensional probes detect shy sets by showing every translated intersection is Lebesgue-null in a finite-dimensional subspace.Every k-shy set is shy, providing the probe-based strategy used to prove Theorem 17.Source: §F.1 · pp. 57–58
F.2 Outline
- The proof identifies U-fine distributions admitting a nondominated fair policy as a relatively shy subset E of a convex set Q, then verifies the topological and measurability conditions required for relative prevalence.The outline constructs probes and uses maximal-support measures, prevalence lemmas, generic utility-support properties, and adaptations for the three fairness criteria.Source: §F.2 · pp. 58–59
F.3 Convexity, Complete Metrizability, and Universal Measurability
- The ambient setup treats U-fine probability distributions as a completely metrizable convex subset of a Banach space, while the relevant fair-policy set is closed and therefore universally measurable.This establishes the background requirements needed to apply relative shyness and prevalence to Theorem 17.Source: §F.3 · p. 60, §F.3.1 · p. 61
F.3.1 Background and notation
- The notation defines the state space separately for each fairness criterion, models distributions as totally bounded Borel measures, and uses total variation, conditional expectation, absolute continuity, and pushforwards to formulate U-fineness.Totally bounded measures have finite total variation; absolute continuity is closed under total-variation limits, supporting the construction of the U-fine measure space.Source: §F.3.1 · pp. 60–62
- The collection of U-fine measures on K forms a Banach space.K is defined through absolute continuity of utility pushforward measures, and is closed and complete as a subspace.Source: §F.3.1 · pp. 62–63
- The mappings from measures to utility-density Radon–Nikodym derivatives are continuous, including conditional and path-specific variants.Lemma 48 states continuity for fµ, fµ,a, and the corresponding path-specific density mapping.Source: §F.3.1 · p. 63
- For U-fine probability measures in Q, threshold policies are defined by their thresholds, and policies with matching thresholds agree µ-a.s.The same conclusion holds for conditional group thresholds and for path-specific fairness variants.Source: §F.3.1 · pp. 63–64
- The threshold-identification result relies on the absence of atoms on the utility scale; allowing atoms can produce counterexamples to Theorem 17.This atomless condition is built into the measures in K and is explicitly noted as essential for the observation.Source: §F.3.1 · p. 63
F.3.2 Convexity, complete metrizability, and universal measurability … F.4.2 Shy sets and probes
This section develops topological, measurability, and probe-based tools for genericity arguments involving supports, conditional expectations, overlap utilities, and shy sets. It establishes results showing support preservation for generic elements of Q, shyness of exceptional policy-splitting sets, and criteria relating relative shyness to finite-dimensional measure-zero sections.
F.3.2 Convexity, complete metrizability, and universal measurability
- Q is convex and completely metrizable, while the density set E admitting an undominated fair policy is closed and therefore universally measurable.Convexity follows by direct calculation; completeness follows because Q is closed in the complete space K. Closedness of E is proved by converging threshold policies and controlling conditional expectations across measures.Source: §F.3.2 · pp. 64, 65, 67, 69
F.4 Shy Sets and Probes
- The shy-set and probe subsection develops technical lemmas ensuring arbitrary measures are absolutely continuous relative to typical perturbations and that useful properties hold generically in Q.These constructions simplify the proof of Theorem 17, including cases where conditional expectations must be shown to be generically well-defined.Source: §F.4 · p. 69, §F.4.2 · p. 71
F.4.1 Maximal support
- Measure-theoretic unions yield measures µmax,a and µmax,a,w in Q whose densities have maximal support across measures, with additional positive-probability guarantees under the stated fairness settings.The construction uses countable subcollections and weighted mixtures; maximal support ensures the measures are well-defined and satisfy the relevant positivity conditions.Source: §F.4.1 · pp. 69–71
F.4.2 Shy sets and probes
- For any Borel event assigned positive probability by some ν ∈ Q, the measures in Q assigning that event positive probability form a prevalent set, because the zero-probability set is shy.A finite-dimensional probe direction makes the exceptional parameter set Lebesgue-null, allowing Proposition 38 to establish shyness relative to Q.Source: §F.4.2 · p. 72
- For a fixed ν supported on E, prevalent µ ∈Q satisfy that ν ◦u−1 is absolutely continuous with respect to (µ ↾E) ◦u−1, so generic fµ ↾E supports everywhere fν is supported.The proof establishes Borel universal measurability, constructs a probe W, and shows the exceptional collection is finitely shy.Source: §F.4.2 · pp. 74–75
- Under the condition that every W-stratum admits some µ with positive probability of u(X)>0, almost every µ ∈Q overlaps utilities.Overlap prevents budget-exhausting multiple-threshold policies from placing thresholds exactly between ω-induced strata and thereby splitting utilities.Source: §F.4.2 · pp. 75–76
- The set of measures admitting a budget-exhausting threshold policy that splits utilities is shy, proved by decomposing it into Borel sets EΓ and constructing finite-dimensional probes for each EΓ.The argument shows each EΓ is shy and then takes their union over Γ ⊆W.Source: §F.4.2 · pp. 76–77
- If E is universally measurable in a convex, completely metrizable set C and every translate of E has zero measure along a suitable finite-dimensional subspace V′, then E is shy relative to C.The proof uses translation invariance of λV′ to establish λV′[E −v] = 0 for arbitrary v.Source: §F.4.2 · pp. 77–78
F.5 Proof of Theorem 17
This section proves that, under specified conditions, fairness constraints—including causal, principal, path-specific, and non-causal equalized-odds criteria—almost always exclude strongly Pareto-undominated policies. It does so by constructing distribution-preserving perturbations and extending the strong Pareto-dominance argument across these fairness notions.
Proof setup
- The argument’s setup relies on E being universally measurable and on the preceding lemmas establishing the Banach-space, convexity, and measurability properties needed to apply Prop. 38.These lemmas reduce the remaining task to constructing W and proving the contrasting measure statements for Q and E.Source: §F.5 · p. 78
Construction of the probe
- W combines lower- and upper-support perturbations: WLo breaks balance for the all-positive policy, while WUp breaks balance for policies with at least one non-trivial positive threshold.Perturbations are designed not to change probability status or within-group available budget, while moving mass across outcome values or thresholds.Source: §F.5 · pp. 78–79
Shyness
- The constructed nontrivial probe W satisfies λW[Q − µ] > 0 for some µ ∈ K, while λW[E − µ] = 0, establishing the required measure-theoretic separation.The proof first shows λW[E − µ] = 0 for counterfactual equalized odds and obtains the positive-measure statement from Lemma 62.Source: §F.5 · pp. 80, 83
Case 1
- In the all-positive-policy case, 1u(X)>0 is the unique feasible budget-exhausting multiple-threshold policy, but almost every probe perturbation violates its counterfactual equalized-odds balance condition.The argument derives uniqueness over µ + ν and shows the required equality is impossible for typical perturbations; Fubini’s theorem then yields λW[E − µ] = 0.Source: §F.5 · pp. 81–83
Case 2
- In the non-all-positive case, fixing νLo yields a unique candidate policy, and the set of νUp for which it satisfies counterfactual equalized odds has λWUp-measure zero.Applying Fubini’s theorem first over νUp and then over νLo gives λW[E − µ] = 0 in this case as well.Source: §F.5 · pp. 83, 85
Conditional principal fairness and path-specific fairness
- The proof extends to conditional principal fairness and path-specific fairness by a minor modification of the probe, with otherwise virtually identical arguments.For conditional principal fairness, the proof uses positivity conditional on W and overlap of ω with utility; path-specific fairness uses an analogous probe parameterized by r_a,w.Source: §F.5 · pp. 85–86
- The preceding proof’s stronger perturbation property moves utility mass across any non-trivial threshold, simplifying one case and replacing y0 and y1 with w0 and w1 in the uniqueness argument.The remaining argument uses utility overlap and conditions on w0, which is well-defined for a generic element of Q.Source: §F.5 · p. 87
- The analysis therefore applies beyond causal fairness: policies constrained by non-counterfactual equalized odds are generically strongly Pareto dominated.This extension is obtained immediately from the preceding proof rather than from a separate causal construction.Source: §F.5 · p. 87
- For almost every U-fine distribution of X and Y, any policy satisfying equalized odds is strongly Pareto dominated under the stated utility-support assumptions.The assumptions require utilities consistent modulo α and, for every group, a U-fine X distribution and utility with positive probability of positive utility.Source: §F.5 · p. 87
- The result follows by embedding observed outcomes into counterfactual outcomes and projecting through any fixed policy, whose composition is the identity.Counterfactual equalized odds holds exactly when equalized odds holds after this projection, allowing Theorem 17 to be applied directly.Source: §F.5 · pp. 87–88
F.6 Proof of Theorem 9
The proof shows that both EFP and EDP are shy by perturbing measures along a constructed probe: each fairness condition holds only on a measure-zero subset of perturbation parameters. The argument uses the characterization of utility-maximizing policies and the definitions of EDP and EFP in terms of risk distributions.
- A policy is utility maximizing exactly when d(X) = 1_{R>t} almost surely under any absolutely continuous measure.The absence of an atom at R = t reduces the inequality characterization to equality almost surely.Source: §F.6 · p. 88
- EDP consists of risk distributions with equal conditional and overall probabilities of R > t across every protected group, while EFP and EDP are Borel sets.EDP represents distributions admitting a utility-maximizing policy satisfying demographic parity, and EFP represents distributions satisfying equalized false positive rates when defined.Source: §F.6 · p. 88
- The proof constructs a signed probe contrasting two protected groups, shifting their above-threshold probabilities in opposite directions under perturbation.The probe assigns opposite below-threshold masses to a0 and a1 while leaving their above-threshold masses unchanged, producing terms e0p0 + β and e1p1 − β.Source: §F.6 · pp. 88–89
- Both EFP and EDP are shy; the cited proof shows λW[EDP + µ] = 0 for a measure-zero subset of β ∈ R and concludes that both sets are shy.Source: §F.6 · p. 89
F.7 Proof of Corollary 18
Under Theorem 17’s hypotheses, utility-maximizing policies almost surely fail counterfactual equalized odds, conditional principal fairness, and path-specific fairness. The proof establishes this by showing feasible policy outcomes are convex and utility maximizers are Pareto efficient.
Corollary 18
- The corollary assumes utility is increasing in both coordinates, m(x) ≥ 0, and each protected-group stratum has positive probability with m(X) > 0.Additional conclusions require finite |Img(ω)| and positive probability for all protected-group and mediator combinations, or for two distinct mediator values.Source: §F.7 · p. 90
Proof
- Convex combinations of feasible policies remain feasible because both weighted utility coordinates are linear in the policy, making the attainable set S convex.The proof applies the displayed expectation identities for each coordinate and concludes convexity.Source: §F.7 · p. 90
- Any policy maximizing some utility u(x) ∈ U is Pareto efficient; otherwise, a policy improving another utility would contradict optimality for a suitable intermediate coefficient.The proof then applies this fact to the policy maximizing u∗, establishing its Pareto efficiency before deriving the fairness impossibility.Source: §F.7 · p. 91
Corollary 18
- Under Theorem 17’s hypotheses, almost every joint distribution yields a utility-maximizing policy satisfying none of counterfactual equalized odds, conditional principal fairness, or path-specific fairness.The corollary separately states these failures under the corresponding support and finiteness conditions.Source: §F.7 · pp. 90–91
F.8 General Measures on K · G. Theorem 11 and Related Results · G.1 Extension to Continuous Covariates
These results show that counterfactual equalized odds can remain Pareto-efficient on an open set of non-U-fine distributions, while continuous-covariate path-specific fairness forces decision policies to be essentially constant under suitable regularity conditions. The section also develops a continuous-covariate variant of Theorem 11 and an alternative finite-Markov-chain proof.
F.8 General Measures on K
- For non-U-fine distributions, atoms on the utility scale create additional threshold-policy degrees of freedom, so extending Theorem 17 to all totally bounded measures is false.These degrees of freedom can sometimes enforce causal fairness, including locally robust counterfactual equalized odds.Source: §F.8 · p. 91
- The set of distributions admitting a Pareto-efficient policy satisfying counterfactual equalized odds need not be relatively shy.Proposition 72 establishes this by constructing parameters for which the set contains an open total-variation ball.Source: §F.8 · pp. 92, 94
- A threshold policy can be budget-exhausting, utility-maximizing, and counterfactually equalized-odds compliant throughout a neighborhood of distributions around the constructed example.For any perturbation within total variation distance 1/64, the construction yields a well-defined threshold policy with all three properties.Source: §F.8 · pp. 92, 94
G. Theorem 11 and Related Results
- Theorem 11 is first extended to continuous covariates, then generalized using finite Markov chains with a proof distinct from the main-text sketch.The finite-covariate sketch starts from a point with maximum decision probability, whereas the continuous case requires handling an essential supremum that need not be attained with positive probability.Source: G. Theorem 11 and Related Results · p. 94, §G.1 · p. 95
G.1 Extension to Continuous Covariates
- Lemma 73 states that a decision policy d(x) satisfies path-specific fairness with W = X if and only if any a′ ∈ A.Source: §G.1 · p. 95
- The continuous-covariate theorem assumes conditional distributions that are sufficiently mutually non-singular relative to X, including compatible atom structure.The presentation restricts to trivial ζ, where ζ(x) is identical for all x.Source: §G.1 · p. 96
- Under the stated continuity and mutual non-singularity conditions, any Π-fair policy with W = X is constant almost surely.The proof shows that the essential maximum decision probability must occur almost surely for every protected group.Source: §G.1 · pp. 96–98
G.2 A Markov Chain Perspective · H. Proofs of Propositions 19 and 20 · H.1 Beta distributions and stochastic dominance
The paper recasts Π-fairness through finite-state Markov chains, showing that fair policies are constant on recurrent classes. It then develops beta-distribution stochastic-dominance tools used to prove Propositions 19 and 20.
G.2 A Markov Chain Perspective
- Π-fairness is equivalent to d being a 1-eigenvector of each stochastic matrix Pa′, for every a′ ∈A.This Markov-chain formulation recasts the fairness condition as an eigenvector condition.Source: §G.2 · p. 99
- Any 1-eigenvector of a finite Markov chain is constant on each recurrent class.The values on transient states are determined by eventual absorption probabilities into recurrent classes.Source: §G.2 · pp. 99–100
- Consequently, if P is the average transition matrix across actions, every Π-fair policy is constant on P’s recurrent classes.The result follows because Π-fairness makes the policy a 1-eigenvector of P.Source: §G.2 · p. 100
- When the averaged transition matrix decomposes into irreducible blocks corresponding to ζ strata, each stratum is a recurrent class, yielding Theorem 11.The proof identifies each stratum with a recurrent class of the block-diagonal stochastic matrix.Source: §G.2 · p. 100
H. Proofs of Propositions 19 and 20
- The proofs of Propositions 19 and 20 share preliminary beta-distribution theory, which the paper reviews before proving the propositions.The review introduces incomplete beta functions, truncated beta distributions, and stochastic dominance.Source: H. Proofs of Propositions 19 and 20 · p. 100, §H.1 · pp. 100–101
H.1 Beta distributions and stochastic dominance
- Stochastic dominance implies weakly larger expectations for every monotonically non-decreasing function and yields strict truncated-expectation inequalities under continuity and strict-dominance conditions.For decreasing positive f, the corresponding lower-tail weighted expectation is strictly larger for the stochastically smaller variable under the stated crossing condition.Source: §H.1 · p. 101
- For X ∼Betat(α0, β0) and Y ∼Betat(α1, β1), α0 ≥α1 and β0 ≤β1 imply Y ≤st X, with strict dominance if either inequality is strict.The result characterizes the required stochastic ordering between incomplete beta distributions by their parameters.Source: §H.1 · p. 102
H.2 Proof of Proposition 19
The proof applies Sard’s theorem to show that Pareto-efficient policies satisfying counterfactual equalized odds occupy a measure-zero set of model parameters. It establishes this by proving the relevant constraint map is smooth and has full-rank Jacobian.
H.2 Proof of Proposition 19
- Sard’s theorem implies that parameter settings admitting Pareto-efficient counterfactual equalized odds policies form a measure-zero set.The constraint zero locus is a smooth 3-manifold, while its projection onto the four distribution parameters has measure zero because every projected point is singular.Source: §H.2 · pp. 103–104
- The conditional risk distributions are Beta distributions, providing the analytic form used to define the constraint map.Conditioned on Y (1) = 1, risks follow Beta(αi + 1, βi); conditioned on Y (1) = 0, they follow Beta(αi, βi + 1).Source: §H.2 · p. 103
- Any Pareto-efficient policy satisfying counterfactual equalized odds must be a multiple-threshold policy with thresholds t0 and t1 satisfying budget exhaustion and equalized positive- and negative-outcome rates.These are the three constraints defining the relevant zero locus.Source: §H.2 · pp. 103–104
- The constraint map is smooth and its Jacobian has full rank three, completing the manifold argument.Smoothness follows from differentiability under the integral sign, while rank follows from three linearly independent Jacobian columns, including a sign-pattern argument for the fourth column.Source: §H.2 · pp. 104–106
H.3 Proof of Proposition 20
The proof reparameterizes the beta distributions and applies a generalized lemma showing that, under the stated ordering of beta parameters, every counterfactual-predictive-parity policy is strongly Pareto dominated. Proposition 20 follows because its mean-based assumptions imply those parameter inequalities.
Parameterization
- The proof switches from mean/sample-size parameters (µ, v) to successes/failures parameters (α, β), with µ = α/(α+β) and v = α + β.This reparameterization is used for mathematical convenience before applying the conditional beta-tail-risk characterization.Source: §H.3 · p. 106
Generalized lemma
- Under beta-distributed conditional outcomes, α_a1 < α_a0 and β_a0 < β_a1 imply that every counterfactual-predictive-parity policy is strongly Pareto dominated.This is the generalized lemma used to establish Proposition 20.Source: §H.3 · p. 106
Contradiction
- The contradiction assumes a Pareto-efficient parity policy, reduces it to threshold form, and shows its group-specific untreated conditional expectations cannot be equal as required by counterfactual predictive parity.At λ = 0, the thresholds satisfy t_a0 ≥ t_a1; the resulting equality requirement conflicts with the distributional assumptions and beta-tail lemmas.Source: §H.3 · pp. 106–107
Proposition 20
- Because µ_a0 > µ_a1 implies α_a0 > α_a1 and β_a0 < β_a1 for common sample size v, Lemma 84 establishes Proposition 20.Thus the original mean-based assumptions satisfy the generalized lemma’s parameter ordering.Source: §H.3 · p. 107
Related papers
This paper cites
Cited by
- Data Governance for Generative and Agentic AI : Technology, Law, Economics, and Security2026
- Law of Large Numbers: Accuracy as Statistical Measure for AI Compliance and Competition2026
- VFR-Audit: Verdict-Level Reliability for Fairness Audits in Hospital Length-of-Stay Prediction2026
- Visualization for Human-Centered AI Tools2024
- On the Opportunities and Risks of Foundation Models2021