PaperlayerView exact revision on arXiv

Source-linked AI summary

Delayed Impact of Fair Machine Learning

Lydia T. Liu, Sarah Dean, Esther Rolf, Max Simchowitz, Moritz Hardt

arXiv:1803.04383v2cs.LGstat.ML

TL;DR

The paper addresses the gap between static fairness criteria and the delayed effects of decisions on population well-being. It introduces a one-step feedback model and characterizes when standard criteria improve, stagnate, or harm group outcomes. The results support temporal and measurement-aware evaluation and motivate direct outcome-based alternatives when an accurate outcome model is available.

  • Problem

    Fairness research often evaluates static classification decisions without modeling how they reshape populations over time or rigorously establishing long-term benefits for disadvantaged groups.

  • Method

    The paper models group score distributions, selection policies, expected score changes, and institution utility under unconstrained, equal-selection-rate, and equal-true-positive-rate policies.

  • Results

    Equal selection rates and equal true positive rates can produce improvement, stagnation, or decline, while unconstrained optimal selection cannot cause decline under a mild assumption.

  • Takeaways & Limitations

    Evaluating fairness criteria requires careful temporal and measurement modeling, and accurate outcome models can support more direct optimization of positive outcomes.

  • Takeaways & Limitations

    The analysis focuses on a single epoch, and realistic applications may require richer outcome models and additional domain knowledge.

Abstract

from arXiv · show

Fairness in machine learning has predominantly been studied in static classification settings without concern for how decisions change the underlying population over time. Conventional wisdom suggests that fairness criteria promote the long-term well-being of those groups they aim to protect. We study how static fairness criteria interact with temporal indicators of well-being, such as long-term improvement, stagnation, and decline in a variable of interest. We demonstrate that even in a one-step feedback model, common fairness criteria in general do not promote improvement over time, and may in fact cause harm in cases where an unconstrained objective would not. We completely characterize the delayed impact of three standard criteria, contrasting the regimes in which these exhibit qualitatively different behavior. In addition, we find that a natural form of measurement error broadens the regime in which fairness criteria perform favorably. Our results highlight the importance of measurement and temporal modeling in the evaluation of fairness criteria, suggesting a range of new challenges and trade-offs.

1 Introduction

The paper studies fairness in machine learning when decisions reshape populations over time, using a one-step feedback model to evaluate long-term group outcomes. It finds that standard fairness criteria can produce improvement, stagnation, or decline, while measurement error and direct outcome optimization alter the comparison.

  • Static fairness criteria are often justified as protecting disadvantaged groups, but rigorous evidence about their long-term effects is limited.
  • The proposed model represents groups by score distributions, selection policies, and expected score changes after decisions, with utility optimized under alternative constraints.
  • Equal selection rates and equal true positive rates can each yield improvement, stagnation, or decline in natural parameter regimes.The paper completely characterizes the regimes associated with each outcome.
  • Equal selection rates can cause decline in settings where equal true positive rates do not, while optimal unconstrained selection cannot cause decline under a mild assumption.
  • Measurement error narrows the regime in which fairness criteria cause decline, making measurement relevant to comparisons with unconstrained selection.
  • The paper argues that careful temporal and outcome modeling enables more direct optimization of positive outcomes than existing fairness constraints, though the analysis focuses on a single epoch.The proposed outcome-based alternative may reduce institutional profit while maximizing improvement in the protected group’s average credit score.

2 Problem Setting

The paper models group-dependent selection as utility maximization under fairness constraints, while tracking how selection changes group well-being over time. Its framework uses score distributions, selection policies, outcome curves, and fairness criteria to characterize improvement, stagnation, harm, and relative harm.

  • Model and objectives: The institution chooses selection policies for two score-distributed groups while maximizing expected utility, optionally subject to fairness constraints.Policies assign selection probabilities by group and score; the unconstrained policy focuses solely on utility.
  • Model and objectives: The framework measures a policy’s average change in a group’s mean score using the expected score change ∆(x) for selected individuals.The outcome measure can also represent abstract well-being and can incorporate mean-preserving external factors.
  • Applications and assumptions: The running credit-scoring example models lending decisions through group-specific thresholds, repayment probabilities, bank utility, and changes in individuals’ financial scores.Advertising and college admissions provide additional applications, although binary decisions and success probabilities may not fully capture real-world phenomena.
  • Outcome regimes: Outcome regimes are defined by the change in average group score: improvement when ∆µj > 0, stagnation when ∆µj = 0, and active harm when ∆µj < 0.Relative harm compares a policy’s outcome with the outcome under the MaxUtil strategy.
  • Outcome curve: The outcome curve maps a group’s selection rate to its mean-score change and partitions policies into active-harm, relative-harm, and no-harm regions.For threshold policies, the curve is concave and can characterize regimes using selection-rate boundaries such as βMaxUtil, β0, and β∗.
  • Fairness criteria: The paper compares MaxUtil, demographic parity, and equal opportunity, which respectively impose no constraint, equal selection rates, and equal true positive rates across groups.Equal opportunity constrains the conditional selection probability given success to be group-independent.

3 Results

The paper characterizes when fairness constraints improve, stagnate, or harm a disadvantaged group relative to unconstrained utility maximization. Outcomes depend on distribution geometry, population proportions, and the mismatch between institutional utility and score change.

  • General outcome regimes: 0 ≤ ∆µMaxUtil ≤ ∆µ∗ under the institution-utility assumption, so unconstrained utility maximization does not cause active harm.The assumption requires u(x) > 0 to imply ∆(x) > 0.
  • Fairness can improve outcomes: There are population-proportion intervals where DemParity or EqOpt selects below MaxUtil’s protected-group rate, yielding relative improvement.The sufficient conditions differ between DemParity and EqOpt and depend on acceptance-rate comparisons involving G(A→B).
  • Fairness can cause harm: For sufficiently small protected-group proportions, DemParity and EqOpt can be over-eager, selecting above a relevant benchmark and causing active or relative harm.In the lending example, excessive lending can worsen the protected group’s average credit score.
  • Comparing EqOpt and DemParity: Neither fairness criterion is uniformly better: comparing DemParity and EqOpt requires the full distributions πA and πB to compute G(A→B).The analysis identifies settings where each criterion improves while the other causes harm.
  • Comparing EqOpt and DemParity: When translated group distributions satisfy the stated conditions, EqOpt can avoid active harm while DemParity causes it; EqOpt is more conservative in that regime.At β = β0, the interval [g1, g2] supports harm under DemParity and improvement under EqOpt.

4 Relaxations of Constrained Fairness

The paper studies soft fairness constraints and systematic score underestimation as relaxations of hard constrained decision-making. Regularization interpolates between unconstrained and hard-constrained policies, while underestimation broadens favorable fairness regimes.

  • Regularized fairness: Soft fairness objectives penalize disparities in acceptance rates or true positive rates instead of enforcing exact equality.Soft-DemParity is represented by a convex penalty on the acceptance-rate difference.
  • Regularized fairness: As λ increases from 0 toward infinity, regularized solutions interpolate between MaxUtil policies and hard DemParity or EqOpt policies.The interpolation is established for convex regularization functions, including Φ(t) = |t|.
  • Measurement error: Systematic score underestimation causes underselection relative to decisions based on the true distribution.The result applies to MaxUtil and, under an additional true-TPR condition, EqOpt.
  • Measurement error: Because fairness criteria encourage higher selection for disadvantaged groups, systematic underestimation widens the regime in which they can yield favorable outcomes.Underestimation also lowers estimated MaxUtil selection, enlarging the relative-improvement region.
  • Outcome-based decision-making: An outcome-based alternative directly maximizes the protected group’s expected score change subject to a limited profit loss.Its optimal policy selects at β = min{β∗, βmax}.
  • Outcome-based decision-making: Outcome-based optimization shifts attention to modeling score changes and requires domain-specific knowledge; robustness to outcome-model errors remains future work.The paper presents approximate implementation as feasible when outcomes can be predicted reasonably well.

5 Optimality of Threshold Policies

The paper reduces policy analysis to threshold policies indexed by group-wise selection rates. Under monotonicity assumptions, threshold transformations preserve selection rates while improving both institutional utility and individual outcomes, and the resulting outcome curves are concave.

  • Threshold policy structure: A threshold policy is a randomized score cutoff: select above c, reject below c, and randomize with γ at c.This representation handles discrete score supports.
  • Threshold policy structure: For a fixed selection rate, the threshold policy is essentially unique and maximizes both institutional utility and group utility.Policies differing only on zero-mass scores are treated as equivalent.
  • Threshold optimality: Replacing any policy with a same-rate threshold policy never reduces institutional utility or individual outcomes when u(x) and ∆(x) increase with score.Equality holds exactly when the original policy is equivalent under the group distribution.
  • Threshold optimality: Optimal MaxUtil and DemParity policies are threshold policies, and optimal EqOpt policies are threshold policies when u(x)/ρ(x) is increasing.The EqOpt result requires the additional ratio-monotonicity assumption.
  • Quantiles and concavity: For discrete score distributions, selection-rate mappings and outcome curves are piecewise linear, with left and right derivatives used to characterize concavity.The outcome curve β 7→ ∆µ(r−1π(β)) is concave when ∆(x) is monotone.
  • Quantiles and concavity: Figure 3 represents DemParity as a slope-1 line and EqOpt as the curve G(A→B), with derivatives taken along βA.Projecting these constraint curves onto βA yields concave utility curves.

6 Proofs of Main Theorems

The proofs characterize fairness-constrained selection rates by reducing each optimization to a single parameter and exploiting concavity. DemParity uses a common acceptance rate, while EqOpt uses a common weighted true positive rate and the transfer function G(A→B).

  • DemParity: DemParity imposes equal acceptance rates, reducing the optimization to a single common rate β across groups.For fixed β, utility-maximizing group policies can be chosen as threshold policies.
  • DemParity: The optimal DemParity selection rates form a continuous interval characterized by first-order conditions on a concave objective.Positive right derivatives indicate rates below the optimum, while negative left derivatives indicate rates above it.
  • EqOpt: EqOpt imposes equality of weighted true positive rates, parameterizing feasible policies by a common TPR t.The largest feasible TPR is tmax = minj∈{A,B}{⟨πj, wj⟩}.
  • EqOpt: When w(x) > 0, the TPR-to-acceptance mapping is bijective, so each feasible TPR uniquely determines group selection rates.This permits translating an optimal interval in t into an interval of acceptance rates.
  • EqOpt: The optimal EqOpt selection rates form a continuous interval obtained from a concave one-dimensional objective and the transfer function G(A→B).G(A→B) maps group A’s acceptance rate to group B’s rate satisfying the EqOpt constraint.

7 Simulations

The simulations apply the framework to FICO credit scores and compare MaxUtil, DemParity, and EqOpt under different bank utility settings. They show that fairness criteria can produce active harm or different degrees of alignment with optimal lending, depending on the setting.

  • Empirical setup: FICO TransUnion TransRisk scores provide an empirical application of the model to two race groups.The dataset contains 301,536 scores from 2003, ranging from 300 to 850.
  • Empirical setup: Figure 5 compares empirical group CDFs with decision thresholds produced by MaxUtil, DemParity, and EqOpt under two bank utility settings.The settings are u−/u+ = −4 and u−/u+ = −10.
  • Results: When u−/u+ = −10, no fairness criterion surpasses the active harm rate β0, and fairness criteria may perform more favorably when unconstrained profit is misaligned with individual outcomes.The latter occurs because fairness can pull the utility curve toward the outcome curve.
  • Results: EqOpt loans much closer to optimal than DemParity when c+ = −2.This behavior is consistent with the setting suggested by Corollary 3.2.
  • Results: For the black group, DemParity causes a negative expected credit score change, whereas EqOpt and MaxUtil produce similar expected changes.For the white group, fairness constraints mainly reduce total expected profit while preserving a utility-curve shape similar to MaxUtil.

8 Conclusion and Future Work

The conclusion argues that delayed outcomes must be modeled to understand the effects of fairness constraints, and that outcome optimization can provide a more direct alternative when an accurate outcome model is available. It also identifies broader impact measures and robustness to modeling and measurement error as future work.

  • Conclusion: Without a careful model of delayed outcomes, the impact of enforcing a fairness criterion on a classification system cannot be foreseen.The claim concerns the temporal effects of consequential decisions rather than static classification alone.
  • Conclusion: With an accurate outcome model, positive outcomes can be optimized more directly than through existing fairness criteria.The paper presents group score improvement optimization subject to institution utility constraints as one possible alternative.
  • Framework: The framework models outcomes through the expected change in a variable of interest caused by an institutional decision.Its outcome-curve formalism supports interpretation and comparison of solutions.
  • Scope: Applying the framework may require domain knowledge and additional research to understand the two-variable causal mechanism translating decisions into outcomes.The authors connect this requirement to the context-sensitive nature of fairness in machine learning.
  • Future work: Future work should consider impact beyond population-mean change, including variance, individual-level outcomes, and robustness to modeling and measurement errors.These dimensions are identified as important extensions of the current analysis.

A.2 Proof of Lemma 5.2

The proof develops optimal-policy characterizations for constrained utility problems and shows when optimizers can be represented by threshold policies. It also analyzes soft constraints through concavity, derivatives, and regularization.

  • Optimality characterization: The proof uses normal-cone and KKT conditions to characterize optimizers of the constrained utility problem.The resulting pointwise rule depends on the sign of a combination of objective terms and a constraint multiplier.
  • Threshold policies: Under monotonicity conditions, an optimizer can be replaced by an equivalent optimal threshold policy.The construction chooses a threshold from the first score where the multiplier-adjusted objective becomes nonnegative.
  • Selection-rate parameterization: The mapping from selection rates to weighted constraint values is an order-preserving bijection when the weights are positive.Its concavity and one-sided derivatives support the inverse and optimization arguments.
  • Soft constraints: Soft constrained optimization penalizes differences between group constraint values and recovers hard constrained solutions for sufficiently large regularization.The formulation includes DemParity and EqOpt as special choices of the constraint weights.
  • Soft constraints: The proof relies on concavity and convexity to characterize optimal selection rates through first-order conditions.The constraint difference is expressed as Δ = tA − tB, linking group constraint values to the optimization conditions.

B.3 Qualitative Behavior of Soft Constraints

The soft-constraint analysis parameterizes solutions by a group constraint value and the difference between group constraint values. As regularization increases, these quantities interpolate between MaxUtil and the hard constrained solution.

  • Parameterization: Soft-constrained solutions are parameterized by tA and Δ, the constraint value for group A and the difference between group constraint values.The solution sets P(λ), D(λ), and TA(λ) describe policies, constraint differences, and group-A constraint values.
  • Interpolation: As λ increases, the soft constraint interpolates from MaxUtil toward the hard constrained solution, with Δ approaching zero.The interval-valued formulation makes this interpolation precise even when optimal policies are not unique.
  • Special case: If zero belongs to D(λ), a MaxUtil solution satisfies the hard constraint, and the soft and hard solution sets coincide for all positive λ.This is stated as P(λ) = P(∞).
  • Monotonicity: The constraint-difference and group-A constraint intervals change monotonically with λ, in opposite directions depending on the sign of D(λ).The proposition bounds these intervals between their MaxUtil values and their hard-constraint limits.
  • Proof strategy: The monotonicity results follow from concavity, super-gradient behavior, and the increasing contribution of the regularizer.The analysis establishes that TA(λ) is non-decreasing and that the magnitude of Δ tends to zero as λ grows.
  • Ordering assumption: The later qualitative results assume CDF domination, meaning the disadvantaged group has lower quantiles than the advantaged group.Under this ordering, the relevant quantile and utility comparisons follow from monotonicity.

C.2 Proof of Corollary 3.2

The proof derives selection-rate conditions under which DemParity and EqOpt produce relative improvement, while also identifying settings where either criterion causes active harm. It uses monotonicity of the utility function and theorem-based rate bounds to characterize these regimes.

  • Rate bounds: For other population proportions, theorem-based bounds place DemParity or EqOpt below a chosen β′, establishing the rate conditions used in the improvement cases.The proof separately derives upper selection-rate bounds for both criteria from utility monotonicity.
  • DemParity: 0 < g0 < g1; for gA ∈[g0, g1], DemParity’s selection rate lies between β and β′, yielding relative improvement.The bounds follow from Theorem 6.1 and the signs u(QA(β)) < 0 and u(QB(β)) > 0.
  • EqOpt: 0 < g2 < g3; for gA ∈[g2, g3], EqOpt’s selection rate lies between β and β′, yielding relative improvement.This follows from Theorem 6.2 and the utility-sign relation 0 < g2 < g3.
  • DemParity: At β = β0, DemParity causes active harm under the harm-threshold definition and concavity of ∆µA.The proof identifies β0 through ∆µA(r−1, πA(β0)) = 0 with ∆µA decreasing at β0.
  • EqOpt: At β = β0, EqOpt causes active harm under the same concavity-based harm-threshold argument.The proof applies Theorem 6.2 to show a population-proportion regime in which EqOpt selects above β before evaluating β0.

C.5 Proof of Corollary 3.5

This proof compares DemParity and EqOpt selection rates under translated group distributions with affine measurement error. It establishes an ordering between the rates and uses concavity of the outcome curve to infer contrasting delayed effects.

  • Selection-rate comparison: βDemParity > β > βEqOpt under the stated translated-distribution and affine-ρ conditions.The comparison is obtained using Lemma C.2 together with Lemma C.3, which gives G(A→B)(β) > β.
  • Delayed impact: At β = β0, DemParity causes active harm under the concavity of ∆µA.The proof applies the rate ordering at the harm threshold β0.
  • Delayed impact: ∆µA(r−1, πA(...)) > 0 for EqOpt because βDemParity > βEqOpt and the outcome curve is concave.The proof contrasts EqOpt with DemParity through their ordered selection rates.
  • Assumptions: The ordering relies on groups whose distributions are identical up to translation, with µA < µB, and an affine measurement function ρ.These are the explicit assumptions of Lemma C.2 and Lemma C.3.

C.6 Proof of Corollary 3.6

The proof examines how measurement error affects conclusions about MaxUtil, DemParity, and EqOpt, and gives an example where EqOpt causes relative harm. It uses stochastic ordering and true-versus-estimated TPR comparisons to transfer conclusions across settings.

  • EqOpt relative harm: The constructed example satisfies πA ≺ πB while still producing the reversed true-positive-rate ordering at MaxUtil.The example is presented as one where EqOpt causes relative harm.
  • EqOpt relative harm: For any ϵ ∈(0, 1/4), TPRB(τMaxUtil) < TPRA(τMaxUtil) in the constructed six-point example.The example uses πA(5) = 1 − 2ϵ, πA(1) = 2ϵ, πB(5) = 1 − ϵ, and πB(3) = ϵ.
  • Measurement error: If estimated distributions satisfy bπ ≺ π, then bQ(β) ≤ Q(β), supporting the conclusions for MaxUtil and DemParity by utility monotonicity.The proof transfers the relevant theorem conclusions through the upper-quantile ordering.
  • Measurement error: If true TPR dominates estimated TPR for every threshold, the EqOpt conclusion follows by applying Theorem 6.2.The proof uses the same argument as in the proof of Corollary 3.6.
  • Thresholds: The proof uses the unique maximizer β∗ of ∆µA and the uniquely defined threshold β0 determined by continuity and Proposition 5.3.β0 is defined relative to βMaxUtil and a tolerance δ.
Loading 1803.04383v2…