PaperlayerView exact revision on arXiv

Source-linked AI summary

Fair Regression with Wasserstein Barycenters

Evgenii Chzhen, Christophe Denis, Mohamed Hebiri, Luca Oneto, Massimiliano Pontil

arXiv:2006.07286v2stat.MLcs.LGmath.STstat.TH

TL;DR

The paper studies squared-error regression under Demographic Parity when the sensitive attribute is available for prediction. It characterizes the optimal fair predictor through a Wasserstein barycenter and derives a post-processing procedure with fairness and risk guarantees.

  • Problem

    The paper asks how to minimize mean squared error while requiring predicted-output distributions to be independent of the sensitive attribute.

    Source: §1 · p. 1

  • Method

    The authors characterize the optimal fair predictor through a Wasserstein barycenter and use it to post-process a base regression estimator.

    Source: §1 · pp. 1–2

  • Results

    The optimal fair predictor’s distribution is the Wasserstein barycenter of the groupwise regression-function distributions, and the post-processing procedure has distribution-free fairness guarantees.

    Source: §1 · pp. 1–2, §2 · p. 3

  • Takeaways & Limitations

    The characterization gives an intuitive interpretation of the optimum and motivates a post-processing transformation that estimates cumulative distribution functions using unlabeled data.

    Source: §1 · p. 1, §6 · p. 8

    The finite-sample risk analysis uses density assumptions: the lower-bound assumption may be strong, while an upper-bounded density is crucial to the proof.

    Source: §4 · p. 6

Abstract

from arXiv · show

We study the problem of learning a real-valued function that satisfies the Demographic Parity constraint. It demands the distribution of the predicted output to be independent of the sensitive attribute. We consider the case that the sensitive attribute is available for prediction. We establish a connection between fair regression and optimal transport theory, based on which we derive a close form expression for the optimal fair predictor. Specifically, we show that the distribution of this optimum is the Wasserstein barycenter of the distributions induced by the standard regression function on the sensitive groups. This result offers an intuitive interpretation of the optimal fair prediction and suggests a simple post-processing algorithm to achieve fairness. We establish risk and distribution-free fairness guarantees for this procedure. Numerical experiments indicate that our method is very effective in learning fair models, with a relative increase in error rate that is inferior to the relative gain in fairness.

1 Introduction

The paper seeks squared-error-optimal regression under Demographic Parity when the sensitive attribute is available for prediction. It links the optimal fair predictor to a Wasserstein barycenter and uses that characterization to motivate post-processing and statistical guarantees.

1 Introduction

  • The target is a real-valued predictor minimizing mean squared error subject to Demographic Parity, which requires output distributions to be independent of the sensitive attribute.The setting assumes the sensitive attribute is available for prediction.Source: §1 · p. 1
  • The optimal fair predictor’s output distribution is characterized as the Wasserstein barycenter of the standard regression-function distributions across sensitive groups.This connects fair regression to optimal transport and gives an interpretation of the optimum.Source: §1 · pp. 1–2
  • For the salary example, the correction matches a candidate’s regression-function ranking within their group to the corresponding ranking under another group’s distribution.The base prediction is the conditional expected market salary given skills and group.Source: §1 · p. 1
  • The characterization motivates post-processing any regression estimator, with a transformation step that uses unlabeled data to estimate cumulative distribution functions.The procedure first estimates the regression function, then transforms its predictions.Source: §1 · pp. 1–2
  • The paper reports distribution-free fairness guarantees independently of the base estimator and finite-sample risk guarantees under additional assumptions when that estimator is accurate.It also presents a numerical comparison with state-of-the-art methods.Source: §1 · p. 2
  • Prior work studied optimal-transport connections mainly for fair classification, while the paper links Wasserstein barycenters to optimal fair regression and addresses learning bounds.The introduction describes fair regression with learning bounds or consistency results as having received limited attention.Source: §1 · p. 2

2 The problem

The section formulates squared-risk regression under Demographic Parity and links the optimal fair predictor to a Wasserstein-barycenter problem. For two groups, the adjustment matches groupwise rankings before combining predictions.

2 The problem

  • The objective is to minimize squared risk among predictors whose output distributions are identical across sensitive groups.Source: §2 · p. 2
  • Theorem 2.3 links the optimal fair prediction to a Wasserstein-barycenter problem, with the fair predictor’s output distribution matching the associated optimizer.Source: §2 · p. 3
  • The characterization is one-dimensional and uses optimal coupling; unlike the independent result [28], this work emphasizes plug-in estimation, distribution-free fairness, and finite-sample risk bounds.Source: §2 · p. 3
  • For two groups, t*(x, 1) selects a counterpart with the same within-group score rank, and g* is a convex combination of that adjusted score and f*(x, 1).Source: §2 · p. 3, Figure 1 · p. 3

3 General form of the estimator

The estimator applies the optimal predictor’s plug-in form using a regression estimate and groupwise unlabeled data. It estimates group distributions and quantiles from split, jittered samples, with stated fairness and computational properties.

3 General form of the estimator

  • Given a base estimator of f*, the post-processing procedure needs labeled data to estimate f* but only unlabeled data for the other required quantities.Source: §3 · p. 4
  • The procedure splits each group’s unlabeled sample, using one half to estimate quantiles and the other to estimate the CDF.Source: §3 · p. 4
  • Independent uniform jitter smooths the scores and resolves ties, a step the authors identify as crucial to distribution-free fairness guarantees.Source: §3 · pp. 4–5
  • σ=10^-5 is suggested as a small value that does not alter the base estimator’s statistical quality, according to Theorem 4.4.Source: §3 · p. 5
  • Preprocessing costs O(∑ Ns log Ns), while evaluating a new point costs O(maxs log Ns); preprocessing is shared across new points.Source: §3 · p. 5

4 Statistical analysis

The post-processing method provides distribution-free finite-sample fairness guarantees for any base learner. Its risk guarantee requires a sufficiently accurate base estimator and distributional assumptions, including a lower bound on conditional densities that may be strong in practice.

Estimation guarantee with accurate base estimator

  • The risk guarantee shows that a processed estimator is accurate when its base estimator approximates the optimal regression function, under mild distributional assumptions.The analysis assumes conditional densities are bounded below and above; the lower bound may be violated in practice, while the upper bound is proof-crucial.Source: §4 · pp. 5–6

Distribution-free post-processing fairness guarantees

  • 6(N_s∧N_s′ + 1)^−1/2 bounds the expected fairness violation between groups s and s′ for the processed estimator.The guarantee applies to any joint distribution and any base estimator constructed on labeled data.Source: §4 · p. 5
  • The fairness guarantees are distribution-free because jittering introduces continuity, enabling rank-statistic and inverse-transform arguments.The expected violation bound controls the fairness-constraint violation averaged over data.Source: §4 · p. 5

Estimation guarantee with accurate base estimator

  • The risk-bound terms reflect base-regression error, unknown conditional feature distributions, and unknown group proportions; with enough unlabeled data, the base-estimator rate dominates.For i.i.d. unlabeled sampling, the conditional-distribution and group-proportion terms are of the same order.Source: §4 · p. 6
  • When the regression function already satisfies demographic parity, the optimal fair predictor equals it, so its minimax estimation rate remains optimal for the fair target.This follows for distributions where f* satisfies demographic parity.Source: §4 · p. 6

5 Empirical study

Across five benchmark datasets, the proposed method is competitive with existing fair-regression methods and generally achieves larger relative fairness gains than accuracy losses. Experiments compare linear and nonlinear estimators using repeated train/test splits and cross-validation for hyperparameter selection.

Experimental protocol

  • Results average performance and standard deviation over 30 repetitions of 70%/30% train/test splits; hyperparameters are selected by two-step 10-fold cross-validation.The second selection step chooses the lowest-KS option among candidates with MSE within 10% of the best MSE.Source: §5 · p. 7

Methods

  • The method is compared with fair-regression alternatives using linear RLS, kernel RLS, and random forests.Each model family includes the proposed method alongside prior fair-regression approaches.Source: §5 · p. 7

Datasets

  • The comparisons cover five datasets—CRIME, LAW, NLSY, STUD, and UNIV—with race or gender as the protected attribute.Tasks include predicting crime rates, GPA, student grades, and first-semester university grades.Source: §5 · p. 7

Comparison w.r.t. state-of-the-art

  • 50% fairness improvement over the closest fair method occurs in all but CRIME+RLS and CRIME+RF scenarios, reaching 80% in some cases, while accuracy decreases by 1%–30%.The authors report that the relative accuracy decrease is much smaller than the relative fairness improvement; Table 1 reports MSE and KS across datasets and methods.Source: §5 · p. 7, Table 1 · p. 8
  • On NLSY+RLS, RLS+Ours has mean MSE=.203 and mean KS=.09, versus RLS+Oneto at mean MSE=.156 and mean KS=.50.The authors characterize this comparison as about a 30% accuracy drop and an 82% fairness gain.Source: §5 · p. 7

6 Conclusion and perspectives

The paper studies Demographic Parity fair regression when the sensitive attribute is available for prediction, deriving a closed-form optimal fair predictor and a post-processing procedure. The authors report that it yields nearly fair predictions and can be accurate when the base estimator is accurate.

6 Conclusion and perspectives

  • The method assumes the sensitive attribute is available for prediction and derives a closed-form optimal fair predictor.The authors identify extensions without the sensitive feature and to other fairness notions as future work.Source: §6 · p. 8
  • The post-processing procedure transforms any base regression estimator into a nearly fair predictor independently of the underlying distribution.When the base estimator is accurate, the method also yields an accurate estimator of the optimal fair predictor.Source: §6 · p. 8
  • Empirical results indicate that the method is effective at imposing fairness in practice.Source: §6 · p. 8

Appendix

The appendix supplies proofs for three main-text results and repeats those results for convenience. It also provides a short overview of classical results.

Appendix

  • The appendix proves Theorem 2.3, Proposition 4.1, and Theorem 4.4.Source: Appendix · p. 11
  • For reader convenience, the appendix repeats all the results.Source: Appendix · p. 11
  • The appendix includes a short overview of classical results.Source: Appendix · p. 11

A Characterization of the optimal

The characterization proof uses one-dimensional optimal transport to connect fair prediction with Wasserstein-2 barycenters. The theorem assumes that each group-conditional distribution of the standard regression function has a density.

A Characterization of the optimal

  • The proof uses the deterministic optimal-transport map result and a closed-form result for one-dimensional Wasserstein barycenters.The paper identifies Theorem A.1 and Lemma A.2 as the two main ingredients.Source: A Characterization of the optimal · p. 11
  • The characterization theorem assumes that each group-conditional distribution of the standard regression function has a density.Source: A Characterization of the optimal · p. 12
  • The proof relates minimizing squared prediction error under fairness to Wasserstein-2 distances from groupwise regression-output distributions to a shared distribution.Source: A Characterization of the optimal · pp. 12–13
  • The proof constructs group-specific transport maps to a shared distribution and uses them to define an optimal fair predictor.The construction relies on one-dimensional transport maps of the form quantile composed with the CDF.Source: A Characterization of the optimal · pp. 11, 13

B Proof of Proposition 4.1

The appendix gives a distribution-free fairness guarantee for the post-processed estimator, including a bound on conditional prediction-distribution differences across groups. Its proof uses rank-statistics arguments and the Dvoretzky–Kiefer–Wolfowitz inequality.

B Proof of Proposition 4.1

  • Proposition 4.1 states that the post-processed estimator satisfies its fairness guarantees for any joint distribution and any base estimator constructed on labeled data.Source: B Proof of Proposition 4.1 · p. 13
  • 6(N_s∧N_s′ + 1)^-1/2 bounds the conditional CDF difference between post-processed predictions for groups s and s′.The bound is conditional on the available data D.Source: B Proof of Proposition 4.1 · p. 13
  • The proof of the first guarantee uses rank-statistics results, while the CDF-difference bound uses the Dvoretzky–Kiefer–Wolfowitz inequality.Source: B Proof of Proposition 4.1 · p. 13

C Proof of Theorem 4.4

The proof establishes an estimation guarantee for the estimator in Eq. (6) under Assumptions 4.2 and 4.3. It derives the bound by controlling an L1 estimation error and combining separate error bounds.

C Proof of Theorem 4.4

  • Assumption 4.2 requires each group’s distribution of standard-regression predictions to have a density bounded below and above.The lower bounds are denoted λ_s, and the upper bounds are denoted λ_s in the supplied passage.Source: C Proof of Theorem 4.4 · p. 15
  • The proof establishes an L1-norm bound between the standard regression function f* and its estimator using Assumption 4.3.It then gathers three separately bounded terms to obtain the theorem’s estimation guarantee.Source: C Proof of Theorem 4.4 · pp. 15, 19
  • Under Assumptions 4.2 and 4.3, Theorem 4.4 gives an estimation guarantee for the estimator defined in Eq. (6).The leading constant depends only on the constants specified in those assumptions.Source: C Proof of Theorem 4.4 · p. 16
  • The proof can be extended under the same assumptions to control Lp error for p in [1, ∞) and expected sup-norm error.The stated proof handles p = 1; the Lp extension requires extending Lemma C.1, while the rest follows using Wasserstein-p deviation results on the real line.Source: C Proof of Theorem 4.4 · p. 19

Related papers

Loading 2006.07286v2…