PaperlayerView exact revision on arXiv

Source-linked AI summary

Wasserstein Fair Classification

Ray Jiang, Aldo Pacchiano, Tom Stepleton, Heinrich Jiang, Silvia Chiappa

arXiv:1907.12059v1stat.MLcs.LG

TL;DR

Machine-learning classifiers can produce unfair decisions when biased data or model inaccuracies affect sensitive groups. The paper enforces output independence through Wasserstein-1 distances, establishes barycenter-based optimality, and reports that its methods outperform previous approaches while offering deployment-time privacy or simpler post-processing.

  • Problem

    Biased data and model inaccuracies can produce unfair decisions, while demographic-parity enforcement for thresholded outputs is limited to particular thresholds and is difficult to impose directly.

  • Method

    The paper enforces independence between model outputs and sensitive attributes by matching group-specific output distributions to a Wasserstein-1 barycenter.

  • Results

    The proposed methods outperform previous approaches in the literature on four benchmark fairness datasets.

  • Takeaways & Limitations

    The methods provide either a constrained approach that need not access sensitive information at deployment or a faster approximation that requires it at test time.

  • Takeaways & Limitations

    Feasibility issues in training led to degenerate classifiers in some cases.

Abstract

from arXiv · show

We propose an approach to fair classification that enforces independence between the classifier outputs and sensitive information by minimizing Wasserstein-1 distances. The approach has desirable theoretical properties and is robust to specific choices of the threshold used to obtain class predictions from model outputs. We introduce different methods that enable hiding sensitive information at test time or have a simple and fast implementation. We show empirical performance against different fairness baselines on several benchmark fairness datasets.

1 INTRODUCTION

The paper addresses unfair decisions from biased data and model inaccuracies by enforcing fairness directly on classifier outputs rather than relying only on threshold-specific predictions or output uncorrelation.

  • Motivation: Machine-learning decisions in high-impact domains can be unfair toward underrepresented or historically discriminated groups because of data bias and model inaccuracies.Examples include health care, criminal risk assessment, social services, hiring, lending, and online advertising.
  • Existing approach: Fair classification commonly enforces statistical independence between class predictions and sensitive attributes, a criterion called demographic parity.
  • Limitations: Threshold-based independence guarantees fairness only for the particular threshold used to convert continuous model outputs into class predictions.
  • Approach: The proposed approach imposes independence directly on model outputs using small Wasserstein distances between output distributions for groups with different sensitive attributes.
  • Methods: The paper introduces Wasserstein-1 penalized logistic regression, a demographically blind variation, and a simpler faster post-processing method.
  • Evaluation: The proposed methods outperform previous approaches on four benchmark fairness datasets.

2 STRONG DEMOGRAPHIC PARITY

The paper defines strong demographic parity as independence between model beliefs and sensitive attributes, making class-level fairness hold regardless of the prediction threshold. It motivates this criterion and connects it to distribution matching.

  • Data and notation: Each data point contains a sensitive attribute, feature vector, and binary class, while group-specific subsets and corresponding random variables describe the data structure.
  • Data and notation: Logistic regression outputs a belief value in [0, 1], which becomes a binary prediction by thresholding the belief at τ.
  • Demographic parity: Demographic parity requires equal positive prediction probabilities across sensitive-attribute groups, equivalently statistical independence between predictions and the sensitive attribute.
  • Limitations of demographic parity: Enforcing demographic parity at one threshold does not necessarily satisfy it at other thresholds, while matching conditional output means is used as an optimization relaxation.
  • Strong demographic parity: Strong demographic parity enforces independence between the model belief S and sensitive attribute A, so thresholded predictions remain independent for every τ.
  • Strong demographic parity: The paper relates the definition to a dependence measure called strong pairwise demographic disparity, with probability notation and expectation notation specified in the surrounding formalism.

3 WASSERSTEIN FAIR CLASSIFICATION

The paper enforces statistical independence between classifier outputs and sensitive attributes by matching group output distributions with Wasserstein-1 methods. It develops optimal post-processing, penalized logistic regression, demographically blind training, and fast quantile matching, with theoretical guarantees for fairness, minimal prediction changes, and generalization.

  • Wasserstein-1 fairness: Wasserstein-1 distance remains computable for group distributions with disjoint supports, making it suitable for empirical fairness constraints.The method matches each group output pdf to a Wasserstein-1 barycenter distribution.
  • Optimality: For belief variables on [0, 1], Wasserstein-1 distance equals the expected threshold-based class prediction changes under optimal transport.Proposition 1 establishes equality between transportation cost, threshold disagreement, and prediction changes.
  • Optimal post-processing: Matching every group distribution to the weighted Wasserstein-1 barycenter achieves statistical demographic parity while minimizing total expected model prediction changes.The optimal post-processing map transports all group distributions to the barycenter.
  • Optimal trade-offs: When exact parity is relaxed, pseudo-SPDD provides the tightest upper bound on demographic-parity disparity over all possible target distributions.Trade-offs are optimized using pseudo-SPDD because SPDD is difficult to derive directly.
  • Generalization: Under i.i.d. sampling and finite Wasserstein distances, the empirical barycenter converges almost surely to the population barycenter, while empirical Wasserstein costs upper-bound population costs up to a small constant under appropriate conditions.These results support using empirical distributions for implementation and test-time performance.
  • Practical methods: The paper introduces Wasserstein-1 penalized logistic regression and a simpler post-processing approach based on quantile matching.The demographically blind variant excludes sensitive attributes from test-time features, while post-processing maps group quantile bins to corresponding barycenter bins.

4 RELATED WORK

Existing fair classification and regression methods can be grouped into three broad approaches, including preprocessing or representation learning that removes sensitive information during training. Wasserstein-based methods in this category transport features to achieve statistical demographic parity.

  • Approaches: The literature organizes fair classification and regression methods into three main approaches.The supplied passage introduces this taxonomy without detailing all three approaches.
  • Preprocessing and representations: One approach preprocesses data or learns representations that do not contain sensitive information during training.The passage also places feature-transport methods using Wasserstein distances within this approach.

5 EXPERIMENTS

The experiments evaluate Wasserstein-based fairness methods against standard baselines on four UCI datasets, using error and demographic-disparity metrics across thresholds. Wasserstein penalty and post-processing variants achieve strong fairness, while training trajectories expose accuracy–fairness trade-offs.

  • Experimental setup: The study evaluates proposed methods on four UCI datasets and compares them with unconstrained, Hardt’s Post-Process, and constrained optimization baselines.The proposed variants include Wass-1 Penalty, a demographically blind Wass-1 Penalty DB, Wass-1 Post-Process, and a simpler post-processing variant.
  • Metrics: The experiments measure binary error, threshold-averaged error, demographic disparity at threshold 0.5, strong demographic disparity, and strong pairwise demographic disparity.SPDD is target-neutral because it compares group belief distributions without depending on the full-dataset belief distribution or barycenter.
  • Results: After 10,000 training steps with α = 0 and β = 100, all four Adult demographic-group belief histograms match their Wasserstein barycenter.The figure compares Black females, Black males, White females, and White males before and after training.
  • Results: Wass-1 Penalty and Wass-1 Penalty DB obtain the lowest SDD and SPDD on different dataset subsets, while Wass-1 Post-Process follows closely.The reported lowest fairness values occur on German and Crime for Wass-1 Penalty and on Adult and Bank for Wass-1 Penalty DB.
  • Results: As training moves toward strong demographic parity, model accuracy often decreases, making early stopping a way to control the accuracy–fairness trade-off.Figure 2 illustrates trade-off curves between Err-Exp and SDD or SPDD during 80,000 training steps.

6 CONCLUSIONS

The paper concludes that Wasserstein-1 distances can make classifier outputs independent of sensitive information while limiting changes to model decisions. It presents constrained and fast approximate methods with different deployment requirements and reports improved performance over prior approaches.

  • The approach uses Wasserstein-1 distance to make classification-system outputs independent of sensitive information.
  • The Wasserstein-1 barycenter reaches independence with minimal modifications to the model decisions.
  • The paper introduces a constrained method that can avoid sensitive information at deployment and a fast approximation requiring sensitive information at test time.
  • The proposed methods outperform previous approaches in the reported experiments.

A Empirical Estimates

The appendix establishes convergence of empirical Wasserstein barycenter estimates under finite-distance assumptions. The proof uses triangle inequalities, barycenter properties, convergence of empirical distributions, and convergence of group proportions.

  • Under finite Wasserstein-1 distances between group distributions, the empirical barycenter estimate converges almost surely to the population barycenter quantity.
  • The proof bounds empirical barycenter distances using triangle inequalities and the fact that population and empirical distributions have corresponding weighted barycenters.

B Generalization

Under bounded, continuous [0, 1]-valued distributions, empirical Wasserstein-1 distances generalize to population distances with high probability, supporting test-time performance when sufficient samples are available.

  • The generalization lemma assumes W1(pSa, p¯S) ≤ L for every group and continuous distributions supported on [0, 1].
  • The proof bounds population Wasserstein-1 distances using empirical distances plus estimation-error terms derived from the triangle inequality.The argument begins with the case p¯S = pS and applies the triangle inequality to each group.
  • Concentration of measure for Bernoulli random variables yields simultaneous inequalities for all groups with probability at least 1 − δ.
  • Consequently, minimizing the empirical objective implies a low population objective with high probability when enough samples are available.The result is stated as a connection between empirical and population Wasserstein objectives and their test-time behavior.
  • Figure 3 illustrates the equality used in Eq. (12) by showing two integrations that calculate the same shaded area.
  • When p¯S is the weighted barycenter of the group-level distributions, the corresponding population objective can be related to empirical Wasserstein distances and an error term.

C Inverse CDFs

For differentiable, invertible cumulative distribution functions on [0, 1], the proof relates differences between inverse CDFs to differences between the original CDFs through shared signed intervals and area calculations.

  • Lemma 6 considers differentiable and invertible cumulative distribution functions f and g over the probability space [0, 1].
  • The proof also uses that invertible CDFs are strictly increasing and satisfy f(0) = g(0) = 0 and f(1) = g(1) = 1.
  • The proof partitions [0, 1] into intervals where h = f − g has a constant sign between consecutive roots.Continuity ensures that each such interval has endpoints at roots and contains no additional roots.
  • Summing the interval-wise relationships produces the result for all intervals, with the proof using change of variables and integration by parts.
  • On any interval where f > g, the inverse ordering reverses, so f^-1 < g^-1; the opposite reversal holds where f < g.
Loading 1907.12059v1…