An Intersectional Definition of Fairness

James Foulds; Rashidul Islam; Kamrun Naher Keya; Shimei Pan

Source-linked AI summary

An Intersectional Definition of Fairness

James Foulds, Rashidul Islam, Kamrun Naher Keya, Shimei Pan

arXiv:1807.08362v3Jul 22, 2018cs.LGcs.CYstat.ML

TL;DR

The paper addresses how AI fairness definitions can account for overlapping protected attributes and structural disadvantages emphasized by intersectionality. It proposes intersectional metrics, proves their theoretical guarantees, and develops a learning algorithm evaluated on census and COMPAS data. The experiments show improved fairness metrics with modest accuracy-related costs.

Problem
Existing AI fairness approaches must address harms associated with intersecting protected attributes and structural disadvantages, not only separate subgroup outcomes.
Method
The paper proposes differential fairness and bias-amplification and confounder-aware variants, proves intersectionality, privacy, economic, and generalization properties, and provides a learning algorithm.
Results
Both DF-Classifier and SF-Classifier substantially improved fairness metrics over a typical classifier on Adult and COMPAS, with modest costs in accuracy, F1 score, and ROC AUC.
Takeaways & Limitations
The criteria can be practically attained and behave more equitably toward minority groups than subgroup fairness in experiments across two datasets.

Abstract

from arXiv · show

We propose definitions of fairness in machine learning and artificial intelligence systems that are informed by the framework of intersectionality, a critical lens arising from the Humanities literature which analyzes how interlocking systems of power and oppression affect individuals along overlapping dimensions including gender, race, sexual orientation, class, and disability. We show that our criteria behave sensibly for any subset of the set of protected attributes, and we prove economic, privacy, and generalization guarantees. We provide a learning algorithm which respects our intersectional fairness criteria. Case studies on census data and the COMPAS criminal recidivism dataset demonstrate the utility of our methods.

I. INTRODUCTION

The paper argues that AI fairness should connect technical criteria to intersectionality, accounting for overlapping protected attributes and structural oppression. It introduces intersectional fairness criteria and supporting analyses, algorithms, guarantees, and empirical studies.

AI systems can reproduce harmful and pervasive bias, motivating mathematical fairness formulations alongside broader social, political, philosophical, and legal analysis.
The paper contributes three fairness metrics, proofs of intersectionality, privacy, economic, and generalization properties, an enforcing learning algorithm, and census and recidivism case studies.
Intersectionality analyzes how interlocking systems of oppression create disadvantages across overlapping dimensions such as gender, race, disability, and socioeconomic class.
Historical data can reflect unfair processes, so accurate prediction of outcomes such as student GPA may not produce a fair decision-making procedure.
The paper’s intersectional fairness desiderata require multiple protected attributes, protection for their intersections and individual values, attention to minorities, and rectification of structural disparities.
These desiderata guide the selection of fairness measures but do not uniquely determine one, leaving legal, political, and contextual considerations relevant to particular tasks.

III. EXISTING FAIRNESS DEFINITIONS

Existing multi-attribute fairness definitions include subgroup fairness and multicalibration, but statistical parity subgroup fairness can downweight minority groups. The paper introduces a setting and criteria addressing intersectional groups while retaining group-outcome comparisons.

Existing approaches: Subgroup fairness and multicalibration are the two dominant multi-attribute fairness approaches discussed.False positive subgroup fairness and multicalibration are related variants, focused respectively on false-positive rates and calibration.
Statistical parity subgroup fairness: A toy example shows that varying P(minority) changes γ-subgroup fairness even when positive-class probabilities remain 0.8 for the majority and 0.1 for the minority.The example describes a more extreme scenario as more acceptable when the minority group is smaller.
Proposed setting: The paper’s framework models a possibly randomized mechanism over instances, discrete protected attributes, and their joint distribution.Protected attributes may be included in the input even when the mechanism is free to disregard them.
Statistical parity subgroup fairness: Statistical parity subgroup fairness weights each group’s deviation from the population average by its population proportion.The mechanism is binary, and smaller γ indicates better fairness.
Differential fairness: The proposed ϵ-DF is constant in P(minority), unlike γ-SF’s size-sensitive treatment of minority groups.Figure 4 compares per-group γ-SF and ϵ-DF across intersectional subgroups and top-level groups in the Adult dataset.
Statistical parity subgroup fairness: Small groups can have little influence on overall γ-SF because per-group unfairness is downweighted by group size.On the Adult dataset, per-group γ increases with group probability, making a small minority group unlikely to determine the worst-case γ-SF value.

IV. DIFFERENTIAL FAIRNESS (DF) MEASURE

Differential fairness (DF) extends the 80% rule to intersectional protected groups and multiple outcomes, measuring disparity on a privacy-inspired sliding scale. The formulation compares outcome probabilities across protected-attribute combinations while supporting privacy-related guarantees and empirical estimation.

Definition and motivation: Differential fairness (DF) extends the 80% rule to protect multi-dimensional intersectional categories across multiple output values.Unlike the fixed 80% threshold, DF measures fairness with a parameter ϵ on a sliding scale related to differential privacy.
Definition and motivation: DF compares outcome-probability ratios between tuples of protected attributes, such as gender, race, and nationality.The definition applies across plausible distributions θ and output values y when the compared groups have positive probability.
Privacy connection: Differential fairness adapts pufferfish privacy by treating protected attributes as secrets whose values should remain indistinguishable.This gives DF a privacy guarantee closely related to differential privacy.
Estimation: DF can be estimated empirically from group outcome counts or with smoothed empirical differential fairness using a symmetric Dirichlet prior.The paper also proposes soft counts from probabilistic predictions for difficult cases involving nearly single-class assignments.

V. DF BIAS AMPLIFICATION MEASURE

The paper extends differential fairness to data and compares a classifier’s fairness with the fairness of its labeled training data. This difference measures the additional fairness cost introduced by the classifier, called bias amplification.

Data and classifier fairness: Differential fairness can measure fairness in data by treating a model of the data-generating process as the mechanism.For a labeled dataset, the model is trained on the dataset and produces outcomes through M(x) = y.
Bias amplification: The difference ϵ2 − ϵ1 measures the additional fairness cost of using mechanism M2 instead of M1 under fixed A and Θ.The comparison is meaningful when the minimum ϵ values are tightly computed.
Bias amplification: When ϵ1 describes the labeled data and ϵ2 describes a classifier on that data, ϵ2−ϵ1 measures classifier-induced bias amplification.The measure captures how much the classifier increases unfairness relative to the original data.
Interpretation: DF bias amplification does not correct unfairness already present in the original dataset, unlike the more progressive ϵ-DF notion.The paper characterizes this metric as more politically conservative and consistent with inframarginality.

VI. ILLUSTRATIVE WORKED EXAMPLES

The worked examples show how differential fairness evaluates intersectional outcome disparities, handles Simpson’s reversal, and motivates a confounder-aware extension.

Differential fairness evaluates outcome probabilities for each intersection of multiple protected attributes.
In the admissions example, gender and race intersections exhibit Simpson’s paradox: gender A has higher admission rates within each race, while gender B leads overall.
The admissions mechanism is ϵ = 1.511-DF when evaluated across Gender × Race pairs.
When confounders are present, overall disparity may be desirable, so parity across protected intersections may no longer be appropriate.
Differential fairness with confounders requires fairness conditional on each confounder value.
In university admissions, the most unfair department determines overall ϵ-DFC, while department-level protection preserves overall differential fairness and its associated guarantees.

VIII. PROPERTIES OF DIFFERENTIAL FAIRNESS

The paper proves that differential fairness at the finest protected intersections extends to any subset of protected attributes, reducing the groups that must be explicitly protected.

Differential fairness explicitly protects intersectional groups and automatically implies the same fairness level for each protected attribute and any subset of attributes.
The intersectionality theorem states that an ϵ-differentially fair mechanism over all protected attributes is ϵ-differentially fair over every nonempty proper subset.
The proof rewrites subset-level outcome probabilities by averaging over the complementary protected attributes.
Figure 7 varies the number of protected attributes while assigning two values to each attribute to count protected groups and intersectional subgroups.
Protecting bottom-level intersectional groups automatically protects higher-level groups and subgroups, avoiding explicit protection of every group.

B. Privacy Interpretation

Differential fairness has a privacy interpretation: conditional on an outcome, an adversary’s beliefs about protected attributes change little relative to prior beliefs.

Differential fairness ensures that an untrusted vendor or adversary learns very little about protected attributes from an observed outcome, assuming priors lie in Θ.
For example, receiving a loan should not substantially change an adversary’s Bayesian posterior beliefs about an individual’s race and gender.
Although differential fairness is population-level, this privacy guarantee applies to individuals and can prevent subsequent discrimination by an untrusted mechanism user.

C. Economic Guarantees

The paper establishes economic and generalization properties for differential fairness, while noting that accurate estimation requires sufficient data within each intersectional group.

Economic Guarantees: An ϵ-differentially fair mechanism permits expected-utility disparity between protected groups by at most exp(ϵ) ≈ 1 + ϵ for small ϵ.
Economic Guarantees: DF bias amplification bounds an algorithm’s multiplicative increase in expected-utility disparity relative to the data-generating process by exp(ϵ2 − ϵ1) ≈ 1 + ϵ2 − ϵ1 for small differences.
Generalization Guarantees: Generalization guarantees for differential fairness require a reasonable number of examples from each intersectional group to estimate ϵ-DF accurately.
Generalization Guarantees: This data requirement is the cost of removing the minority-biasing term used in statistical-fairness bounds.
Generalization Guarantees: The intended intersectional applications often involve only around 2^2–2^5 intersectional groups, making sufficient per-group data more attainable.

IX. LEARNING ALGORITHM

The paper trains differentially fair classifiers by adding a tunable fairness penalty to the prediction loss, then evaluates fairness–performance trade-offs on Adult and COMPAS data. DF-Classifier improves intersectional and minority-group fairness, generally with modest accuracy costs.

Learning algorithm: DF-Classifier minimizes prediction loss plus a fairness penalty weighted by λ, using gradient descent and backpropagation to train neural networks.The penalty can target differential fairness or bias amplification through the threshold ϵ1.
Experimental design: Experiments compare DF-Classifier, subgroup-fair SF-Classifier, and an unpenalized typical classifier on Adult and COMPAS datasets.Adult uses race, gender, and nationality; COMPAS uses race and gender.
Overall results: Both fairness-aware classifiers substantially improve fairness metrics over the typical classifier, with modest costs in accuracy, F1 score, and ROC AUC.The fairness–performance trade-off varies roughly monotonically with the target threshold ϵ1 or γ1.
Overall results: ϵ = 0.180 on COMPAS improves from ϵ = 0.773, with a 1.4-percentage-point accuracy loss and worst-case utility differing by approximately eϵ ≈ 1.2.On Adult, ϵ improves from 1.646 to 0.428 with a 2.8-percentage-point accuracy loss.
Minority groups: DF-Classifier improves per-group fairness for groups of all sizes, whereas SF-Classifier fails to improve per-group γ-SF for small groups.The γ-SF overall measurement can ignore minority groups because it emphasizes large groups.
Minority groups: DF-Classifier provides greater protection to minority groups than subgroup fairness while incurring a minor performance loss.This conclusion is based on the COMPAS per-group analysis.

B. Inequity of Fairness Measures

The paper evaluates whether fairness metrics distribute attention equitably across intersectional groups. Across Adult and COMPAS, ϵ-DF is substantially more equitable than γ-SF and preserves intersectionality when protected attributes are removed.

Gini-based inequity: The Gini coefficient measures inequity in how a fairness metric’s consideration is allocated across intersectional groups, with lower values indicating more equitable distribution.The calculation weights each group’s fairness measure by its population fraction.
Gini-based inequity: ϵ-DF has much lower Gini coefficients than γ-SF on both Adult and COMPAS, for labeled data and logistic-regression classifiers.The authors attribute this observation provisionally to ϵ-DF’s more inclusive treatment of groups.
Intersectionality property: ϵ-DF obeys the intersectionality property, whereas γ-SF violates it on COMPAS and Adult when protected attributes are removed.The violations occur because γ for gender exceeds γ for race × gender on COMPAS and γ for gender × nationality on Adult.
Conclusion: The conclusion reports that the proposed metrics, proofs, and learning algorithm satisfy intersectional fairness desiderata and show practical utility across two datasets.The paper also identifies data sparsity in multi-attribute settings as future work.

APPENDIX

The appendix derives equivalent formulations of differential fairness by applying logarithms, reciprocal symmetry, and worst-case bounds over protected-group pairs. These steps reduce pairwise probability-ratio constraints to a compact criterion.

Differential fairness proof: Differential fairness is defined over every model parameter, outcome, and pair of protected groups with positive conditional probability.The proof quantifies over θ, y, and (si, sj).
Log transformation: Taking logarithms transforms the probability-ratio condition into a bounded difference between log outcome probabilities for two protected groups.The resulting bound is expressed using ϵ.
Worst-case formulation: The pairwise inequalities can be simplified by bounding the expression with the worst case over protected-group pairs.The proof states that this bound is achievable and therefore tight.
Symmetry argument: For confounder-aware comparisons, reversing the protected-group pair and taking reciprocals establishes the second inequality.The argument applies for valid θ, y, c, and protected-group pairs.

C. Related Work

The paper positions differential fairness against established fairness, privacy, and intersectionality approaches. Its distinctive choice is to treat intersecting protected categories equally while bounding outcome-probability ratios across them.

Scope: The related-work discussion situates differential fairness among mathematical fairness definitions and examines its relationships to fairness, privacy, and protected-group subsets.The paper briefly reviews influential alternatives.
Existing fairness definitions: The 80% rule detects disparate impact using a favorable-outcome probability ratio, corresponding to ϵ ≥ −log 0.8 = 0.2231 in a restricted differential-fairness test.That version considers only the favorable outcome y = 1.
Existing fairness definitions: Demographic parity matches outcome distributions across protected groups, while equalized odds instead requires equal error rates across groups.Equalized odds is described as rewarding accurate classification and penalizing majority-only performance.
Individual and causal fairness: Individual fairness requires similar individuals to receive similar outcomes but depends on defining similarity fairly and may not generalize beyond the training set.The approach can preserve privacy when classification users are untrusted.
Individual and causal fairness: Counterfactual fairness holds non-causally-dependent factors fixed while changing protected attributes, but requires an accurate fine-grained causal model.The paper presents implementation of this requirement as difficult.
Intersectional fairness: Unlike threshold tests that account for risk differences within protected categories, differential fairness requires intersecting protected subsets to be treated equally.This contrast follows the paper’s intersectionality-based criterion.
Privacy: Differential fairness resembles differential privacy by bounding outcome-probability ratios, but compares protected-attribute values rather than neighboring databases.It can be satisfied by deterministic mechanisms because it incorporates randomness in data as well as mechanisms.
Privacy: Differential fairness constrains the final algorithm’s behavior with respect to protected attributes, unlike related work focused on privacy during the learning process.The comparison concerns fair learning with untrusted vendors.