Source-linked AI summary
A Clarification of the Nuances in the Fairness Metrics Landscape
Alessandro Castelnovo, Riccardo Crupi, Greta Greco, Daniele Regoli, Ilaria Giuseppina Penco, Andrea Claudio Cosentini
TL;DR
Fairness in ML has accumulated many definitions, but their differences, implications, and relationships remain unclear. This paper organizes the landscape by analyzing relationships among metrics across group versus individual and observational versus causality-based notions, concluding that they form intertwined possibilities rather than clear-cut distinctions.
Problem
Fairness research offers many definitions capturing different aspects of fairness, but the overall landscape and interdependence among metrics remain unclear.
Method
The paper provides a mainly qualitative analysis grounded in rigorous definitions to clarify relationships among fairness metrics across group versus individual and observational versus causality-based notions.
Results
The fairness metrics landscape comprises intertwined possibilities rather than clear-cut distinctions, particularly across the individual-versus-group and observational-versus-causality-based dimensions.
Takeaways & Limitations
Understanding fairness metrics requires examining their relationships and the distinct aspects of fairness they capture.
Takeaways & Limitations
Enforcing demographic parity by favoring a group can amplify the bias it aims to break, while suppressing sensitive-attribute information may reduce decision accuracy.
Abstract
from arXiv · showhide
In recent years, the problem of addressing fairness in Machine Learning (ML) and automatic decision-making has attracted a lot of attention in the scientific communities dealing with Artificial Intelligence. A plethora of different definitions of fairness in ML have been proposed, that consider different notions of what is a "fair decision" in situations impacting individuals in the population. The precise differences, implications and "orthogonality" between these notions have not yet been fully analyzed in the literature. In this work, we try to make some order out of this zoo of definitions.
Introduction
Fairness in ML has attracted substantial attention as automated decisions raise concerns about discrimination and bias. This paper organizes the expanding landscape of fairness metrics around their relationships, distinctions, and incompatibilities.
- Fairness research addresses how to measure fairness and how to mitigate bias in data-driven decision-making models.
- The proliferation of fairness definitions reflects fairness’s multiple meanings and context-dependent nuances, but creates a complex landscape for researchers and practitioners.
- Existing fairness notions broadly concern statistical parity across groups, equal treatment of similar individuals, or causal disentanglement of unfair impacts.
- These notions can be organized along observational versus causality-based and group versus individual dimensions.
- Bias-mitigation methods operate at three pipeline stages: pre-processing data, in-processing training, or post-processing model outcomes.
- The paper focuses mainly on binary classification with one sensitive attribute and qualitatively analyzes links and incompatibilities among fairness metrics.
1. The problem of bias in data-driven decisions
Bias in data-driven decisions can arise from unrepresentative data, measurement and recording problems, minority underrepresentation, or historically biased decisions. These sources differ from poor modeling but can interact in practice.
- Statistical or representation bias occurs when training data are not representative of the true population, including through selection bias.
- Systematic measurement errors and missing or poorly recorded data can distort information unevenly across population groups.
- Even representative data can produce unequal accuracy or neglected errors when a protected minority group is highly imbalanced.
- Poor modeling may also be considered a source of biased decisions, although the boundary between modeling and data problems is subtle.
- Historical or societal bias occurs when learned data reflect past prejudice or biased decisions used to create target labels.
2. Individual fairness
Individual fairness requires similar individuals to receive similar decisions, but assessing similarity and accounting for sensitive-attribute correlations are difficult. The paper contrasts unawareness, similarity-based, and counterfactual approaches.
- Individual fairness compares decisions for similar individuals, unlike group fairness, which evaluates equality of treatment across groups.
- Similarity-based criteria: Similarity-based fairness uses a feature-space distance so that nearby individuals receive nearby outcomes, often enforced through constrained training optimization.
- Similarity-based criteria: The central difficulty is defining an issue-specific similarity distance that captures decision-relevant features without improperly mixing sensitive attributes.
- Similarity-based criteria: Fairness Through Unawareness requires decisions to remain unchanged when only the protected attribute changes.
- Similarity-based criteria: Fairness Through Unawareness can be difficult to assess because it concerns model behavior rather than only properties of observed outputs.
- Similarity-based criteria: Removing the sensitive attribute alone may fail when other features encode information correlated with it, while suppression can discard legitimate information.
- Similarity-based criteria: Counterfactual frameworks define similarity by comparing an individual with that same individual in a world where the sensitive attribute differs, while accounting for causal changes in other features.
3. Group fairness
Observational group fairness is expressed through conditional-independence notions of independence, separation, and sufficiency. These criteria differ in whether they use the target variable and can be affected by target-label bias and selection bias.
- Independence corresponds to Demographic or Statistical Parity, separation to Equality of Odds, and sufficiency to calibration or Predictive Parity.
- Independence uses protected attributes, features, and decisions, whereas separation and sufficiency also use the target variable Y.
- When Y represents a loan officer’s decision, separation- and sufficiency-based criteria require care because Y may itself be biased against groups.
- Even repayment outcomes can be affected by selection bias because repayment information is observed only for applicants who received loans.
- Disparate mistreatment denotes group fairness criteria based on disparities in errors comparing predicted decisions with true outcomes.
- Independence generally is not satisfied by a perfect classifier, while error-rate parity trivially holds for a perfect predictor.
3.1. Independence
Independence requires decisions to be independent of sensitive attributes, commonly operationalized as equal positive prediction ratios across groups. Demographic parity can require group-specific treatment when non-sensitive features correlate with protected attributes, and may trade off against accuracy or future outcomes.
- Independence requires the decision to be independent of the sensitive attribute A.
- Demographic parity requires equal positive prediction ratios across groups, such as equal loan-granting rates for men and women.
- The 4/5 rule treats a DP ratio above 80% as a rule of thumb, with exceptions for business or job-related necessities.
- Subtleties of demographic parity: Removing gender is insufficient when rating correlates with gender; achieving DP may require different thresholds for men and women.
- Subtleties of demographic parity: Suppressing gender-related information can achieve DP but may discard useful information and reduce decision accuracy.
- Subtleties of demographic parity: Imposing DP can produce future harms by increasing acceptance among lower-skilled applicants and potentially reinforcing negative perceptions of that group.
- Conditional demographic parity: Conditional demographic parity requires equal decisions across groups conditional on the same rating or other features, moving toward an individual fairness requirement.
3.2. Separation
Separation requires parity conditional on the true outcome, focusing on equal error rates across sensitive groups. Equality of Odds imposes equal false-positive and false-negative rates, while relaxed versions impose only one of these equalities.
- Separation focuses on parity among people sharing the same true outcome, taking the perspective of individuals subject to model decisions.
- Separation requires the decision to be independent of the sensitive attribute conditional on the true outcome Y.
- Fairness assessments based on Y require caution because biased labels or selection bias can distort comparisons with the target.
- Equality of Odds requires equal false-positive and false-negative rates across groups.
- Predictive Equality requires equality of false-positive rates, whereas Equality of Opportunity requires equality of false-negative rates.
- Separation is suitable when the target is an objective ground truth and discrimination may be justified by trustworthy data.
3.3. Sufficiency
Sufficiency requires parity among people receiving the same model decision, focusing on predictive error rates rather than errors relative to the true outcome. Predictive Parity is its precision-based form.
- Sufficiency requires parity among individuals receiving the same model decision, irrespective of sensitive features.
- Separation measures errors relative to the ground truth, whereas sufficiency measures non-repayment among people granted a loan.
- The distinction corresponds to recall for separation and precision for sufficiency.
- Predictive Parity requires the same precision across sensitive groups.
- Predictive Parity and sufficiency take the decision maker’s perspective by grouping people according to model decisions.
- Other group criteria can require equal accuracy across groups by equalizing unconditional correctness rates.
3.4. Group fairness on scores
Score-based fairness extends group criteria from binary decisions to model scores, imposing distributional or average-score constraints. Score-based sufficiency is connected to calibration within groups.
- Most classification models output a real-valued score S, with a final binary decision produced by thresholding the score.
- Score-based criteria impose stronger constraints than analogous criteria defined only on binary decisions.
- Group-dependent thresholds can satisfy score versions of independence, separation, and sufficiency, but not every score-based criterion.
- Balance of the Negative Class corresponds to Predictive Equality, while Balance of the Positive Class corresponds to Equality of Opportunity.
- AUC parity is the score-based analogue of equality of accuracy across groups.
- Calibration within groups requires that people assigned score s have an average positive-outcome rate of s, and follows from score-based sufficiency.
3.5. Incompatibility statements
The paper shows that major group fairness criteria are generally incompatible except in trivial, degenerate, or specially balanced cases, making the choice of criterion consequential. The COMPAS debate illustrates how different fairness notions can produce conflicting assessments of the same model.
- Separation and independence are incompatible when the target depends on both the sensitive attribute and the score-related variable.
- Sufficiency and independence cannot both hold when base rates differ across sensitive groups.
- Separation and sufficiency are incompatible under non-independent targets and strictly positive joint distributions, except with equal base rates or perfect prediction.
- In the COMPAS debate, Predictive Parity and Equality of Odds could not both hold when true recidivism rates differed by race.
- Multiple group fairness criteria cannot generally be satisfied simultaneously except in trivial or degenerate scenarios.
3.6. Multiple sensitive features
With multiple sensitive features, assessing fairness separately for each attribute is generally insufficient, while intersectional fairness theoretically addresses subgroup bias. In practice, the resulting subgroup explosion and sparse samples make assessment difficult or infeasible.
- The discussion of multiple sensitive features is limited to a brief overview because the manuscript primarily assumes one categorical sensitive variable.
- Independent group fairness evaluates fairness separately for each sensitive attribute but is generally insufficient for preventing bias in combined subgroups.
- Intersectional group fairness combines sensitive attributes into one feature and imposes fairness constraints across all resulting subgroups.
- Intersectional constraints theoretically address intersectional bias, but practical and computational problems remain.
- The number of subgroups grows exponentially as sensitive features are added, while finite samples leave many subgroups empty or sparsely observed.
- Assessing group fairness across multiple sensitive attributes may therefore be unfeasible in most practical cases, despite such attributes commonly occurring together.
4. Group vs. individual fairness
The paper argues that group and individual fairness are not generally opposed: observational criteria can be organized along shared dimensions, with ethical and legal assumptions determining the appropriate point. Group fairness may still permit within-group discrimination, while suppressing sensitive information can reduce predictive performance or individual disparate treatment.
- Group versus individual fairness: Group fairness constrains outcomes on average across groups, leaving room for discriminatory treatment within groups.Different thresholds can achieve demographic parity while treating people with comparable ratings differently.
- Group versus individual fairness: Suppressing gender and correlated features can prevent group and subgroup discrimination, but discards information useful for approximating outcomes.The amount of information removed depends on how many variables correlated with the sensitive attribute are discarded.
- Landscape of observational criteria: The fairness landscape uses individual-level fairness and retained information about the sensitive attribute as qualitative dimensions.Figure 4 places observational criteria in this plane, while conditional demographic parity lies between unconditional demographic parity and fairness through unawareness depending on the conditioning variables.
- Group versus individual fairness: Fair representation can satisfy demographic parity without generally satisfying individual fairness, because similar individuals may map to different representations and outcomes.The paper illustrates this with a representation retaining information orthogonal to the sensitive attribute.
- Performance and information: The performance and information trade-off is qualitative rather than universal: using all features is generally more efficient, but demographic-parity models can outperform fairness-through-unawareness models in some scenarios.The paper explicitly notes a case where the outcome is independent of the sensitive attribute while the features are not.
- Compatibility of fairness notions: Individual and group fairness can occupy the same continuum, from separation to fairness through unawareness, with conditional metrics between these extremes.The chosen position depends on which variables are ethically and legally allowed to influence decisions; the same position can be expressed through a distance function or conditional group parity.
5. Causality-based criteria
Causality-based fairness criteria use causal structures to assess how protected attributes affect decisions, ranging from interventions to stronger counterfactual analyses. Compared with observational criteria, they offer more intuitive questions but require stronger assumptions and may be difficult or impossible to validate from observed data.
- Observational criteria enforce equal acceptance or error metrics across groups, whereas causality-based criteria ask how decisions would change under interventions or alternate attributes.Counterfactual questions can be more intuitive, but their validity depends on assumptions about the data-generating mechanism.
- Causality-based criteria use domain knowledge and inference to uncover causal relationships before assessing fairness in a specific setting.They differ from observational criteria, which rely on statistical computations over observed distributions and predictions.
- Counterfactual fairness can be difficult to compute from observational data and is not generally falsifiable because infinitely many counterfactual models may fit the same causal structure.Intervention quantities may also require an SCM or physical experimentation when the relevant post-intervention events are unobserved.
- Causal analysis distinguishes intervention-level fairness from counterfactual fairness, with the latter requiring a more detailed structural causal model.A causal graph suffices for interventions, while counterfactuals require structural equations and assumptions about latent variables and functional relationships.
- Counterfactual fairness compares an observed individual with that same individual in a counterfactual world where the protected attribute differs.Unlike interventions averaging over compatible exogenous factors, counterfactuals condition on exogenous values compatible with the individual's factual observation.
- Intervention fairness compares acceptance probabilities for individuals with the same observed features under different exogenous settings of the protected attribute.The intervention do(A = a) removes incoming causal paths to A and sets A externally; this corresponds theoretically to a randomized experiment.
- CFF and fair representation both remove causal information about A from the feature space, combining a causal analogue of demographic parity with an individual-level condition.The comparison frames CFF as forbidding all causal routes from A to the decision while conditioning on the individual.
6. Conclusions
Fairness in ML has produced many quantitative definitions because fairness is multifaceted, yet their relationships remain unclear. The paper argues that fairness criteria form an intertwined landscape spanning individual versus group and observational versus causality-based perspectives.
- Fairness and discrimination in decision making have many nuances, extending beyond ML and becoming harder to inspect when decisions come from opaque algorithms.The paper connects this broader issue with the rise of Explainable AI and black-box decision systems.
- The paper highlights continuing confusion about how fairness metrics relate across the individual-versus-group and observational-versus-causality-based dimensions.Its conclusion rejects clear-cut separations in favor of a landscape of intertwined possibilities.
- Quantitative fairness research can contribute to understanding bias discrimination by exposing shortcomings and subtleties in fairness notions.The paper acknowledges reasonable doubts about whether quantitative methods can capture the full complexity of equity and fairness.
Appendix A. Experiments
The experiments compare four fairness-oriented algorithmic strategies across one Adult and two synthetic datasets using information, fairness, and performance metrics. Results broadly corroborate the qualitative trade-offs among retaining information about A, group fairness, individual fairness, and predictive performance.
- Datasets and setup: Experiments use the Adult dataset and two synthetic datasets designed to capture different dependencies between features, A, and target variables.The synthetic datasets use high- and low-correlation target settings, while Adult uses gender as the sensitive attribute.
- Algorithms and metrics: Four strategies are evaluated: FTU, two Suppression variants, CDP, and DP, targeting different ways of mitigating bias.FTU excludes A, Suppression filters features by correlation with A, and DP imposes unconditional acceptance-rate parity; CDP is also evaluated.
- Algorithms and metrics: The experiments track normalized mutual information U, DP-ratio, Flip, and ROC AUC to measure information about A, group fairness, individual fairness, and classification performance.Symmetric Uncertainty normalizes mutual information by the mean entropy of its two variables; DP-ratio equals 1 under perfect parity.
- Results: FTU exploits the highest fraction of information about A and is consistently the best-performing approach, while full Suppression and DP-compliant models rank lowest in performance.CDP and mild Suppression fall between these extremes in information use and performance.
- Results: FTU and Suppression are fully individual under Flip by design, whereas DP-compliant approaches negatively affect Flip and FTU reaches very low DP-ratio values.FTU’s low DP-ratio is especially pronounced when mutual information between Y and A is high.
- Results: When mutual information between Y and A is high, DP-compliant methods and full Suppression perform poorly because enforcing Ŷ ⊥⊥ A conflicts with high predictive performance.Suppression’s behavior depends on which variables remain after applying its correlation thresholds.