Source-linked AI summary
Measuring Fairness in Ranked Outputs
Ke Yang, Julia Stoyanovich
TL;DR
The paper asks how to measure statistical parity in rankings that can disadvantage protected groups. It proposes rank-aware fairness measures, a controllable synthetic-data generator, and evaluations on real datasets and optimization. The results demonstrate unfairness in real rankings and preliminary potential to improve fairness while maintaining accuracy.
Problem
Ranked outputs can appear objective while systematically disadvantaging protected groups, motivating measures of statistical parity that account for rank position.
Method
The paper proposes ranked fairness measures, generates synthetic rankings with controllable fairness, evaluates real datasets, and incorporates the measures into an optimization framework.
Results
The measures identify unfairness in real rankings, while preliminary optimization results show fairness measures converging to low values and accuracy varying across ranking setups.
Takeaways & Limitations
Rank-aware fairness quantification supports evaluating and potentially improving ranked outputs, with preliminary evidence of fairness improvement while retaining accuracy.
Takeaways & Limitations
The rRD measure is applicable only when the protected group is a minority and the fairness probability is below 0.5.
Abstract
from arXiv · showhide
Ranking and scoring are ubiquitous. We consider the setting in which an institution, called a ranker, evaluates a set of individuals based on demographic, behavioral or other characteristics. The final output is a ranking that represents the relative quality of the individuals. While automatic and therefore seemingly objective, rankers can, and often do, discriminate against individuals and systematically disadvantage members of protected groups. This warrants a careful study of the fairness of a ranking scheme. In this paper we propose fairness measures for ranked outputs. We develop a data generation procedure that allows us to systematically control the degree of unfairness in the output, and study the behavior of our measures on these datasets. We then apply our proposed measures to several real datasets, and demonstrate cases of unfairness. Finally, we show preliminary results of incorporating our ranked fairness measures into an optimization framework, and show potential for improving fairness of ranked outputs while maintaining accuracy.
1. INTRODUCTION
The paper studies statistical parity in ranked outputs because ranking-based decisions can disadvantage protected groups despite appearing objective. It proposes rank-aware fairness measures to support transparency and better algorithmic rankers.
- Rankings influence consequential decisions such as admissions, hiring, lending, and grant making, affecting individuals’ livelihood and well-being.
- Statistical parity requires the demographics receiving a positive outcome to match population demographics; this paper focuses on one binary protected group at a time.
- The paper proposes several measures that quantify statistical parity, or its absence, in ranked outputs.
- Rankings require rank-aware fairness because higher positions provide better outcomes than lower positions rather than assigning strictly positive or negative outcomes.
- The measures are evaluated on synthetic and real datasets, followed by preliminary optimization results showing fairness improvement while tighter fairness-accuracy integration remains necessary.
2. PRELIMINARIES
The preliminaries represent individuals as items with binary protected-group membership and define rankings as bijections assigning each item a position. They also introduce top-i subsets for measuring protected-group representation.
- Each item has an identifier, binary protected-group attribute s, and descriptive attributes x1, . . . , xm.
- The protected items form S+, while all remaining items form S− = I \ S+.
- A ranking τ is a bijection assigning every item a position from 1 through N, with smaller positions preferred.
- The top-i ranking portion is denoted τ 1..i, and S+ 1...i contains protected items appearing within that portion.
Data generator
The data generator creates synthetic rankings with controllable fairness probability, enabling systematic evaluation across different degrees of unfairness. It preserves the original order within each protected-group status.
- The generator takes a ranking τ and produces synthetic ranked datasets spanning varying degrees of fairness.
- The procedure does not change the relative order of two items belonging to the same protected or unprotected group.
- Algorithm 1 initializes an output ranking and repeatedly selects from protected or unprotected sublists according to a random draw and fairness probability.
- The fairness probability controls group mixing: f = 0 places all male individuals ahead, f = 0.3 favors males at top ranks, and f = 0.5 mixes genders equally.
3. FAIRNESS MEASURES
The proposed measures assess whether protected-group membership influences rank position by combining representation checks at multiple cutoffs with greater weight on higher ranks. They are normalized for interpretation.
- Statistical parity in rankings means protected-group membership does not influence an item’s output position.
- The measures compute set-based fairness at discrete cutoffs and combine the values using logarithmic discounting.
- Higher positions receive greater importance, so fairness at the top-10 is weighted more heavily than fairness at the top-100.
- All measures are normalized to [0, 1], with 0 representing the fairest value and 1 the worst.
Normalized discounted difference (rND)
Normalized discounted difference (rND) measures statistical-parity differences between protected-group representation at selected ranking depths and overall population representation, emphasizing higher ranks through logarithmic discounting. On synthetic data, it is lowest when fairness probability matches the protected-group proportion, but its nondifferentiability limits optimization use.
- Definition: rND computes the difference between protected-group proportions at top-i and in the overall population, then accumulates these differences with logarithmic discounting and normalization.The normalizer is the highest possible rND for the given ranking size and protected-group size.
- Synthetic behavior: rND is lowest when fairness probability matches the protected-group proportion: 0.2, 0.5, or 0.8 for 200, 500, or 800 protected members among 1000 items.
- Synthetic behavior: A low proportion of either protected or non-protected members at high ranks produces a high unfairness score, so rND treats the groups symmetrically.
- Optimization: rND is convex and continuous but not differentiable at 0, limiting its usefulness in an optimization framework.
Normalized discounted KL-divergence (rKL)
Normalized discounted KL-divergence (rKL) applies KL-divergence to protected-group proportions at ranking depths versus the overall population, with discounted and normalized aggregation. Its synthetic behavior resembles rND, while its smoother form and extension beyond binary groups support broader optimization and representation analyses.
- Definition: KL-divergence measures the expected logarithmic difference between two discrete probability distributions, P and Q.
- Definition: rKL computes the expected divergence between protected-group membership at top-i and membership in the overall population, then normalizes the discounted values.
- Synthetic behavior: On synthetic datasets, rKL behaves similarly to rND but appears smoother and may therefore be more convenient to optimize robustly.
- Scope: rKL can be used without modification for nonbinary protected-group membership, including proportions of different racial or age groups.
Normalized discounted ratio (rRD)
Normalized discounted ratio (rRD) modifies rND by changing the denominators in its group-representation fractions. Although it reaches its best value at the same points as rND and rKL, its asymmetric behavior restricts its meaningful use to minority protected groups and fairness probabilities below 0.5.
- Definition: rRD is formulated similarly to rND, but uses group sizes in the denominators of its representation fractions rather than the ranking depth i.When a fraction's numerator or denominator is 0, its value is set to 0.
- Behavior: rRD reaches its lowest value at the same fairness-probability points as rND and rKL, but exhibits different trends because it treats the groups asymmetrically.
- Scope limitation: rRD is meaningful only when the protected group is the population minority, comprising at most 50% of the population, and fairness probability is below 0.5.
Evaluation with real datasets
The proposed fairness measures were applied to ProPublica and German Credit rankings, revealing measurable disparities across protected attributes and ranking criteria.
- Evaluation scope: The evaluation compared fairness signals across real datasets, while additional datasets were omitted because of space limitations.The highlighted real datasets were ProPublica and German Credit.
- ProPublica: Gender-based ProPublica rankings had rND values from 0.11 to 0.15, rKL values from 0.01 to 0.02, and rRD values from 0.14 to 0.20.These values covered recidivism, violent recidivism, and prior-arrests rankings.
- German Credit: German Credit rankings used duration, credit amount, selected ordered attributes, and an equal-weight sum of normalized attribute values.The dataset contained financial information about 1,000 loan applicants.
- German Credit: Across German Credit experiments, rKL ranged from 0.01 to 0.15 and rND from 0.05 to 0.41 for gender and age protections.rRD applied only to the age-below-25 protected group and ranged from 0.08 to 0.12.
4. LEARNING FAIR RANKINGS
The paper adapts a fair-representations framework to ranked outputs by optimizing utility and statistical parity. Preliminary German Credit results show low fairness values, with accuracy depending on the ranking criterion.
- Framework: The optimization approach modifies the fair representations framework of Zemel et al. to make it applicable to rankings.The framework introduces an intermediate representation between individual attributes and outcomes.
- Framework: The learned representation is intended to satisfy statistical parity while preserving useful information and predictive accuracy.The formulation uses a multi-criteria loss combining representation, prediction, and parity objectives.
- Evaluation: All fairness measures converged to low values, but accuracy was optimized more effectively for the normalized attribute-sum ranking than for credit-amount ranking.The credit-amount setting retained a considerable score difference.
5. CONCLUSIONS AND FUTURE WORK
The paper concludes that its novel ranked statistical-parity measures were evaluated on synthetic and real datasets, with preliminary optimization results. Future work must establish their real-world applicability and mathematical properties while integrating fairness and accuracy more tightly.
- The paper presents novel measures for quantifying statistical parity in rankings and evaluates them on synthetic and real datasets.
- Preliminary optimization results indicate that the measures can be incorporated into an optimization framework.
- Future work includes assessing applicability in real settings and formally establishing the measures’ mathematical properties.
- The optimization framework must integrate fairness and accuracy more tightly to preserve accuracy while improving fairness.
- Running time is a bottleneck, motivating efforts to make fairness measures more computationally friendly and improve their implementation.