Source-linked AI summary
Reducing Disparate Exposure in Ranking: A Learning To Rank Approach
Meike Zehlike, Carlos Castillo
TL;DR
Rankings can shape opportunities, while biased training data can yield disparate exposure for protected groups. DELTR adds a listwise fairness objective to learning-to-rank so it can optimize relevance and reduce exposure disparities during training. The experiments show that its fairness objective need not reduce accuracy, and relevance can increase in non-discrimination settings.
Problem
Ranking models trained on biased data can reproduce unequal group exposure, which may translate into unequal access to opportunities.
Source: §1 · pp. 1–2Method
DELTR extends ListNet by combining an accuracy loss with a listwise objective that reduces disparate exposure during training.
Source: §4 · p. 3Results
Aiming for listwise fairness increases relevance in non-discrimination cases, and the objective does not necessarily trade off against accuracy.
Source: §7 · p. 6Takeaways & Limitations
Colorblindness can be among the best or worst choices for relevance and fairness depending on the amount and type of training-data bias.
Source: §1 · p. 2The approach assumes two groups, one protected, although its exposure criterion can be extended to multiple protected groups.
Source: §4 · pp. 2–3
Abstract
from arXiv · showhide
Ranked search results have become the main mechanism by which we find content, products, places, and people online. Thus their ordering contributes not only to the satisfaction of the searcher, but also to career and business opportunities, educational placement, and even social success of those being ranked. Researchers have become increasingly concerned with systematic biases in data-driven ranking models, and various post-processing methods have been proposed to mitigate discrimination and inequality of opportunity. This approach, however, has the disadvantage that it still allows an unfair ranking model to be trained. In this paper we explore a new in-processing approach: DELTR, a learning-to-rank framework that addresses potential issues of discrimination and unequal opportunity in rankings at training time. We measure these problems in terms of discrepancies in the average group exposure and design a ranker that optimizes search results in terms of relevance and in terms of reducing such discrepancies. We perform an extensive experimental study showing that being "colorblind" can be among the best or the worst choices from the perspective of relevance and exposure, depending on how much and which kind of bias is present in the training set. We show that our in-processing method performs better in terms of relevance and exposure than a pre-processing and a post-processing method across all tested scenarios.
1 INTRODUCTION
Rankings can shape opportunities as well as searcher utility, and biased data may reproduce unequal exposure for disadvantaged groups. The paper introduces DELTR, an in-processing learning-to-rank approach that addresses disparate exposure while preserving relevance.
1 INTRODUCTION
- Rankings of people, businesses, and places can affect access to economic or social opportunities through systematic differences in exposure.Disparate impact concerns loss of opportunity for a protected group, regardless of intentional differential treatment.Source: §1 · p. 1
- Unequal exposure can arise from positional bias, legal visibility requirements, differences in how documents are constructed, or biased ground-truth rankings.The paper distinguishes reducing historical disadvantage (equal opportunity) from allocating resources without regard to irrelevant attributes (non-discrimination).Source: §1 · p. 1
- DELTR is an in-processing, listwise learning-to-rank approach that reduces disparate impact during training, unlike post-processing methods that reorder rankings afterward.The paper also introduces a listwise fairness metric and experiments on expert search and student-performance ranking.Source: §1 · p. 2
- Reducing bias against a protected group can increase relevance, challenging the assumption that fairness and relevance must always trade off.The paper notes that post-processing still permits an unfair model to be trained.Source: §1 · p. 2
- Being colorblind can be among the best or worst choices for relevance and fairness, depending on the amount and type of bias in training data.The experiments also find that pre-processing can be difficult: bias may be embedded in all attributes, and fair reordering can reduce performance or produce reverse discrimination.Source: §1 · p. 2
2 RELATED WORK
Fair-ranking work has often adjusted rankings after scoring, whereas DELTR learns a ranking function with an added objective that reduces disparate exposure. Its listwise focus addresses disparate impact through group exposure rather than disparate treatment alone.
2 RELATED WORK
- Earlier fair-ranking methods commonly post-process scores by re-ranking items to meet a fairness objective.Fairness in ranking also concerns groups' representation and treatment across positions, where attention decreases rapidly lower in the ranking.Source: §2 · p. 2
- DELTR extends ListNet with an in-processing objective that reduces disparate exposure, directly optimizing the exposure a protected group receives.The paper presents it as the first listwise in-processing approach for reducing discrimination and inequality of opportunity in rankings.Source: §2 · p. 2
- Unlike the cited pairwise approach focused on disparate treatment, DELTR targets disparate impact and does not incorporate user feedback.The authors treat user feedback as a separate potential source of unconscious bias to study.Source: §2 · p. 2
3 BACKGROUND: LISTNET IN A NUTSHELL
ListNet makes listwise ranking tractable by representing judgments as top-position probabilities rather than optimizing over document permutations. It learns a ranking function by minimizing the loss between training and predicted judgments.
3 BACKGROUND: LISTNET IN A NUTSHELL
- ListNet learns a ranking function that predicts judgments for document feature vectors and minimizes the loss between training and predicted judgments.The standard objective sums losses across queries.Source: §3 · p. 2
- Instead of searching over document permutations, which has exponential execution time, ListNet focuses on each document's probability of appearing at the top position.These top-one probabilities form a probability distribution over the documents in a query.Source: §3 · p. 2
- ListNet measures the loss between ground-truth and predicted top-one probabilities using cross entropy.The passage attributes this loss formulation to Cao et al..Source: §3 · p. 2
4 DELTR: DISPARATE EXPOSURE IN LEARNING TO RANK
DELTR combines a listwise accuracy loss with a disparity penalty based on average group exposure, using a weight to balance them. The method is framed around two groups, with a protected group whose exposure may be disadvantaged, and the penalty does not reward overcompensation.
4 DELTR: DISPARATE EXPOSURE IN LEARNING TO RANK
- DELTR assumes two social groups, one protected, and seeks rankings that are relevant without disparate exposure.The training input consists of queries with ordered item lists; testing supplies a query and document collection.Source: §4 · pp. 2–3
- Biased training data may cause rankings to reproduce or reinforce protected groups' exposure disadvantage and unequal opportunities.The paper identifies historical discrimination and erratic data collection as possible sources of disadvantage.Source: §4 · p. 3
- The fairness measure adapts document exposure to ListNet's top-one probabilities and defines group exposure as average exposure across group members.Position 1 is weighted by its relative importance to users of the ranking system.Source: §4 · p. 3
- The squared-hinge criterion seeks to prevent protected-group exposure from falling below non-protected-group exposure, but does not penalize the reverse disparity.When the protected group already receives at least as much exposure, the objective optimizes only for relevance.Source: §4 · p. 3
- The fairness objective can extend to multiple protected groups by comparing each protected group's average or maximum exposure difference with the non-protected group.In synthetic tests, increasing γ raised protected-group exposure only until group exposure was equal, without overcompensation.Source: §4 · p. 3
- DELTR minimizes a weighted combination of the accuracy loss and listwise fairness measure, with γ controlling their trade-off.Larger γ emphasizes reducing disparate exposure; smaller γ emphasizes matching the training data.Source: §4 · p. 3
5 EXPERIMENTS
Experiments use three real-world datasets to test DELTR across different sources of bias and compare it with colorblind, standard, pre-processing, and post-processing rankers. The evaluation varies DELTR’s γ and includes distinct dataset constructions and FA*IR baseline settings.
5 EXPERIMENTS
- Three real-world datasets test non-discrimination under unrelated bias and group score differences at equal relevance, plus substantive equality of opportunity under pre-existing utility differences.The study expects utility and list-wise fairness to be jointly achievable in the first two cases and tests two γ values in each dataset.Source: §5 · p. 3
- The W3C experts dataset simulates bias by ordering equally expert candidates as male experts, female experts, male non-experts, then female non-experts.Its source task retrieves experts for a topic from candidates’ email corpus.Source: §5 · pp. 3–4
- The engineering-student data uses PSU scores, high-school grades, and first-year credits, while the law-student data samples 10% while preserving gender and ethnicity.The law dataset has one query, with training and test sets split 80/20.Source: §5 · p. 4, Table 1 · p. 4
- The comparison includes colorblind and all-feature ListNet rankers, FA*IR pre-processing and post-processing baselines, and DELTR with small and large γ values.Pre-processing applies FA*IR to training data before fitting ListNet; post-processing applies it to ListNet’s predicted rankings.Source: §5 · pp. 3–4
- FA*IR pre-processing tests target proportions p*=the protected-candidate ratio, p+=p*+0.1, and p−=p*−0.1; post-processing uses the same settings.FA*IR enforces a minimum protected-group proportion at every ranking prefix.Source: §5 · p. 4
6 EXPERIMENTAL RESULTS
Across the tested settings, whether colorblind training helps depends on the kind of bias in the training data. DELTR generally offers a better relevance–exposure balance than pre- and post-processing when those objectives trade off.
6.1 Bias Unrelated to Utility – W3C Experts
- With bias unrelated to expertise, colorblind LTR achieved the best relevance and near-equal exposure, while DELTR had the best relevance among fair algorithms.A too-small p did not affect exposure, while a too-large p could over-represent women and reduce relevance; DELTR optimizes relevance and fairness together.Source: §6.1 · pp. 4–5
6.2 Bias due to Different Score Distributions – Engineering Students (high school type)
- With public-school students protected, colorblind LTR ranked among the worst in relevance and exposure, while standard LTR was among the best; DELTR handled both bias types without knowing their nature.At equal admission-test scores, public-school students tended to have higher university grades.Source: §6.2 · p. 5
6.3 Achieving Substantive Equal Opportunity
- Across substantive-equality tests, DELTR consistently outperformed pre- and post-processing in accuracy and list-wise fairness, losing less relevance at matched exposure without over-representation.FA*IR’s too-small p often had no mitigation effect, while a too-large p could over-represent protected candidates and sharply reduce relevance.Source: §6.3 · p. 6
- Including protected features improved accuracy but reduced exposure in the gender tests, whereas in the race test it reduced both accuracy and exposure.The authors suspect the race-test reversal reflects widely separated ground-truth performance distributions, but note that this effect is not further studied.Source: §6.3 · p. 6
7 CONCLUSIONS
LTR models can reproduce and exaggerate group-visibility disparities present in training data. DELTR adds a listwise fairness objective to ListNet, and experiments show that fairness need not trade off with relevance.
7 CONCLUSIONS
- DELTR extends ListNet with a listwise fairness objective that reduces the extent to which protected elements receive less exposure.Source: §7 · p. 6
- Experiments show the additional fairness objective does not necessarily trade off with accuracy and can increase relevance in non-discrimination cases.Source: §7 · p. 6
- The study examines achieving non-discrimination by either excluding or including the protected feature and analyzes the underlying biases in each case.Source: §7 · p. 6
- Because the bias present is difficult to know in advance, DELTR offers an approach intended to handle both situations.Source: §7 · p. 6
- LTR models can reproduce and exaggerate discrepancies in average group visibility found in training data.Source: §7 · p. 6
- The fairness parameter γ offers flexibility, but systematic guidance for setting it remains future work; the multiple-protected-group extension also lacks experimental validation.Source: §7 · p. 6