Source-linked AI summary
Fairness in representation: quantifying stereotyping as a representational harm
Mohsen Abbasi, Sorelle A. Friedler, Carlos Scheidegger, Suresh Venkatasubramanian
TL;DR
Fairness research has focused more on allocation harms than on harms caused by how people are represented before training. This paper formalizes stereotyping mechanisms and proposes mitigation; on synthetic data, stereotyping skews outcomes and the proposed strategies can largely remove its effects.
Problem
Harms of representation have received less attention than allocation disparities, while stereotyping can distort how group members are represented.
Source: §1 · p. 1Method
Results
As α increases, Rand index decreases and clustering becomes less balanced; synthetic-data experiments also show mitigation can largely remove added stereotype effects.
Source: §7 · p. 8Takeaways & Limitations
Stereotyping can skew outcomes when stereotyped data representations are used in a machine-learning pipeline.
Source: §7 · p. 8The paper does not formally analyze clustering because its global objective makes perturbation effects difficult to predict, and its WAE assumption may not hold in every scenario.
Source: §4.3 · p. 5, §5 · p. 5
Abstract
from arXiv · showhide
While harms of allocation have been increasingly studied as part of the subfield of algorithmic fairness, harms of representation have received considerably less attention. In this paper, we formalize two notions of stereotyping and show how they manifest in later allocative harms within the machine learning pipeline. We also propose mitigation strategies and demonstrate their effectiveness on synthetic datasets.
1 Introduction
Algorithmic fairness has chiefly studied allocation disparities, while harms can also arise from how people are represented in feature space before a model is trained. This paper focuses on quantifying stereotyping as representational distortion and proposes formal mechanisms and mitigation strategies.
1 Introduction
- Fairness research often asks whether decisions differ between social groups, framing discrimination as a harm of allocation.Source: §1 · p. 1
- The paper treats stereotyping as representational distortion: generalizing characteristics from a few group members to all members.It applies a general framework for measuring distortion specifically to stereotyping.Source: §1 · p. 1
- The paper formalizes a mechanism for stereotyping as a function from construct space to observed space, with probabilistic and geometric interpretations.The probabilistic interpretation aligns with psychological literature on how people form stereotypes.Source: §1 · p. 1
- Stereotyping can affect model building, and the paper demonstrates its effects.Source: §1 · p. 1
- The paper proposes mitigation that attempts to invert stereotyping and presents experimental evidence of its effectiveness.Source: §1 · p. 1
2 Literature review
The literature describes stereotyping as overgeneralizing about groups and models it through several cognitive mechanisms. Related work on algorithmic fairness largely focuses on outcome disparities or on representations adjusted to guide learning, rather than bias in representation itself.
2 Literature review
- Psychological accounts describe stereotypes as overgeneralized group beliefs that can cause people to overlook individual differences and contribute to prejudice.Source: §2 · pp. 1–2
- Models of stereotyping include prototypes, exemplars, associative networks, schemas, and distorted perceptions of a type's relative frequency.The representativeness model draws on the representativeness heuristic.Source: §2 · p. 2
- Statistical discrimination models employers using group-average productivity as a proxy when they cannot assess an individual worker's productivity.This account treats stereotyping as a response to insufficient individual information, rather than as a potentially distorting representation choice.Source: §2 · p. 2
- Algorithmic-fairness work has generally measured bias through outcome-based measures or modified input representations to guide fair learning.Examples include preprocessing to reverse training-data skew [26] [6] [16] and fair clustering [12, 31, 3] and PCA [30, 28].Source: §2 · p. 2
- Some work studies bias in learned word-vector representations by showing how language biases persist after embedding.Source: §2 · p. 2
3 Modeling Stereotypes
The paper models stereotyping through geometric and probabilistic mechanisms, then places them in a shared algebraic framework. These mechanisms differ in how they can enter data, including during collection or feature selection.
3 Modeling Stereotypes
- Each mechanism has an associated measure, with larger values indicating greater stereotyping.Source: §3 · p. 2
3.1 Stereotyping via Exemplars.
- Exemplar stereotyping generalizes features attributed to a small subset of a group to all its members by drawing points toward an exemplar.In the figure, points in the stereotyped group appear closer to the exemplar in the observed representation.Source: §3 · p. 2, Figure 1 · p. 2, §3.1 · p. 3
- Exemplar stereotyping can affect only a subspace of relevant features, leaving other dimensions uninfluenced.The mechanism reorders features so the stereotyped dimensions are first.Source: §3.1 · p. 3
3.2 Stereotyping via Representativeness.
- The probabilistic mechanism amplifies perceived probabilities according to how representative a type is, with ρ controlling the strength of that influence.The specified weighting function is h(x) = x^ρ; larger ρ means representativeness has more influence.Source: §3.2 · p. 3
3.3 A unified view of stereotyping.
- A shared linear framework expresses stereotyping as a transformation of feature representations and recovers geometric and probabilistic mechanisms as instances.The framework uses an invertible mapping into a feature space and transformation parameters A and B.Source: §3.2 · p. 3
- The mechanisms have different possible entry points: representativeness can affect data collection, while exemplar and feature stereotyping can arise during feature selection.The examples include increased observation of crimes associated with minorities and omitted differentiating features that make points appear closer.Source: §3.3 · p. 4
4 Harms of Stereotyping
The section analyzes how stereotyping in data representations can produce group disparities in machine-learning outcomes. It examines the mechanism in Naive Bayes and linear regression, and discusses clustering qualitatively.
4 Harms of Stereotyping
- Stereotyped representations can lead to group disparities in Naive Bayes classification, linear regression, and clustering.The section analyzes Naive Bayes under the probabilistic interpretation and regression and clustering under the geometric interpretation.Source: §4 · p. 4
4.1 Naive Bayes
- Stereotyping changes Naive Bayes feature probabilities by conditioning a relevant feature on the sensitive attribute, and the classification effect relates directly to representativeness.The perceived probability for the target group may differ from its actual value; the degree of difference is related to the ratio of perceived to actual probabilities.Source: §4.1 · p. 4
4.2 Linear Regression.
- The regression analysis studies how stereotyping-induced input perturbations change least-squares coefficients, a setting distinct from the cited analyses of random input noise.The authors state that existing analyses of random perturbations do not directly apply to their structured perturbations.Source: §4.2 · p. 4
- For structured regression perturbations, coefficient component p1 changes linearly with α, while p2 changes quadratically.The perturbation is applied to a single coordinate while the remaining coordinates and dependent variable stay fixed.Source: §4.2 · pp. 4–5
- Clustering is not analyzed formally: moving points toward an exemplar can make clusters more homogeneous by group identity, while an outlier exemplar can increase k-means cost.The cost increase is related to the exemplar’s distance from the cluster mean and the number of points moved.Source: §4.3 · p. 5
5 Mitigating the Effects of Stereotyping
The mitigation methods use a We’re All Equal (WAE) assumption to estimate a less-stereotyped representation from observed data. They choose conservative reconstructions for exemplar-based stereotyping, its feature-based variant, and representativeness.
5 Mitigating the Effects of Stereotyping
- The methods assume groups would have essentially the same distribution before stereotyping, using the majority group to estimate the minority group’s original representation.The paper notes that this WAE assumption may not hold in every scenario, such as average height differences between women and men.Source: §5 · p. 5
5.1 Mitigation of exemplar-based stereotyping.
- For exemplar-based mitigation, feasible exemplars lie in a cone constrained by the WAE bound, and the method selects the candidate pair (c, α) with minimum α.The minimum α is the most conservative stereotyping measure consistent with the observed data and WAE assumption; Figure 2 illustrates candidate c, α computation.Source: §5.1 · pp. 5–6, Figure 2 · p. 6
5.1.1 Mitigating stereotyping with respect to features.
- The feature-based mitigation extends the exemplar approach to a k-dimensional problem and assumes majority and minority type probabilities are similar within a bound.The stated bound also implies that the KL-divergence between the two conditional distributions is at most ϵ.Source: §5.1.1 · p. 6
5.2 Mitigating Representativeness.
- For representativeness mitigation, feasible ρ values begin at the bound derived from the maximum observed λ′(t), and the smallest feasible ρ gives the most conservative reconstruction.Each feasible ρ can be used to reconstruct the original probabilities p(t|G); the KL-divergence decreases monotonically with ρ.Source: §5.2 · p. 6
6 Experiments
Synthetic experiments examine stereotyping and mitigation in Naive Bayes classification, linear regression, and K-means clustering. Across these settings, the reported results show stereotype effects and reductions from the corresponding mitigation strategies.
6 Experiments
- Boosting representativeness increases selected Asians without changing the other group’s results, and representativeness mitigation significantly reduces the effect.At large λ(t) and ρ, type probabilities may saturate, preventing the method from recovering the original probabilities.Source: §6.1 · p. 7, Figure 3 · p. 7
6.2 Linear
- Stereotyping creates disparity in the groups’ regression values, while exemplar-based mitigation reduces that disparity.The regression coefficients remain roughly the same as α increases.Source: §6.2 · p. 7, Figure 4 · p. 7
6.3 Clustering.
- Stereotyping strongly affects K-means clustering at larger α, while mitigation removes that effect under both Rand-index and balance measures.The fair-clustering solution of Chierichetti et al. achieves fair clustering but increases clustering cost in the presence of stereotyping.Source: §6.3 · p. 8, Figure 5 · p. 8
7 Discussion and Conclusion
The paper concludes that stereotyped representations can skew outcomes in machine-learning pipelines and that the proposed mitigation strategies can largely remove added stereotype effects in synthetic-data experiments. It also identifies limits in the stereotyping models considered.
7 Discussion and Conclusion
- As α increases, the Rand index decreases and clustering becomes less balanced.These outcomes are identified in the Figure 6 panel descriptions.Source: §7 · p. 8, Figure 6 · p. 8
- Experiments on synthetic data show that the proposed mitigation strategies can largely remove stereotype effects added to the data.The paper reports formalizing two notions of stereotyping and demonstrating skewed outcomes when stereotyped representations enter a machine-learning pipeline.Source: §7 · p. 8
- The framework uses a single exemplar, and its geometric exemplar model has not been validated with human subjects.The paper also notes that other cognitive models and stereotype effects varying across individuals are not included.Source: §7 · p. 8