Source-linked AI summary
Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations
Alex Beutel, Jilin Chen, Zhe Zhao, Ed H. Chi
TL;DR
The paper asks how to learn fair classifiers when sensitive attributes are difficult to obtain during training or unavailable at serving. It uses adversarial training to remove sensitive information from latent representations and studies how adversarial-data choices affect fairness. The authors find that small adversarial samples can help, while balanced sensitive-attribute distributions substantially improve fairness.
Problem
Fair classification is difficult when sensitive attributes are costly or impossible to observe during training or model serving, especially with limited or skewed protected-group data.
Method
A multi-head neural network predicts the target while adversarial training prevents its latent representation from encoding the sensitive attribute.
Results
Small adversarial samples can improve fairness, and balanced distributions over the sensitive attribute significantly improve fairness compared with random samples.
Takeaways & Limitations
Adversarial-data quantity and distribution are important design choices because the distribution empirically drives the adversary’s fairness notion.
Takeaways & Limitations
The method assumes that the sensitive feature is observable for at least a subset S of inputs and may correlate with observed features.
Abstract
from arXiv · showhide
How can we learn a classifier that is "fair" for a protected or sensitive group, when we do not know if the input to the classifier belongs to the protected group? How can we train such a classifier when data on the protected group is difficult to attain? In many settings, finding out the sensitive input attribute can be prohibitively expensive even during model training, and sometimes impossible during model serving. For example, in recommender systems, if we want to predict if a user will click on a given recommendation, we often do not know many attributes of the user, e.g., race or age, and many attributes of the content are hard to determine, e.g., the language or topic. Thus, it is not feasible to use a different classifier calibrated based on knowledge of the sensitive attribute. Here, we use an adversarial training procedure to remove information about the sensitive attribute from the latent representation learned by a neural network. In particular, we study how the choice of data for the adversarial training effects the resulting fairness properties. We find two interesting results: a small amount of data is needed to train these adversarial models, and the data distribution empirically drives the adversary's notion of fairness.
1 INTRODUCTION
The paper addresses fair classification when sensitive attributes are difficult to observe during training or unavailable at serving. It studies adversarial de-biasing, focusing on data requirements and how adversarial-data distributions affect fairness.
- Sensitive attributes may be private, imprecisely defined, expensive to label, or unavailable when predictions are served.Skewed distributions can make it especially difficult to obtain enough protected-group examples.
- De-biasing methods must account for limited and skewed sensitive-attribute data during training.Otherwise, the de-biasing process can reproduce the same data problems as the original model.
- The proposed approach jointly predicts the target while preventing a second network head from predicting the sensitive attribute.The adversarial procedure removes sensitive-attribute information from the learned latent representation.
- The paper connects fairness definitions to adversarial objectives and the dataset used for adversarial training.
- The study evaluates how much adversarial data is needed and how its distribution affects resulting fairness.
2 RELATED WORK
Related work defines competing fairness criteria, proposes post-processing methods requiring sensitive attributes, and develops fair representations through adversarial training. The paper identifies limited precision about optimized fairness definitions and adversarial data choices.
- Fairness research contrasts demographic parity with formulations that focus more closely on model accuracy.
- Hardt et al. achieve equality of opportunity through post-processing that takes predictions and the sensitive attribute as input.
- Existing calibration approaches are problematic when the sensitive attribute is unobservable at inference time.
- Prior adversarial fairness work seeks to prevent biased latent representations but is generally imprecise about its fairness definition and adversarial-objective data.
3 MODEL STRUCTURE AND LEARNING
The model learns a target prediction from an embedding while an adversarial classifier predicts a sensitive feature from that embedding using labeled sensitive-attribute data for a subset of examples. A negative-gradient identity reverses the adversary’s signal for the embedding.
- Model Structure: The primary model represents predictions as Y = f(g(X)), where g produces embedding h and f maps h to the target prediction.
- Model Structure: The sensitive feature Z may correlate with observed inputs even when Z is excluded from the embedding model’s direct inputs.
- Model Structure: For a subset S with observed Z, an adversarial classifier a(g(S)) predicts Z from the same hidden embedding h.
- Learning Algorithm: The primary loss predicts Y, while the adversarial loss trains a to predict Z and trains g to remove information useful for predicting Z.
- Learning Algorithm: A negative-gradient identity prevents minimizing LY + LZ from encouraging the representation to encode Z.
- Learning Algorithm: λ controls the trade-off between target accuracy and removing sensitive-attribute information.
4 DATA SELECTION & FAIRNESS DEFINITION
The distribution of adversarial-training data over target Y determines which fairness relationship the representation is encouraged to satisfy. Using both target classes supports demographic parity, while conditioning adversarial training on one class yields equality-of-opportunity variants.
- The distribution of adversarial dataset S over Y and Z is a central design choice because it maps training procedures to different fairness definitions.
- Using adversarial data containing both Y = 1 and Y = 0 encourages h to be uncorrelated with Z and supports demographic parity.This yields P(Ŷ = 1|Z = 1) = P(Ŷ = 1|Z = 0).
- Training the adversary only on Y = 1 removes topic information conditionally for that class while allowing features relevant to dangerous content to remain encoded.
- Training only on Y = 0 enforces the reciprocal equality-of-opportunity relation for the negative class.
- The paper empirically tests how adversarial data distributions affect accuracy and fairness metrics.
5 EXPERIMENTS
The experiments vary adversarial-training data across sensitive-attribute balance, primary-label distribution, and dataset size. Balanced sensitive-attribute data strengthens and stabilizes fairness improvements, while primary-label distributions produce different fairness outcomes and small samples can suffice.
- 5 EXPERIMENTS: The experiments evaluate accuracy and fairness while varying sensitive-attribute distribution, primary-label distribution, and adversarial dataset size.The study uses the Adult dataset and reports demographic-parity and equality-of-opportunity measures alongside accuracy.
- 5 EXPERIMENTS: Without an adversarial head, the baseline accuracy is 0.8233 and Parity is 0.1911.The baseline also reports Equality≤50K = 0.1076 and Equality>50K = 0.0589.
- 5.1 Skew in Sensitive Attribute: Balanced sensitive-attribute data produces stronger, more stable fairness improvements across metrics, while also reducing accuracy.The same training procedure yields smaller standard deviations with balanced data.
- 5.2 Skew in Primary Label: Different primary-label distributions produce significantly different fairness results.High-income examples most improve equality of opportunity for the low-income label, while using both groups helps across all metrics.
- 5.3 Amount of Data: 500 sensitive-attribute examples, or 1.5% of training data, significantly affect fairness metrics in most cases.A strong effect remains in a conservative setting using only low-income, gender-balanced test samples.
6 DISCUSSION
The discussion reports that small adversarial samples can improve fairness, while balanced sensitive-attribute data improves fairness more substantially. The latent-representation objective may impose a stronger condition than equality of outcomes, creating an accuracy–fairness trade-off controlled by λ.
- 6 DISCUSSION: Small samples of adversarial examples can improve model fairness.The authors describe this as an encouraging result for settings where sensitive attributes are difficult to observe.
- 6 DISCUSSION: Balanced sensitive-attribute distributions significantly improve fairness, though they may require more time or complex techniques.The discussion contrasts this benefit with the practical cost of constructing balanced data.
- 6 DISCUSSION: Removing sensitive information from latent representations appears stronger than enforcing equality of outcomes, potentially reducing accuracy.More sensitive tuning of λ can produce more amenable accuracy–fairness trade-offs.
7 CONCLUSION
The paper connects fairness definitions to adversarial-training data distributions and finds that balanced sensitive-attribute data and remarkably small adversarial datasets can improve fairness.
- The paper connects varying theoretical definitions of fairness to adversarial training procedures using different data distributions.
- Balanced distributions over the sensitive attribute are more effective for adversarial training than random samples.
- The adversarial training data distribution empirically connects to the fairness metrics obtained by the model.
- Remarkably small datasets can effectively encourage fairer latent representations during adversarial training.