Source-linked AI summary
Fairness without Demographics through Adversarially Reweighted Learning
Preethi Lahoti, Alex Beutel, Jilin Chen, Kang Lee, Flavien Prost, Nithum Thain, Xuezhi Wang, Ed H. Chi
TL;DR
The paper asks how to improve fairness when protected features are unavailable because privacy, legal, or regulatory constraints prevent their collection or use. It proposes Adversarially Reweighted Learning, which uses observed features, labels, and computationally identifiable error regions to reweight training examples. ARL improves worst-case protected-group AUC across multiple datasets and outperforms state-of-the-art alternatives, though it depends on identifiable group signal and can degrade with noisy labels.
Problem
Fairness methods often assume protected features are available, but practical restrictions can prevent their use during training or inference.
Method
ARL trains an adversary over observed features and labels to identify high-error regions and reweight examples for improving performance over unobserved protected groups.
Results
ARL improves worst-case protected-group AUC across datasets, including a 6.5 percentage point improvement over baseline for Adult, and outperforms DRO.
Takeaways & Limitations
Improving worst-off group performance can provide a fairness objective distinct from merely reducing performance gaps between groups.
Takeaways & Limitations
ARL relies on protected groups being computationally identifiable, and its performance drops when training labels are noisy.
Abstract
from arXiv · showhide
Much of the previous machine learning (ML) fairness literature assumes that protected features such as race and sex are present in the dataset, and relies upon them to mitigate fairness concerns. However, in practice factors like privacy and regulation often preclude the collection of protected features, or their use for training or inference, severely limiting the applicability of traditional fairness research. Therefore we ask: How can we train an ML model to improve fairness when we do not even know the protected group memberships? In this work we address this problem by proposing Adversarially Reweighted Learning (ARL). In particular, we hypothesize that non-protected features and task labels are valuable for identifying fairness issues, and can be used to co-train an adversarial reweighting approach for improving fairness. Our results show that {ARL} improves Rawlsian Max-Min fairness, with notable AUC improvements for worst-case protected groups in multiple datasets, outperforming state-of-the-art alternatives.
1 Introduction
The paper addresses fairness when protected features cannot be collected or used, proposing ARL to improve utility for worst-off protected groups using observable correlates and error regions.
- Motivation: Fairness research commonly assumes access to protected features, although privacy, legal, and regulatory restrictions can make collecting or using them infeasible.This creates a practical need for fairness methods that operate without protected group memberships.
- Motivation: The central question is how to improve fairness when protected features are unavailable during both training and inference.
- Fairness objective: Rawlsian Max-Min fairness seeks to maximize minimum expected utility across protected groups, unlike parity notions that minimize performance gaps.The paper argues that improving worst-off groups can be preferable to reducing gaps by lowering better-performing groups’ accuracy.
- Approach: ARL uses an adversary over observed features and labels to identify computationally identifiable error regions and reweight examples to improve unobserved groups’ worst-case performance.The approach relies on correlations between protected groups, observed features, and class labels.
- Empirical benefits: Across three real-world datasets, ARL improves AUC for worst-case protected groups, outperforms state-of-the-art alternatives, and improves overall AUC on two datasets.The paper also analyzes learned example weights and robustness to representation bias and group base-rate differences.
2 Related Work
Prior fairness methods typically require protected attributes or approximate them with proxies, while ARL addresses demographic-free fairness through computationally identifiable error regions.
- Fairness approaches: Fairness methods include individual fairness, group fairness based on parity, and approaches such as Rawlsian Max-Min fairness that improve per-group performance.
- Protected-attribute access: Most existing methods assume protected attributes are available to optimize fairness metrics directly, although such information may be missing or sparse in practice.
- Fairness without demographics: Prior demographic-free approaches use proxy features, perturbed attributes, transfer learning, federated learning, or trusted parties holding protected data.Proxy-based approaches can be prone to estimation bias, while privacy-preserving approaches may still provide models with protected features.
- Closest prior work: DRO optimizes worst-case performance over any distribution exceeding a size threshold, whereas ARL targets computationally identifiable groups through observed inputs and labels.
- Modeling inspirations: ARL draws technical inspiration from re-weighting, adversarial learning, focal loss, and domain adaptation under covariate or label shift.
3 Model
ARL formulates fairness without observed protected memberships as maximizing the worst-group utility, then approximates this objective with an alternating learner–adversary minimax game. The adversary identifies high-loss regions from non-protected features and labels, reweights them, and trains the learner to improve performance there.
- Problem formulation: The problem is to learn a binary classifier that is fair to unobserved protected groups using only non-protected features and labels.Protected memberships are unavailable at both training and inference time, although fairness is evaluated with respect to those groups.
- Fairness objective: Rawlsian Max-Min fairness maximizes the minimum utility across protected groups, with utility interpreted through standard accuracy metrics.The evaluation uses AUC as utility and reports the minimum protected-group utility as AUC(min).
- Adversarial reweighting: Because protected memberships are unavailable, ARL replaces direct group-weight optimization with an adversary that identifies computationally-identifiable high-loss regions.Computational identifiability is defined through a function over X × Y that recognizes subgroup membership; ARL uses a neural network whose outputs increase in higher-loss regions.
- Minimax optimization: ARL trains a learner and adversary alternately: the learner minimizes classification loss while the adversary maximizes weighted loss through example weights.This forms a zero-sum minimax game in which the adversary assigns weights to emphasize regions where the learner makes significant errors.
- Model design: The adversary’s expressiveness controls the granularity of upweighting, with finer-grained models also risking overfitting to outliers.The paper notes that any differentiable model can implement the adversary, while its experiments found a linear adversary effective on small academic datasets.
- Implementation: The experiments implement both learner and adversary as feed-forward networks and summarize their interaction in the ARL computational graph.The learner uses two fully connected hidden layers, while the experiments report that a linear adversary performed best on the small academic datasets.
4 Experimental results
Across Adult, LSAC, and COMPAS, ARL improves worst-case subgroup performance without protected features, often matching or exceeding reweighting and group-fairness alternatives. ARL also shows robustness to training-data biases, while noisy labels constrain performance.
- Robustness and evaluation: ARL is robust to representation bias and differences in group base-rate but degrades with noisy ground-truth labels.The experiments use three real-world datasets and evaluate performance with AUC-based subgroup metrics.
- Main results: ARL outperforms DRO and achieves the best AUC (minority) on all three datasets.Improvements over the baseline are 6.5 pp for Adult, 0.8 pp for LSAC, and 1.1 pp for COMPAS.
- Utility-fairness trade-off: ARL improves overall AUC by approximately 1 pp on both Adult and LSAC.The improvement is reported for AUC (avg) and AUC (macro-avg).
- Comparison with reweighting: ARL achieves comparable or better results than both IPW variants on all datasets without demographic features.The learned adversarial weights improve protected-group errors despite the absence of group labels.
- Comparison with group-fairness methods: ARL substantially outperforms Min-Diff on Adult and COMPAS and achieves comparable performance on LSAC.Min-Diff reduces FPR gaps, but lower gaps do not consistently improve AUC for worst-off groups.
- Utility-fairness trade-off: Min-Diff incurs a 5 pp overall-AUC drop on Adult and a 2 pp drop on COMPAS, whereas ARL improves overall AUC on Adult and LSAC.The paper attributes this difference to ARL’s objective of improving protected-group performance while preserving overall utility.
5 Analysis
The analysis examines whether protected groups can be identified from observed features and labels, how ARL weights examples, and how robust ARL is to training-data bias. It finds that ARL benefits computationally identifiable groups and representation-bias settings but is sensitive to noisy labels.
- Computational identifiability: Adult and LSAC show significant correlations between observed features, labels, and unobserved protected groups, whereas COMPAS groups are less computationally identifiable.The authors relate these patterns to improvements for Adult and LSAC and no gain or loss for COMPAS.
- Robustness to training distributions: Both DRO and ARL are robust to representation bias, with ARL outperforming DRO and the baseline at all tested points.When female examples are over- or under-sampled at fixed training-set size, ARL and DRO cope better than the baseline with the resulting information loss.
- Robustness to training distributions: ARL and DRO perform nearly identically when fewer than 0.2 of training labels are incorrect, but DRO declines substantially beyond that outlier fraction.Both methods are sensitive to label bias because they upweight examples with prediction errors.
- Learnt example weights: Misclassified examples are upweighted by ARL, while correctly classified examples are generally not.ARL also assigns high weights to correctly classified class-1 examples in the imbalanced Adult dataset, where class 1 comprises 23% of examples.
- Learnt example weights: At a female class-1 base-rate of 0.1, ARL assigns significantly higher mean weights to class-1 than class-0 examples.The experiment varies the female subgroup’s training base-rate and measures mean subgroup weights.
6 Conclusion
The paper addresses fairness without directly observing protected features by focusing worst-case protected-group performance. It presents ARL as a foundation for improving AUC across protected groups and training-data biases without demographic access.
- Conclusion: ARL focuses on computationally identifiable regions of errors to improve AUC for worst-case protected groups without directly observing protected features.The conclusion describes this as the paper’s key insight for pursuing fairness without access to demographics.
- Conclusion: Across multiple datasets and training-data biases, ARL is reported to improve AUC for worst-case protected groups.The conclusion contrasts this with prior worst-case-distribution methods’ vulnerability to noisy outliers.
7 Broader Impact
The paper frames fairness without demographic features as a practical challenge created by tension between fairness objectives and privacy constraints. It also identifies evaluation and generalization limits for demographic-blind fairness methods.
- Broader impact: Many fairness methods require protected demographic features, creating tension between fairness goals and privacy, legal, or regulatory restrictions.The paper motivates methods that improve worst-case protected-group performance without protected group information in the dataset.
- Broader impact: Evaluating effectiveness in real applications is difficult when demographic information is unavailable.The authors state that further challenges remain in evaluating debiasing methods under this setting.
- Broader impact: ARL relies on protected groups being computationally identifiable from remaining features and labels, and its performance drops with noisy training labels.The authors call for validation across real-world applications beyond the studied datasets.
8 Supplementary Material
The supplementary material analyzes ARL variants, experimental comparisons, datasets, preprocessing, model baselines, and evaluation procedures. Results emphasize the competitiveness of group-agnostic ARL and the importance of including class labels in adversarial inputs.
- Additional results: Vanilla ARL uses non-protected features X and class labels Y as adversarial inputs, while variants additionally provide protected features S.The variants include ARL(adv: S), ARL(adv: S+Y), and ARL(adv: X+Y+S).
- Additional results: Group-agnostic ARL is competitive with variants that access protected groups, except on COMPAS.The authors present this as evidence that ARL can improve fairness without demographic access.
- Additional results: Variants receiving class labels generally outperform variants without labels; ARL(S+Y) has higher AUC than ARL(S) for all groups across all datasets.The paper attributes this to the ability to treat positive and negative examples differently under group base-rate differences.
- Blind fairness: ARL(adv: X+Y+S) performs comparably to ARL(adv: Y+S), while access to X improves fairness in some cases such as Adult.The authors suggest X enables finer distinctions among disadvantaged candidates within a protected group.
- Experimental setup: Main experiments use Adult, LSAC, and COMPAS datasets, with AUC reported as mean ± standard deviation for protected groups.The datasets cover income, bar-exam passage, and recidivism prediction tasks.
- Reproducibility: The implementations, preprocessing scripts, and synthetic-data generation code are publicly available.Categorical features are one-hot encoded and feature vectors standardized to zero mean and unit variance.
- Experimental setup: All methods use matched data splits and tuning procedures, while baselines receive additional hidden units to compensate for ARL’s example-weight capacity.Datasets are split 70%/30%, and training data use five-fold cross-validation for hyperparameter selection.