Source-linked AI summary
Fairness in Deep Learning: A Computational Perspective
Mengnan Du, Fan Yang, Na Zou, Xia Hu
TL;DR
Deep learning models can reproduce or amplify societal biases in high-stakes decisions, while their opacity complicates diagnosis and mitigation. This review surveys computational fairness work, emphasizing interpretability for detecting discriminatory mechanisms and organizing mitigation methods across the deep learning life cycle. It concludes that fairness-sensitive DNN systems require bias reduction alongside attention to practical challenges such as fairness–utility trade-offs and inadequate benchmark datasets.
Problem
Deep learning models may discriminate against protected groups by reproducing or amplifying biases in data, and their opacity makes fairness problems difficult to diagnose and address.
Method
The paper reviews computational approaches to DNN fairness, using interpretability for diagnosis and categorizing mitigation strategies across three deep learning life-cycle stages.
Results
The review identifies interpretability as a useful tool for diagnosing reasons behind algorithmic discrimination and surveys mitigation approaches for deep learning fairness.
Takeaways & Limitations
Fairness-sensitive DNN applications should reduce undesirable biases rather than amplify them, while considering the computational challenges identified by the review.
Takeaways & Limitations
Removing bias can compromise main-task prediction accuracy, especially for non-protected groups, making simultaneous fairness and satisfactory utility an ongoing challenge.
Abstract
from arXiv · showhide
Deep learning is increasingly being used in high-stake decision making applications that affect individual lives. However, deep learning models might exhibit algorithmic discrimination behaviors with respect to protected groups, potentially posing negative impacts on individuals and society. Therefore, fairness in deep learning has attracted tremendous attention recently. We provide a review covering recent progresses to tackle algorithmic fairness problems of deep learning from the computational perspective. Specifically, we show that interpretability can serve as a useful ingredient to diagnose the reasons that lead to algorithmic discrimination. We also discuss fairness mitigation approaches categorized according to three stages of deep learning life-cycle, aiming to push forward the area of fairness in deep learning and build genuinely fair and reliable deep learning systems.
1 INTRODUCTION
Deep learning models are increasingly used in high-stakes applications, but data-driven learning can reproduce or amplify societal biases. Their opacity makes diagnosing and mitigating such unfairness difficult, motivating this computational review of interpretability and mitigation strategies.
- Deep learning is increasingly deployed in high-stakes applications such as employment, criminal justice, and personalized medicine.
- Data-driven learning can cause models to replicate and amplify historical prejudices and demographic inequalities present in training data.
- Opaque deep learning models make it difficult to determine whether decisions rely on justified reasons or bias.
- The article reviews fairness in deep learning from a computational perspective, emphasizing interpretability and mitigation strategies across three life-cycle stages.
2 DNN FAIRNESS
DNN unfairness is framed as either discriminatory outcomes or unequal prediction quality. The section introduces group-fairness measurements and interpretability tools for understanding how models use inputs and learned representations.
- DNN fairness problems: DNN unfairness comprises prediction outcome discrimination and prediction quality disparity.
- Prediction outcome discrimination: Prediction outcome discrimination can arise from biased training data and correlated features even when protected attributes are not explicit inputs.
- Prediction quality disparity: Prediction quality disparity concerns unequal service quality across protected groups, including differences in facial-recognition and language-processing performance.
- Measurements of fairness: Group fairness compares prediction statistics across sensitive-attribute groups, while individual fairness seeks similar predictions for similar inputs.
- Measurements of fairness: Demographic parity compares favorable-outcome rates across groups, whereas equality of opportunity compares true-positive rates using ground-truth labels.
- Interpretability: Local interpretation explains a specific prediction through feature attributions, while global interpretation examines knowledge captured by a pretrained DNN.
3 DETECTION OF MODELING BIAS
Interpretability supports computational detection of modeling bias by locating influential sensitive inputs and testing protected concepts encoded in deep representations. Examples show that local and global analyses can reveal discriminatory behavior and subgroup disparities.
- Bias detection uses interpretability to analyze discriminatory decisions through input features and intermediate representations.
- Discrimination via input: Local interpretation ranks influential input features so potentially fairness-sensitive features can be distinguished from task-relevant features.
- Discrimination via input: Sentiment models produced statistically significant race bias for sentences containing African American versus European American first names, violating demographic parity.
- Discrimination via representation: Global interpretation first tests whether a protected concept is encoded in intermediate activations, then measures its contribution to final predictions.
- Discrimination via representation: In a CNN doctorness classifier, the not-woman concept dramatically increased doctorness confidence, indicating discrimination toward women and reflecting a gender stereotype.
- Prediction quality disparity: Prediction-quality disparity is detected by splitting data into sensitive-attribute subgroups and comparing accuracy across demographic groups.
- Prediction quality disparity: Class activation maps were used to inspect which image regions models attended to for lighter- and darker-skin groups.
4 MITIGATION OF MODELING BIAS
Fairness mitigation methods are organized around the deep learning pipeline, with pre-processing, in-processing, and post-processing addressing dataset, training, and inference stages. Input regularization reduces reliance on sensitive features using feature-wise annotations.
- Mitigation methods are categorized into pre-processing, in-processing, and post-processing according to dataset construction, model training, and inference stages.
- Regularization-based input mitigation requires feature-wise annotations identifying fairness-sensitive and task-relevant subsets of input features.
4.1 Discrimination via Input
Input-focused mitigation methods modify data, training objectives, or inference-time predictions to reduce discrimination while balancing fairness against model utility. The review evaluates these approaches on Adult Census Income and COMPAS, finding recurring trade-offs and metric conflicts.
- Pre-processing: Pre-processing removes or replaces fairness-sensitive features, but deleting them can reduce model performance and utility.Examples include removing surname and ZIP code, or substituting alternative values.
- In-processing: In-processing regularization adds implicit or explicit fairness constraints during model training.Implicit methods use local interpretations to discourage reliance on spurious protected-attribute correlations, while explicit methods optimize accuracy against fairness metrics.
- Mitigation framework: Table 2 organizes representative mitigation algorithms by dataset construction, model training, and model inference.These correspond respectively to pre-processing, in-processing, and post-processing in the deep learning pipeline.
- Post-processing: Post-processing calibration adjusts predictions using protected attributes during inference to approach a target distribution or fairness metric.These methods may be impractical because protected attributes are usually unavailable at inference time.
- Evaluation: Across Adult Census Income and COMPAS, unmitigated DNNs amplify training-data bias, while mitigation often compromises accuracy and produces conflicting fairness measurements.The evaluation uses 48,842 and 6,167 instances, respectively, with protected attributes gender and race.
4.2 Discrimination via Representation
Representation-focused mitigation targets protected-attribute information encoded in intermediate DNN representations. Reviewed approaches include balanced data, adversarial training, residual decomposition, and inference-time neuron suppression, each with stated limitations or trade-offs.
- Pre-processing: Balanced or gender-swapped datasets can reduce representation bias, but balanced training data does not guarantee that DNNs stop encoding protected attributes.Prior studies find that intermediate representations may still capture gender or race even after balancing.
- Evaluation: Table 3 compares five input-discrimination mitigation methods using accuracy, demographic parity, equality of opportunity, and equality of odds.Accuracy and demographic parity are better closer to 1, whereas equality of opportunity and equality of odds are better closer to 0.
- In-processing: Adversarial learning trains a predictor and adversarial classifier together so representations remain useful for the main task while making protected attributes less predictable.The method improves demographic parity and equality of opportunity, but strong regularization can harm accuracy and training stability.
- In-processing: Residual decomposition adds a constraint that keeps the debiased representation in the same space as the original input while suppressing protected attributes.This supplements the requirements that the representation predict the main task and reduce protected-attribute information.
- Post-processing: Inference-stage representation mitigation identifies neurons associated with protected attributes and sets their outgoing activations to zero.The process uses global interpretation first, followed by neuron suppression during inference.
4.3 Prediction Quality Disparity
Prediction-quality mitigation seeks to improve outcomes for underrepresented groups through more diverse data and model-training strategies. Reviewed examples report gains in subgroup accuracy, overall accuracy, or predictive quality parity.
- Data: Diversifying training data can improve prediction quality for underrepresented groups by collecting more comprehensive sources.The Faces of the World dataset targets uniform representation across two genders and four ethnic groups.
- Model training: Transfer learning can address unequal face-attribute detection performance across race and gender subgroups.A source-domain dataset rich in minority-group data is used before adapting the model to the target setting.
- Model training: Multi-task learning jointly classifies gender, race, and age and can significantly improve accuracy across demographic subgroups.The reported gains promote fairness in predictive quality parity.
5 RESEARCH CHALLENGES
Fairness in deep learning remains constrained by gaps in evaluation, intersectional analysis, metric formalization, utility preservation, and large-scale training effects. These challenges motivate more systematic benchmarks, demographic metadata, and methods that address fairness without sacrificing predictive performance.
- Benchmark Datasets: Benchmark datasets are lacking, while ordinary hold-out sets may reproduce training biases and fail to reveal unfairness in trained DNNs.The paper recommends benchmark datasets as supplementary tests and demographic statistics such as geography, gender, and ethnicity where available.
- Intersectional Fairness: Intersectional fairness research remains limited because mitigating one bias can leave discrimination toward groups defined by multiple protected attributes.Examples include gender-debiased systems that may still disadvantage African American women, older women, or groups differentiated by race and age.
- Fairness and Utility Trade-off: Bias mitigation can improve fairness while reducing main-task performance, including accuracy for non-protected groups.Adversarial training is cited as an example of a method that may increase fairness but compromise overall prediction accuracy.
- Formalization of Fairness: There is no consensus on fairness measurements, and satisfying one metric can conflict with another.A loan approval tool may satisfy demographic parity while violating equality of opportunity, so measurement choices must fit application requirements.
- Large-scale Training: Large-scale pretraining can capture and propagate biases across downstream tasks, while billion-word corpora and multi-day training make bias correction difficult.The paper identifies pretrained language models such as BERT and XLNet as examples of models whose learned biases may transfer to later tasks.
6 CONCLUSIONS
As DNNs enter high-stake applications, their algorithmic unfairness has become a central concern. The review surveys computational detection and mitigation techniques, emphasizing interpretability and interdisciplinary work to reduce rather than amplify societal biases.
- 6 CONCLUSIONS: DNN unfairness in high-stake applications motivates a computational overview of bias detection and mitigation techniques.The applications named include job hunting, criminal justice, and loan approval.
- 6 CONCLUSIONS: Interpretability is the review’s particular focus for understanding DNN bias.The paper connects this focus to the broader goal of addressing undesirable algorithmic unfairness.
- 6 CONCLUSIONS: The paper argues that DNN models should reduce societal biases instead of amplifying them to benefit society.It calls for contributions from computer science, statistics, and cognitive science to eliminate disparity and promote fairness.