Source-linked AI summary
NeuFair: Neural Network Fairness Repair with Dropout
Vishnu Asutosh Dasu, Ashish Kumar, Saeid Tizpaz-Niari, Gang Tan
TL;DR
DNN-based systems can encode discrimination, while existing fairness repair for pre-trained networks faces challenges including the combinatorial search over neurons. NeuFair uses randomized inference-time dropout search to balance fairness and utility. Experiments report fairness improvement of up to 69% with minimal utility degradation in most cases, and the conclusion reports outperforming a state-of-the-art post-processing mitigator.
Problem
Mitigating unfairness in pre-trained DNNs is challenging, and finding a neuron subset to drop requires searching an exponentially large set of possibilities.
Source: §1 · pp. 1–2, §3.2 · p. 3Method
NeuFair applies randomized search, including simulated annealing and random walk, to select inference-time neuron dropouts using a cost function encoding fairness and utility.
Source: §1 · p. 2Results
NeuFair improved fairness by up to 69% with minimal utility degradation in most cases and outperformed a state-of-the-art post-processing bias mitigator.
Source: §1 · p. 2, §8 · p. 11Takeaways & Limitations
The experiments indicate that selected neurons can disparately contribute to unfairness, and dropping them can improve fairness while maintaining utility in the studied settings.
Source: §1 · p. 2, §8 · p. 11Because the approach uses randomized algorithms, its results may be sub-optimal.
Source: §6 · p. 10
Abstract
from arXiv · showhide
This paper investigates neuron dropout as a post-processing bias mitigation for deep neural networks (DNNs). Neural-driven software solutions are increasingly applied in socially critical domains with significant fairness implications. While neural networks are exceptionally good at finding statistical patterns from data, they may encode and amplify existing biases from the historical data. Existing bias mitigation algorithms often require modifying the input dataset or the learning algorithms. We posit that the prevalent dropout methods that prevent over-fitting during training by randomly dropping neurons may be an effective and less intrusive approach to improve the fairness of pre-trained DNNs. However, finding the ideal set of neurons to drop is a combinatorial problem. We propose NeuFair, a family of post-processing randomized algorithms that mitigate unfairness in pre-trained DNNs via dropouts during inference after training. Our randomized search is guided by an objective to minimize discrimination while maintaining the model's utility. We show that our design of randomized algorithms is effective and efficient in improving fairness (up to 69%) with minimal or no model performance degradation. We provide intuitive explanations of these phenomena and carefully examine the influence of various hyperparameters of search algorithms on the results. Finally, we empirically and conceptually compare NeuFair to different state-of-the-art bias mitigators.
1 Introduction
The paper motivates mitigating discrimination in DNN-based systems and proposes inference-time neuron dropout as post-processing for pre-trained models. NeuFair uses randomized search to balance fairness and utility, with experiments reporting fairness gains and comparisons to other mitigators.
1 Introduction
- Discrimination in automated systems can disadvantage protected groups and raise legal-compliance concerns, motivating fairness mitigation in socially critical software.The introduction gives examples involving parole, hiring, and tax audits.Source: §1 · p. 1
- The paper presents dropout as a post-processing method for improving fairness in pre-trained DNNs, distinguishing this use from dropout’s established role in preventing overfitting during training.The authors describe it as the first systematic use of dropout for this purpose, to their knowledge.Source: §1 · p. 2
- Because exhaustive search over neuron-dropout combinations is intractable, the paper explores randomized algorithms, including simulated annealing and random walk, to search efficiently.The algorithms encode fairness and utility frontiers in a cost function.Source: §1 · p. 2
- NeuFair improved fairness by up to 69% with minimal utility degradation in most cases, sometimes improving fairness and utility together.The authors also report that simulated annealing outperformed random walk and a state-of-the-art post-processing mitigator.Source: §1 · p. 2
- The evaluation covers seven DNN benchmarks across five socially critical applications and examines randomized algorithms’ effectiveness and efficiency against state-of-the-art techniques.The authors also examine search hyperparameters and report a pathological case and reasons for a failure.Source: §1 · p. 2
2 Background
The background introduces model utility and fairness measures used in the paper. It contrasts utility metrics for imbalanced data and distinguishes fairness criteria before identifying Equalized Odds as the study’s criterion.
2 Background
- Section 2 frames the model utility and fairness metrics used in the paper.The background section introduces these measures before the problem statement.Source: §2 · p. 2
2.1 Notions of Model Utility
- For imbalanced datasets, accuracy can be misleading: predicting every Bank dataset sample as negative yields 88% accuracy but an F1 score of 0 or undefined.F1 is the harmonic mean of precision and recall, so low precision or recall reduces it substantially.Source: §2.1 · p. 2
2.2 Notions of Model Fairness
- Demographic Parity requires prediction independence from the sensitive attribute, while Equalized Odds requires conditional independence given the label; Equal Opportunity relaxes Equalized Odds to true-positive-rate parity.The paper uses Equalized Odds, measuring disparity by the maximum absolute difference in true- and false-positive rates across sensitive groups.Source: §2.2 · p. 2
3 Problem Statement
The paper considers binary classifiers built from pre-trained DNNs, with protected and non-protected inputs. It formulates fairness repair as selecting hidden neurons to drop at inference while preserving utility, then motivates randomized search because exhaustive search is exponential.
3 Problem Statement
- The problem concerns pre-trained binary DNN classifiers whose inputs include protected and non-protected variables and whose outputs are favorable or unfavorable outcomes.The model’s predicted label is the index of the higher-likelihood output.Source: §3 · p. 3, §3.1 · p. 3
3.1 Syntax and Semantics of DNN
- A DNN maps inputs through hidden layers to output likelihoods, and the paper assumes some hidden neurons contribute disparately to unfairness.The network is represented with input, hidden, and output layers and their weights.Source: §3.1 · p. 3
3.2 Inference Time Dropout for Fairness
- Inference-time dropout searches for a neuron subset that reduces bias while minimally degrading performance, keeping the model’s input, output, and hidden-layer structure unchanged.Unlike training dropout, this approach selects neurons on the fairness-performance tradeoff after training.Source: §3.2 · p. 3
- Exhaustive search over neuron subsets is exponential in network size, so the paper explores randomized algorithms to find desirable dropout states more efficiently.There are 2^|N| possible subsets for N neurons.Source: §3.2 · p. 3
4 Approach
NeuFair uses randomized search over inference-time neuron-dropout choices to reduce unfairness while guarding model utility. Its approach defines a constrained dropout-state space and uses Simulated Annealing and Random Walk to explore it.
4 Approach
- NeuFair searches neuron-dropout states with Simulated Annealing and Random Walk, minimizing unfairness while penalizing states whose F1 falls below a threshold.The cost function balances the candidate state's unfairness against a penalty tied to the original model's unfairness when the candidate crosses the F1 threshold.Source: §4.1 · pp. 3, 5, §4 · p. 3
4.1 Simulated Annealing Search
- Each candidate dropout pattern is a binary state over the network's neurons, restricted to a range of permitted dropout counts.The bounds avoid dropping so many neurons that utility becomes unacceptable or so few that fairness gains are marginal, yet the bounded search can remain prohibitively large.Source: §4.1 · pp. 3–4
- Simulated Annealing explores neighboring states that differ by one neuron, accepting lower-cost moves and sometimes accepting higher-cost moves according to temperature.The neighborhood is defined by Hamming distance one, and higher temperature makes acceptance of a higher-cost transition more likely.Source: §4.1 · pp. 3–5
- The logarithmic cooling schedule has convergence guarantees, although the stated worst-case iteration bound is practically infeasible.The paper initializes temperature using the algorithm from [3], which estimates a starting temperature from acceptance probabilities for positive transitions.Source: §4.1 · p. 5
4.2 Random Walk Search
- Random Walk uses the same cost function, always moves to a new neighboring state regardless of cost, and records the best state found.The paper characterizes Random Walk as an SA run with infinite temperature, making its transition probability always 1.Source: §4.2 · p. 5
5 Experiments
Across seven benchmarks, NeuFair’s randomized dropout search improves fairness, with simulated annealing (SA) generally outperforming random walk (RW). Results also show a trade-off between F1 and fairness, while accuracy can rise on imbalanced datasets.
5.1 Datasets and Models
- The evaluation covers five binary-classification datasets and seven benchmarks, using DNNs trained for their respective prediction tasks.Two datasets are evaluated with two protected groups each.Source: §5.1 · p. 6
5.3 Experimental Setup
- Training, validation, and test sets use a 60%/20%/20% split; NeuFair selects dropouts on validation data and fairness is evaluated on test data.The model is selected at its highest validation F1, with a 2% F1 degradation tolerance.Source: §5.3 · p. 6
5.3.1 Effectiveness of randomized algorithms on mitigating unfair-
- SA and RW found a desirable state meeting the 0.98× initial F1 threshold in 69 of 70 runs; both improved test fairness, though COMPAS (Sex) was pathological.Source: §5.3.1 · pp. 6, 8
- Both algorithms improve test EOD; SA leads on four datasets and reduces Default EOD by 69.43% (8.962% to 2.749%).Source: Table 4 · p. 7, §5.3.1 · p. 8
5.3.2 Dropout strategies improving both fairness and utility (RQ2).
- Across datasets, F1 decreases as fairness improves, while accuracy rises for Bank, Default, and MEPS16 amid their imbalanced class distributions.The authors attribute the opposing trends to more negative predictions favoring true negatives.Source: §5.3.2 · pp. 8–9
5.3.3 Hyperparameters of randomized algorithms and fairness (RQ3).
- A lower F1 threshold, more neurons dropped, and longer time-outs can improve fairness; the F1 penalty multiplier tunes SA’s exploration–exploitation balance.Higher penalties encourage local exploitation, while lower penalties encourage broader exploration.Source: §5.3.3 · p. 9
5.3.4 Comparing to the state-of-the-art (RQ4).
- For RQ4, NeuFair outperforms the state-of-the-art unfairness mitigation algorithm Dice [36] across all benchmarks; its relative improvement is 56% higher than Dice.Source: §5.3.4 · p. 10
6 Discussion
NeuFair’s simulated annealing search finds strong dropout configurations in a space where exhaustive search is costly. The discussion relates this performance to the search-space structure and notes limitations in the reported fairness evaluation.
Effectiveness of Randomized Algorithms
- 43,076,484 states made exhaustive search take 60 hours, while simulated annealing (SA) used a one-hour timeout and found the global optimum for two seeds.The largest reported EOD Delta cost difference between SA and brute force was 0.054.Source: §6 · p. 10
- Over 90% of states were bad, while good states occupied small pockets that SA’s cost gradient and probabilistic transitions could help it find.Probabilistic transitions can move to worse states, potentially helping SA escape local minima.Source: §6 · p. 10
Limitations
- Test-set fairness improvements were smaller than validation-set improvements for both algorithms, and experiments used only Equalized Odds Difference (EOD).The authors note that the cost function can be modified to use other fairness metrics.Source: §6 · p. 10
7 Related Work
Related mitigation methods intervene in training data, training procedures, or model outputs. NeuFair is positioned as inference-time neuron dropout on pre-trained DNNs, without retraining or access to the training dataset.
Pre-processing Techniques
- Pre-processing methods alter training data, including reweighting, resampling, and feature debiasing; the authors note that such changes may be infeasible or distort group representations.The reviewed approaches include reweighting, Fair-SMOTE [8], and LTDD.Source: §7 · pp. 10–11
In-processing techniques
- In-processing methods improve fairness during training, often by changing the loss function, model parameters, or training algorithm.The section describes adversarial debiasing [52], prejudice remover, and Parfait-ML [46].Source: §7 · p. 11
Post-processing techniques
- Post-processing methods modify predictions or model behavior; unlike EOP [25], NeuFair does not require an auxiliary model, retraining, or training-data access.Compared with FairNeuron [21], NeuFair uses deterministic inference-time dropout to select neurons while seeking to preserve F1 and improve fairness.Source: §7 · p. 11
8 Conclusion
The conclusion presents NeuFair as a randomized approach to mitigating unfairness in pre-trained DNNs. Experiments report up to 69% fairness improvement, while future work proposes broader settings and alternative search strategies.
Conclusion
- Up to 69% fairness improvement was reported, and NeuFair outperformed a state-of-the-art post-processing bias mitigator.The authors attribute the improvement to identifying a subset of neurons to drop that disparately contributes to unfairness.Source: §8 · p. 11
Future Work
- Future work proposes analyzing top-k states to potentially reduce validation–test fairness-improvement differences and extending to multiple or multi-valued protected attributes.Examples include age groups and optimizing for race and sex together.Source: §8 · p. 11
- The authors also propose studying dropout during fine-tuning for large language model fairness and toxicity, and exploring genetic algorithms for the combinatorial search.They note that genetic algorithms could explore more states simultaneously by maintaining a population of candidate solutions.Source: §8 · p. 11