Source-linked AI summary
Automated Directed Fairness Testing
Sakshi Udeshi, Pryanshu Arora, Sudipta Chattopadhyay
TL;DR
AEQUITAS addresses the challenge of automatically finding individual-fairness violations in machine-learning models. It uses directed probabilistic search guided by model robustness, then retrains models with generated discriminatory inputs; across six classifiers, it generated up to 70% discriminatory inputs and improved fairness up to 94%.
Problem
Automatically locating individual-fairness violations is difficult because discriminatory input pairs may occur only in specific regions of a model’s input space.
Method
AEQUITAS uses three probabilistic search strategies guided by model robustness to generate discriminatory inputs and automatically retrains models using them.
Results
Across six state-of-the-art classifiers, AEQUITAS generated up to 70% discriminatory inputs and improved fairness up to 94%.
Takeaways & Limitations
AEQUITAS provides directed fairness testing and systematically leverages generated tests to improve the fairness of machine-learning models.
Takeaways & Limitations
AEQUITAS requires access to the model’s training data and training mechanism, and users must carefully choose its probability-change parameters.
Abstract
from arXiv · showhide
Fairness is a critical trait in decision making. As machine-learning models are increasingly being used in sensitive application domains (e.g. education and employment) for decision making, it is crucial that the decisions computed by such models are free of unintended bias. But how can we automatically validate the fairness of arbitrary machine-learning models? For a given machine-learning model and a set of sensitive input parameters, our AEQUITAS approach automatically discovers discriminatory inputs that highlight fairness violation. At the core of AEQUITAS are three novel strategies to employ probabilistic search over the input space with the objective of uncovering fairness violation. Our AEQUITAS approach leverages inherent robustness property in common machine-learning models to design and implement scalable test generation methodologies. An appealing feature of our generated test inputs is that they can be systematically added to the training set of the underlying model and improve its fairness. To this end, we design a fully automated module that guarantees to improve the fairness of the underlying model. We implemented AEQUITAS and we have evaluated it on six state-of-the-art classifiers, including a classifier that was designed with fairness constraints. We show that AEQUITAS effectively generates inputs to uncover fairness violation in all the subject classifiers and systematically improves the fairness of the respective models using the generated test inputs. In our evaluation, AEQUITAS generates up to 70% discriminatory inputs (w.r.t. the total number of inputs generated) and leverages these inputs to improve the fairness up to 94%.
1 INTRODUCTION
Aeqitas addresses the need to validate fairness in machine-learning decisions by discovering discriminatory inputs and using them to improve model fairness. Its directed testing and automated retraining are evaluated on six classifiers, producing substantial reported gains.
- Motivation: Machine-learning models used in sensitive domains can inherit training-data bias, making automated discrimination validation critically important.
- Problem and approach: Aeqitas targets individual fairness by finding similar input pairs that receive different model decisions because of sensitive features.
- Approach: Aeqitas directs test generation through random sampling followed by neighborhood search around discovered discriminatory inputs.
- Fairness improvement: Aeqitas systematically retrains models with generated discriminatory inputs to reduce unfairness through a fully automated module.
2 BACKGROUND
The background frames fairness testing as the search for violations of individual fairness in difficult-to-specify machine-learning behavior. Aeqitas differs from prior testing by directing exploration toward specialized discriminatory regions while exploiting model robustness.
- Importance of fairness: Fairness testing matters because machine-learning decisions increasingly affect law enforcement, credit, education, and employment, where unnecessary bias is consequential.
- Fairness in Aeqitas: Aeqitas defines its fairness target as similar inputs receiving different classifications because of potentially discriminatory input parameters.
- Fairness in Aeqitas: Discriminatory inputs may occupy specialized regions of the input space, requiring directed techniques for rapid discovery.
- State-of-the-art in fairness testing: Unlike random fairness testing, Aeqitas directs generation toward input regions that violate individual fairness.
- Robustness in machine learning: Aeqitas leverages robustness, under which small input changes are expected not to dramatically change model outputs, to guide nearby test generation.
3 APPROACH AT A GLANCE
Aeqitas combines global sampling with local probabilistic search to find discriminatory inputs, relying on model robustness to explore neighborhoods efficiently. Its approach includes three schemes and estimates discriminatory-input prevalence from repeated trials.
- Approach overview: Aeqitas proposes three schemes with different complexity levels for systematically uncovering software fairness problems.
- Global search: Global search uniformly samples inputs and records discriminatory cases, providing high-probability discovery when such cases exist.
- Local search: Local search assumes discriminatory inputs cluster near one another and explores neighborhoods around an initially discovered discriminatory input.
- Probabilistic strategies: The schemes vary how they choose parameters and perturbations, including probability updates when perturbations repeatedly produce discriminatory inputs.
- Estimation: 400 trials were sufficient for the estimated discriminatory-input percentage to converge when K was 1000.
4 DETAILED APPROACH
AEQUITAS defines discriminatory inputs through changes in sensitive parameters and uses global search followed by local perturbation search to discover fairness violations. Its local strategies adapt perturbation probabilities, exploiting model robustness to focus testing near discriminatory inputs.
- Definitions: A discriminatory input is one whose output differs from a minimally changed counterpart by more than the threshold γ.The counterpart differs on a non-empty subset of discriminatory parameters while matching on all others.
- Overall approach: AEQUITAS comprises global search and local search, with optional retraining using generated test inputs to improve fairness.Global search supplies starting points for local exploration over the input domain.
- Global search: Global search samples inputs, enumerates sensitive-parameter values, and records an input when two resulting variants violate the discrimination threshold.The method returns the collected discriminatory inputs as seeds for local search.
- Local search: Model robustness motivates neighborhood search because nearby inputs are expected to produce similar outputs and may therefore share discriminatory behavior.This assumption supports expanding from discovered discriminatory inputs to nearby test cases.
- Search strategies: AEQUITAS provides random, semi-directed, and fully-directed probability-update strategies to focus test generation on discriminatory inputs.The random strategy leaves parameter and direction probabilities unchanged while still searching around global-search seeds.
Aequitas random.
AEQUITAS random explores neighborhoods of globally discovered discriminatory inputs without updating its initial perturbation probabilities.
- Aequitas random: AEQUITAS random chooses both the perturbation direction and parameter randomly around discriminatory inputs.It does not update the initial probabilities σpr and σv, yet empirically outperforms purely random input-space search.
Aequitas semi-directed.
AEQUITAS semi-directed updates direction probabilities during local search, increasing the likelihood of directions associated with discriminatory perturbations while selecting parameters randomly.
- Aequitas semi-directed: AEQUITAS semi-directed increases σv[p] by Δv when perturbing parameter p produces a discriminatory input.The parameter remains randomly selected; σv[p] controls the probability of choosing δ = −1.
- Aequitas fully-directed: The fully-directed strategy additionally increases σpr[p] by Δpr when perturbing p produces a discriminatory input.This prioritizes parameters whose perturbations have yielded discriminatory inputs.
4.3 Estimation using LLN
AEQUITAS estimates the discriminatory-input percentage with repeated random sampling and uses generated discriminatory inputs in automated fairness-improving retraining. The retraining adds only selected portions and stops when estimated fairness no longer improves.
- LLN estimation: The Law of Large Numbers estimates the discriminatory-input percentage by averaging ratios from repeated IID random-input experiments.Each experiment counts discriminatory inputs among uniformly generated samples, and the averages approach the expected percentage as trials increase.
- Fairness-improving retraining: AEQUITAS automatically adds portions of generated discriminatory inputs to the training dataset to reduce their estimated percentage.The retraining module is described as guaranteeing a reduction in the percentage of discriminatory inputs in I.
- Fairness-improving retraining: Adding all generated discriminatory inputs may fail because targeted tests are unlikely to follow the training-data distribution and can bias it toward generated inputs.The strategy therefore selects only portions of Test rather than adding the entire set blindly.
- Fairness-improving retraining: Retraining samples a percentage of Test from exponentially increasing intervals and terminates when adding inputs no longer decreases estimated discriminatory inputs.The procedure compares fairness estimates before and after retraining and exits when the new estimate is not lower.
- Scope boundary: Labeling generated test data remains an orthogonal problem outside AEQUITAS's scope.The paper identifies labeling as a challenge and mentions majority voting as one possible strategy.
4.5 Termination
Aeqitas supports termination after a specified number of discriminatory inputs or within a time bound, allowing use-case-specific testing and evaluation.
- 4.5 Termination: Aeqitas can terminate after generating a user-specified number of discriminatory inputs from I.This supports testing, evaluation, or retraining when a target number of inputs is needed.
- 4.5 Termination: Aeqitas can also terminate within a given time bound to quickly check whether a model exhibits discrimination for selected sensitive parameters.The evaluation used both termination criteria to assess effectiveness and efficiency.
5 RESULTS
Aeqitas evaluates directed fairness testing across six classifiers and finds that directed generation produces more discriminatory inputs, does so faster, and supports fairness-improving retraining.
- RQ1: Effectiveness: Aeqitas fully-directed consistently outperforms its random and semi-directed variants, while requiring more computational effort than simpler strategies.The three methodologies differ in their sophistication for systematically searching the input space.
- RQ1: Effectiveness: Aeqitas-generated discriminatory inputs contain no false positives under the chosen discrimination threshold.Every generated discriminatory input is discriminatory to the model under test subject to that threshold.
- RQ1: Effectiveness: 20.4 is the maximum factor by which Aeqitas fully-directed outperforms purely random testing in discriminatory inputs generated.Aeqitas semi-directed is up to 64.9% better than Aeqitas random, and fully-directed is up to 56.56% better than semi-directed.
- RQ2: Efficiency: 83.27% is the average speed improvement of Aeqitas fully-directed over the state of the art when generating 10,000 discriminatory inputs.The maximum improvement is 96.62% for Multi Layer Perceptron; reported time includes test generation and execution.
- RQ3: Fairness improvement: 43.2% is the average reduction in discriminatory inputs achieved by retraining with Aeqitas-generated tests, reaching 94.36% at maximum.The retraining module added an average of 7,463 datapoints, or 22.92% of the original training data.
6 RELATED WORK
Prior work addresses fair model design, fairness definitions, and robustness testing, whereas Aeqitas applies directed search specifically to fairness testing.
- Fair Machine Learning Models: Fair-machine-learning research primarily designs classifiers or preprocesses training data to limit discrimination.Aeqitas is presented as complementary to approaches that design fair machine-learning models.
- Fairness Testing: Existing fairness-testing work defines fairness and discrimination but generates tests randomly rather than using Aeqitas’s directed strategy.Aeqitas instead focuses on directed test generation for locating discriminatory inputs.
- Testing and Verification of Machine Learning Models: DeepXplore and DeepTest evaluate robustness or erroneous behavior in neural networks, not the distinct fairness property targeted by Aeqitas.The paper states that fairness validation requires special attention to systematic test generation.
- Search-based testing: Aeqitas is described as the first known use of a novel search algorithm to test fairness in machine-learning systems.The authors believe the approach can be ported to a wider machine-learning context.
7 THREATS TO VALIDITY
Aeqitas depends on robustness, access to training data and retraining mechanisms, and real-valued inputs. Its effectiveness is also sensitive to probability-change settings and has been evaluated only with gender as the discriminatory feature.
- Assumptions: Aeqitas assumes that models under test exhibit robustness, allowing it to localize search near discriminatory input regions.The evaluation found this assumption effective for state-of-the-art models.
- Access requirements: Aeqitas requires access to training data and the model’s training mechanism to improve fairness through retraining.Generated discriminatory inputs augment the original training set but are insufficient to train a model from scratch.
- Input structure: Aeqitas currently works on real-valued inputs and does not handle images, sound, or video.Additional techniques are needed to automatically generate valid inputs for image- and video-based models.
- Parameter sensitivity: Users must tune ∆v and ∆pr carefully because overly high values can overshoot discriminatory regions, while overly low values can make directed methods resemble random testing.The authors evaluated several parameter values before results stabilized.
- Evaluation scope: Aeqitas was evaluated with gender as the discriminatory input feature, so its effectiveness for other discriminatory features cannot be concluded from these experiments.The authors state that the underlying mechanism is generic and may extend further.
8 CONCLUSION
The paper presents Aeqitas as an automated, directed strategy for generating discriminatory machine-learning test inputs and using them to improve model fairness. Evaluation shows effectiveness on state-of-the-art classifiers, while cause localization remains an open limitation.
- 8 CONCLUSION: Aeqitas combines fully automated directed test generation with strategies that use discriminatory inputs to improve model fairness.Its search exploits robustness in common machine-learning models.
- 8 CONCLUSION: Aeqitas provides statistical evidence about the number of discriminatory inputs in a model under test.This capability supports fairness testing of machine-learning models.
- 8 CONCLUSION: Aeqitas was effective in generating discriminatory test inputs and improving the fairness of state-of-the-art classifiers.The conclusion reports both testing and fairness-improvement outcomes.
- 8 CONCLUSION: Aeqitas cannot currently localize the cause of discrimination in a model, and further work is required to isolate that cause.This limitation remains despite the approach’s test-generation and fairness-improvement capabilities.
- 8 CONCLUSION: The authors envision extending Aeqitas beyond fairness testing to machine-learning models with complex inputs such as images and videos.They also connect the approach to broader validation of machine-learning applications in sensitive domains.