Fairness Testing: A Comprehensive Survey and Analysis of Trends

• ML software has produced unfair decisions involving sensitive attributes, including higher recidivism-risk labels for black defendants than white defendants.Such harms can disadvantage minority groups and protected categories, creating serious ethical concerns.Source: §1 · p. 1
• Fairness testing detects fairness bugs by executing ML software to verify fairness requirements and support related software-repair activities.The survey defines fairness bugs as mismatches between desired fairness conditions and actual outcomes.Source: §1 · p. 2
• Fairness testing is challenging because its oracle must compare inputs and outputs across demographic groups, while fairness definitions may conflict or be impossible to satisfy simultaneously [123].Different fairness definitions can therefore require different test oracles.Source: §1 · p. 2
• 89% of fairness-testing publications appeared since 2019, demonstrating the field’s recent growth and motivating this survey.Figure 1 reports the cumulative number of fairness-testing publications through 2023.Source: §1 · pp. 2–3, Figure 1 · p. 2
• The survey covers 100 papers and organizes fairness testing by workflow and components, while identifying trends, opportunities, public datasets, and open-source tools.It focuses specifically on fairness testing of ML software and is presented as the first comprehensive survey dedicated to this literature.Source: §1 · p. 3

2.1 Definition of Fairness

• Fairness definitions comprise individual fairness for similar outcomes among similar people and group fairness for comparable treatment across demographic groups, using sensitive attributes to distinguish privileged and unprivileged groups.Common sensitive attributes include sex, race, age, and physical ability.Source: §2.1 · p. 4

2.1.1 Individual Fairness.

• Individual fairness includes unawareness, awareness, counterfactual fairness, and causal fairness, which differ in whether and how sensitive attributes and their causal relationships affect predictions.Awareness compares outcome distributions using similarity metrics; counterfactual fairness changes sensitive attributes and causally linked features, whereas causal fairness modifies sensitive attributes without modeling those feature relationships.Source: §2.1.1 · pp. 5–6

2.1.2 Group Fairness.

• Group fairness includes statistical parity, equalized odds, and equal opportunity, requiring respectively equal favorable-outcome rates, equal true- and false-positive rates, or equal true-positive rates across demographic groups.These definitions evaluate predictions conditional on sensitive attributes and, for the latter two criteria, the actual outcome.Source: §2.1.2 · pp. 6–7

2.2 Definition of Fairness Bug and Fairness Testing

• A fairness bug is an imperfection that violates the required fairness conditions, while fairness testing executes inputs on the system to reveal discordance between required and existing fairness conditions.The required conditions depend on the fairness definition adopted by the developers, and the testing definition excludes manual inspection and formal verification.Source: §2.2 · p. 7

2.3 Fairness Testing Workflow

• The fairness-testing workflow specifies fairness conditions, creates test oracles, obtains or generates test inputs, executes them, assesses test adequacy, reports bugs, and iterates repairs before deployment and runtime monitoring.Individual-fairness testing typically uses pairs of data instances, whereas group-fairness testing uses sets with diverse demographic information.Source: §2.3 · p. 8

2.4 Fairness Testing Components

• Fairness testing should cover training data, ML programs, frameworks, models, and non-ML components because biases can arise across the interconnected ML software stack.Data testing targets feature, label, and selection bias; ML program testing targets processing, algorithm-selection, and hyper-parameter issues; model testing may be white-, black-, or gray-box.Source: §2.4 · pp. 8–10
• Existing fairness-testing research primarily addresses data, ML program, and model testing, while no known studies test fairness in ML frameworks or non-ML components.Non-ML components can affect fairness through practices such as demographic underrepresentation in data storage or biased user interfaces.Source: §2.4 · p. 10

3.1 Survey Scope

• The scope includes papers introducing fairness testing or presenting ML software fairness-testing approaches, studies, frameworks, or tools.The survey excludes fairness issues in network and hardware systems, fairness definitions without testing, and SE process topics such as gender diversity or cognitive bias.Source: §3.1 · p. 10

3.2 Paper Collection

• The authors searched DBLP with iteratively refined keyword combinations, manually screened 7,674 hits, and selected 67 relevant papers.The final search comprised 48 searches conducted on March 12, 2023, using terms related to fairness, software, learning, defects, testing, detection, and discovery.Source: §3.2 · p. 11
• The survey analyzes 100 papers collected through keyword searching, snowballing, and author suggestions.Table 4 summarizes the collection process: 67 papers came from DBLP searching, 25 from snowballing, and 8 from author suggestions.Source: §3.2 · p. 11, Table 4 · p. 12

3.3 Paper Analysis

• The authors used thematic synthesis and full-text manual analysis to categorize papers by testing workflow and test components.Disagreements were resolved through co-author discussion, all authors independently double-checked the content, and collected-paper authors provided feedback and validation.Source: §3.3 · p. 12

4 FAIRNESS TESTING WORKFLOW

• The workflow surveys techniques supporting the key activities of fairness testing.Source: §4 · p. 13
• One activity is generating test inputs for fairness testing.Source: §4 · p. 13
• The other activity is identifying test oracles for fairness testing.Source: §4 · p. 13

4.1.1 Random Test Input Generation.

• Random generation varies sensitive attributes while randomly assigning non-sensitive values, and Themis [160] measures causal-fairness and statistical-parity discrimination frequencies.Themis quantifies discriminatory instances by observing output changes after sensitive-attribute alterations and favorable-outcome disparities across demographic groups.Source: §4.1.1 · p. 14

4.1.2 Search-based Test Input Generation.

• Search-based generation addresses random testing’s low discriminatory-input success rate [156] by combining global exploration with local neighborhood search guided by meta-heuristics.Global search finds initial discriminatory pairs, while local search exploits the hypothesis that nearby inputs are similarly discriminatory for robust models. Approaches include Aequitas, KOSEI, explanation-guided methods, DNN-specific gradient or neuron-guided techniques, reinforcement learning, and subgroup search.Source: §4.1.2 · pp. 14–15

4.1.3 Verification-based Test Input Generation.

• Verification-based generation translates a fairness property and approximated white-box model into SMT formulas, then uses satisfiability solving to generate violating test cases.VBT-CT adds combinatorial t-way sampling, while VBT-X uses hash-based sampling to produce more diverse solutions with reasonable computational overhead.Source: §4.1.3 · p. 17

4.1.4 Domain-specific Test Input Generation.

• Domain-specific test-input generation targets natural inputs from practical application data distributions; in NLP, approaches mutate words tied to sensitive attributes or use handcrafted templates to compare fairness-related outcomes.BiasAsker measures absolute and relative bias using a dataset of 841 social groups and 8,110 biased properties.Source: §4.1.4 · pp. 17–19
• For computer vision, fairness tests alter a person’s sensitive attribute while holding other factors constant, but GANs struggle to make precise isolated changes.Latent-direction, FaderNetwork, and CycleGAN methods [149] improve controlled image generation, while YCrCb-based luminance shifts and optimal transport target skin-type changes without confounding attributes.Source: §4.1.4 · pp. 19–20
• ImageCFGen incorporates causal-graph knowledge and inference within a GAN-like framework to generate discriminatory images satisfying counterfactual fairness.It addresses unrealistic attribute changes, such as altering age without accounting for its effect on organ size in chest MRI images.Source: §4.1.4 · p. 20
• Transects modify multiple sensitive attributes simultaneously, while makeup perturbations and speech transformations test whether robustness differs across demographic groups [106].AequeVox applies eight real-life speech perturbations and flags fairness bugs when group error-rate increases differ beyond a preset threshold; salient lip regions most affect makeup-based deepfake fairness.Source: §4.1.4 · pp. 20–21

4.1.5 Test Input for Data Testing.

• Data-testing programs detect feature bias by modeling sensitive-feature relationships, causal pathways, latent-space directions, or privileged-to-unprivileged counterparts [117].These implementations identify correlated or causally unfair features, unknown biased attributes, and unfairness through flipset analysis.Source: §4.1.5 · p. 21
• Situation testing and facial-action-unit programs detect label bias, while statistical association and representation analyses identify selection bias and demographic underrepresentation [125] [126] [127] [133].Researchers apply these programs to biased instances, training-data associations between sensitive attributes and outcomes, and dataset coverage across demographic groups.Source: §4.1.5 · p. 21

4.2 Test Oracle Identification

• Fairness testing primarily uses metamorphic relations and statistical measurements as test oracles for determining whether fairness requirements are violated.Metamorphic relations compare expected output changes across executions, while statistical measurements quantify fairness and flag bugs when values cross specified thresholds.Source: §4.2 · p. 21, §4.2.2 · p. 23

4.2.1 Metamorphic Relations as Test Oracles.

• Metamorphic fairness tests mutate sensitive or non-sensitive attributes and compare predictions, with task-specific relations for classification, regression, generation, and translation.Classification commonly expects similar instances differing only in sensitive attributes to receive identical outcomes; regression uses thresholds or fairness degree, while generation uses similarity measures such as sentiment, perplexity, semantics, or regard.Source: §4.2.1 · pp. 21–23

4.2.2 Statistical Measurements as Test Oracles.

• Statistical oracles span predicted outcomes, actual outcomes, probabilities, neuron activations, situation testing, optimal transport, rankings, and multiple subgroups, but selecting measures and thresholds remains difficult.Examples include SPD, DI, EOD, AOD, calibration, intersectional fairness, pairwise ranking fairness, and FairRec’s performance, popularity-alignment, and diversity measurements; statistical testing can reduce reliance on manually chosen thresholds.Source: §4.2.2 · pp. 23, 25

4.2.3 Test Oracle for Data Testing.

• Data-testing oracles identify feature bias through correlations or causal links, label bias through group-specific model disagreements, and selection bias through unequal associations or representation.Some selection-bias oracles combine equal favorable rates between privileged and unprivileged groups with equal group counts in the training data.Source: §4.2.3 · p. 26

5 FAIRNESS TESTING COMPONENTS

• Fairness testing can target different software parts, including training data, ML programs, ML models, ML frameworks, and non-ML components.The section presents this categorization from the perspective of “what to test.”Source: §5 · p. 26
• Existing fairness-testing studies primarily examine training data, ML programs, and ML models.Source: §5 · p. 26

5.1 Data Testing

• Data testing detects bias in training-data features, labels, and distributions before such bugs are learned and propagated through ML model development.These approaches target fairness bugs in the data-driven development pipeline.Source: §5.1 · p. 26

5.1.1 Detecting Feature Bias.

• Feature selection substantially affects model fairness, while removing sensitive attributes may not prevent unfairness when correlated proxies, such as Zipcode, enable indirect inference.Studies also use regression, causal graphs, latent-space counterfactuals, and flipsets to identify features associated with direct, indirect, or inconsistent discrimination.Source: §5.1.1 · pp. 27–28

5.1.2 Detecting Label Bias.

• Label-bias testing identifies divergent group-model predictions and reveals significant gender disparities in facial-expression labels that conventional fairness repair cannot completely mitigate.Situation testing compares models trained separately on privileged and unprivileged groups to flag potentially biased labels.Source: §5.1.2 · p. 28

5.1.3 Detecting Selection Bias.

• Distribution testing exposes selection bias through unequal group rates, representation, and dataset coverage, including Caucasian overrepresentation, skewed face-detector demographics, and sex-linked lie labels.Such selection bias can induce demographic performance disparities and produce deception detectors comparable to state-of-the-art systems despite relying only on sex as a proxy.Source: §5.1.3 · pp. 28–29

5.2 ML Program Testing.

• Because data processing, decision logic, and run-time configurations can create discordance with desired fairness conditions, testing targets the ML program implementation itself [134].Hyper-parameters are treated as a search-based fairness-testing problem because different settings can produce different fairness levels.Source: §5.2 · p. 29, §5.2.2 · p. 29

5.2.1 Testing Data Processing.

• Data-processing choices can introduce fairness bugs or improve fairness, and researchers test both preprocessing interventions and missing-value imputation strategies [116].These tests intervene on processing methods while holding other settings unchanged or examine fairness implications of alternative imputation techniques.Source: §5.2.1 · p. 29

5.2.2 Testing Hyper-parameters.

• Hyper-parameter testing searches for configurations that balance fairness and accuracy, using multi-objective optimization, Pareto-front search, statistical analysis, repair, and ensemble design [127] [129] [167].Fairway [127] [129] uses sequential model-based optimization, while Parfait-ML offers random, black-box evolutionary, and gray-box evolutionary searches.Source: §5.2.2 · p. 30, §5.2.3 · p. 30

5.2.3 Testing Fairness Repair Algorithms.

• Fairness repair techniques often improved fairness at the cost of accuracy or stability, while evaluation outcomes depended on bias type, measurement, and dataset [248, 311, 235, 236].Qian et al. reported reduced accuracy or increased fairness and accuracy variance; Zhang and Sun also found cases where fairness and accuracy both worsened.Source: §5.2.3 · p. 30
• Fairness-repair testing compares fairness-performance trade-offs and checks side effects such as lost desirable discrimination, with Fairea providing a unified baseline for such comparisons.Chen et al. [138] evaluated 17 mitigation methods across 12 performance metrics, 4 fairness metrics, and 24 trade-off measurements; fairness repair can also remove useful dependence on anti-protected attributes.Source: §5.2.3 · p. 30

5.2.4 Testing Compression Algorithms.

• Compression algorithms can change fairness inconsistently: pruning and quantization amplified gender bias, distillation reduced language-model bias, and facial-expression systems showed no consistent pattern.Testing commonly targets complex NLP and computer-vision models because compression is applied to large deep-learning models.Source: §5.2.4 · p. 31

5.3 Model Testing

• Individual-model fairness testing uses black-box or white-box approaches, distinguished by testers’ access to training data and internal model information.These techniques apply directly to final ML models [156] [160].Source: §5.3 · p. 31

5.3.1 Black-box Model Testing

• Black-box testing detects fairness bugs from input-output behavior using statistical comparisons, metamorphic transformations, domain-specific templates, and searches for discriminatory inputs.Examples include Tramèr et al., Bae, Themis [160], Aequitas, and ExpGA [156]. It is applied to NLP, computer vision, and ranking systems [102, 196, 290, 150, 185, 257, 256, 222, 286, 146, 148, 149, 170, 191, 193, 314, 114, 200, 232, 259, 297, 173].Source: §5.3.1 · pp. 31–32
• Some black-box methods approximate models with white-box surrogates, enabling symbolic execution or property-based generation to expose discriminatory inputs and fairness-property violations.Aggarwal uses a decision-tree approximation, while Sharma and Wehrheim specify fairness requirements through assume-assert constructs.Source: §5.3.1 · p. 32

5.3.2 White-box Model Testing

• White-box testing examines training data or accessible internal model information to identify fairness bugs, including bias explanation, bias-amplification measurement against training-data fairness baselines, subgroup fairness testing, and gradient-guided test-input generation.Supported examples include Chakraborty et al. [128], Zhao et al., Wang and Russakovsky, FAIRVIS [122], ADF, and EIDIG.Source: §5.3.2 · pp. 32–33

6.1 Research Venues

• Software engineering venues publish 49% of fairness-testing papers and AI venues 42%, while the field also spans NLP, HCI, security, and mobile-computing communities.Six recognized papers across software engineering, NLP, and HCI demonstrate substantial interest and research quality in fairness testing.Source: Figure 6 · p. 34, §6.1 · p. 34

6.2 Machine Learning Categories

• Deep-learning fairness testing accounts for 50% of papers, versus 41% for general ML and 8% covering both, with DL surpassing general ML after 2019.DL’s prominence reflects broad adoption and lower interpretability than traditional algorithms, which makes direct fairness reasoning more difficult.Source: §6.2 · pp. 34–35, Figure 7 · p. 35

6.3 Data Types

• Tabular-data applications comprise 55% of papers, while text and image applications each comprise 24%; speech and recommendation systems receive only about 1% each.Within software-engineering venues, tabular data rises to 79%, compared with 17% for text and 8% for image applications.Source: Figure 8 · p. 35, §6.3 · p. 36, Figure 9 · p. 36

6.4 Fairness Categories

• Research efforts are evenly divided between group fairness and individual fairness at 46% each, while 8% address both perspectives.Real-world data facilitates group-fairness inputs, whereas finding instance pairs differing only in a protected attribute makes individual-fairness test generation harder.Source: Figure 10 · p. 36, §6.4 · pp. 36–37

6.5 Testing Manners

• Black-box methods constitute 53% of fairness-testing papers, compared with 47% for white-box methods, partly because internal access is restricted by privacy and legal concerns.The survey classifies a technique requiring training-data access as white-box, even when its authors describe it as black-box or gray-box.Source: §6.5 · p. 37

6.6 Tasks and Sensitive Attributes

• Income prediction, credit-risk prediction, and recidivism/crime prediction are the most studied tasks, while sex/gender, race/ethnicity, and age are the leading sensitive attributes.Sex/gender, race/ethnicity, and age appear in 89%, 57%, and 41% of collected papers, respectively; the task pattern also matches bias-mitigation literature.Source: §6.6 · pp. 37–38, Table 6 · p. 38, Table 7 · p. 38

7.1 Public Datasets

• The survey compiles public fairness-testing datasets with information on size, data type, sensitive attributes, usage scenarios, and access links, many of which are tabular and suited to traditional ML classifiers.Text and image datasets have also become more available, but some impose usage constraints; CelebA, for example, is restricted to non-commercial research.Source: §7.1 · pp. 39–40

7.2 Open-source Testing Tools

• Despite proliferating fairness tools, practitioners face a steep learning curve and limited guidance on adoption.The survey’s tool summary is intended to help researchers and practitioners select suitable tools.Source: §7.2 · p. 40
• The section summarizes 41 open-source fairness-testing tools across general ML, DL, NLP, computer vision, and speech recognition to guide tool selection.Examples include FairTest, ADF, EIDIG, ASTRAEA, BiasFinder [102], REVISE, and AequeVox.Source: §7.2 · p. 40

8 RESEARCH OPPORTUNITIES

• Fairness testing remains relatively embryonic despite rapid research growth, leaving numerous open opportunities for improving how ML software fairness is evaluated.The survey frames the following challenges and directions as open problems in fairness testing.Source: §8 · p. 40

8.1 Absence of or with Multiple Sensitive Attributes

• Fairness testing must address unavailable or imperfect sensitive attributes and compounded effects across multiple intersecting attributes.Inferring sensitive attributes can be unsatisfactory, limited in scope, or create unintended consequences, while most existing work tests one attribute at a time.Source: §8.1 · pp. 40–42

8.2 Test Oracle for Fairness Testing

• Reliable automatic fairness oracles remain an open challenge because existing metamorphic and statistical oracles require human ingenuity, while fairness measurements are numerous and often mutually incompatible.IBM AIF360 alone offers more than 70 fairness measurements, making it impractical to use all available measures as test oracles.Source: §8.2 · p. 42

8.3 Test Input Generation for Fairness Testing

• Fairness test inputs should be both legitimate and natural, since feature perturbations can violate real-world constraints even when they induce the intended output behavior.Existing causal-fairness techniques often alter sensitive attributes without guaranteeing natural instances or constraining perturbation magnitude.Source: §8.3 · p. 42
• Counterfactual and multiple-attribute input generation remains immature because it requires causal analysis of feature relationships and impacts from simultaneous sensitive-attribute changes.Generating inputs for group fairness is also underexplored, partly because suitable real-world data can be difficult to obtain.Source: §8.3 · pp. 42–43

8.4 Test Adequacy for Fairness Testing

• Test adequacy is unaddressed in fairness testing, although adequacy criteria could provide confidence in testing and guide test generation.Traditional metrics include line, branch, and dataflow coverage; ML metrics include neuron coverage, layer coverage, and surprise adequacy.Source: §8.4 · p. 43
• No empirical evidence establishes that existing adequacy metrics detect fairness bugs or demonstrate sufficient fairness testing, motivating validation and fairness-specific metrics.Further research must assess the suitability of adapted traditional and ML adequacy measures and develop metrics capturing fairness-testing requirements.Source: §8.4 · p. 43

8.5 Test Cost Reduction

• Fairness testing needs cost-reduction techniques such as test selection, prioritization, and minimization, while supporting devices with limited computing power, memory, and energy.Existing fairness testing often requires model retraining, repeated predictions, or extensive data generation; resource-constrained platforms therefore require adapted methodologies.Source: §8.5 · p. 43

8.6 Fairness and Other Testing Properties

• Future fairness testing should examine trade-offs and relationships with accuracy, robustness, security, efficiency, interpretability, privacy, and explainability.Explainability can clarify why outcomes are unfair, guide targeted repairs, and help non-technical stakeholders understand software bias.Source: §8.6 · p. 44

8.7 Fairness Testing of More Applications

• Fairness testing should expand beyond tabular, natural-language, computer-vision, and classification systems to speech, video, multimodal, recommendation, regression, clustering, and emerging AI applications.The paper specifically highlights LLMs as an important but difficult target for fairness testing.Source: §8.7 · p. 44
• LLM fairness testing faces open-domain oracle and input-generation challenges, diverse responses, limited coverage of question types, opaque proprietary mechanisms, and inapplicability of conventional data and program testing.Existing solutions address oracle problems for Yes-No, Choice, and Wh questions, while prediction, explanatory, and recommendation questions remain unexplored.Source: §8.7 · pp. 44–45

8.8 More Fairness Testing Activities and Components

• Research should extend fairness testing from offline to online testing and investigate activities such as bug report analysis, bug triage, and test evaluation.Online testing can provide insights for software maintenance and system evolution.Source: §8.8 · p. 45
• Fairness testing should further examine ML frameworks and non-ML components, which remain underexplored compared with traditional ML testing.This gap represents a promising direction for broadening the components covered by fairness testing.Source: §8.8 · p. 45

8.9 More Fairness Testing Tools

• Fairness testing tools should become more accessible to compliance officers, policymakers, legal practitioners, and other non-technical stakeholders who may lack programming skills.Existing tools listed in Table 9 generally require programming expertise and are therefore unfriendly to these users.Source: §8.9 · p. 45

9.1 Stakeholders in Fairness Testing

• Fairness testing should involve engineers, data scientists, algorithm designers, legal practitioners, compliance officers, policymakers, and affected users.User-centered design and feedback loops can expose harms that conventional algorithmic testing misses, as illustrated by Yelp’s review system.Source: §9.1 · p. 45

9.2 Algorithmic Fairness and Societal Fairness

• Algorithmic fairness and societal fairness should coexist: improving algorithms offers short-term discrimination reduction while broader social change addresses underlying societal bias.Algorithmic fairness also raises awareness of societal consequences and provides a tangible step toward broader societal change.Source: §9.2 · p. 46
• Fairness requires empowering affected people to choose problems, guide development, and contribute lived experiences so training data reflects society’s diverse perspectives.Active participation helps address power imbalances and reduces participation-washing.Source: §9.2 · p. 46

9.3 Threats to Validity

• The survey may omit relevant studies because its DBLP-based search and manually defined strings do not cover every publication venue or study.Backward and forward snowballing, plus contacting collected-paper authors, were used to mitigate these collection risks.Source: §9.3 · p. 46
• Manual paper analysis may introduce human bias, mitigated through dual-author review, disagreement discussions, full-team checks, and validation by collected-paper authors.The reviewers had prior experience publishing fairness-related work in top-tier software-engineering venues.Source: §9.3 · pp. 46–47

10 CONCLUSION

• The survey synthesizes 100 papers across fairness-testing workflows and components, identifies trends and promising directions, and catalogs public datasets and open-source tools.It covers test input generation, test oracle identification, data testing, ML program testing, and model testing.Source: §10 · p. 47