PaperlayerView exact revision on arXiv

Source-linked AI summary

Information-Theoretic Testing and Debugging of Fairness Defects in Deep Neural Networks

Verya Monjezi, Ashutosh Trivedi, Gang Tan, Saeid Tizpaz-Niari

arXiv:2304.04199v1cs.SEcs.LG

TL;DR

Fairness testing can find discriminatory DNN decisions, but developers also need ways to assess severity and determine whether defects reflect training data or inadequate training. DICE quantifies protected information in decisions and uses search and causal debugging; experiments report improved instance generation and neuron-guided mitigation across ten socio-critical DNNs.

  • Problem

    Existing search-based testing discovers fairness defects, but developers lack severity and causal explanations to assess whether defects reflect training data or inadequate training.

    Source: §I · p. 1

  • Method

    DICE measures protected information in DNN decisions with QID, searches for discriminatory inputs using quantitative feedback, and applies causal debugging to localize layers and neurons.

    Source: §I · p. 1

  • Results

    Across 10 socio-critical DNNs, DICE outperformed three state-of-the-art techniques in generating discriminatory instances and reduced QID to 15% of its initial level with at most 5% accuracy loss.

    Source: §I · p. 2

  • Takeaways & Limitations

    Severity ordering and causal localization provide developers with aids for triaging fairness defects and identifying layers or neurons associated with them.

    Source: §I · p. 1

    Perturbing all protected values can produce unrealistic counterfactuals and false positives, though domain-specific constraints can mitigate this overapproximation.

    Source: §VII · p. 9

Abstract

from arXiv · show

The deep feedforward neural networks (DNNs) are increasingly deployed in socioeconomic critical decision support software systems. DNNs are exceptionally good at finding minimal, sufficient statistical patterns within their training data. Consequently, DNNs may learn to encode decisions -- amplifying existing biases or introducing new ones -- that may disadvantage protected individuals/groups and may stand to violate legal protections. While the existing search based software testing approaches have been effective in discovering fairness defects, they do not supplement these defects with debugging aids -- such as severity and causal explanations -- crucial to help developers triage and decide on the next course of action. Can we measure the severity of fairness defects in DNNs? Are these defects symptomatic of improper training or they merely reflect biases present in the training data? To answer such questions, we present DICE: an information-theoretic testing and debugging framework to discover and localize fairness defects in DNNs. The key goal of DICE is to assist software developers in triaging fairness defects by ordering them by their severity. Towards this goal, we quantify fairness in terms of protected information (in bits) used in decision making. A quantitative view of fairness defects not only helps in ordering these defects, our empirical evaluation shows that it improves the search efficiency due to resulting smoothness of the search space. Guided by the quantitative fairness, we present a causal debugging framework to localize inadequately trained layers and neurons responsible for fairness defects. Our experiments over ten DNNs, developed for socially critical tasks, show that DICE efficiently characterizes the amounts of discrimination, effectively generates discriminatory instances, and localizes layers/neurons with significant biases.

I. INTRODUCTION

DICE quantifies protected information in DNN decisions to rank fairness defects, search for discriminatory inputs, and localize layers and neurons associated with those defects. Experiments across ten socio-critical DNNs report improved discovery and a neuron-guided mitigation that reduces QID while limiting accuracy loss.

Quantifying Fairness

  • Existing fairness testing can discover defects, but developers need severity and causal explanations to triage them and assess whether they reflect training data or inadequate training.DNNs may encode or amplify biases that disadvantage protected groups and may violate statutory requirements.Source: §I · p. 1
  • QID measures protected information used to derive an outcome, allowing discriminatory inputs to be ordered by severity; zero QID implies no individual discrimination.The measure uses Shannon or min entropy.Source: §I · pp. 1–2

Search-Based Testing

  • QID-based search provides smooth quantitative feedback, unlike prior fairness-testing methods that use binary feedback and create a discontinuous search domain.The paper investigates whether this guided search finds richer discriminatory instances.Source: §I · p. 2

Causal Explanations

  • DICE uses causal interventions to localize layers and neurons affecting QID, distinguishing input-layer causes indicative of training-data discrimination from internal-layer causes.For internal layers, the framework seeks neurons and counterfactual parameters that can mitigate defects while maintaining accuracy.Source: §I · p. 2

Experiments

  • Across 10 socio-critical DNNs, DICE outperformed three state-of-the-art techniques in generating discriminatory instances and reduced QID to 15% of its initial level with at most 5% accuracy loss.The localized neurons guided a simple mitigation strategy.Source: §I · p. 2

Experimental Evaluation

  • DICE supports multiple protected attributes and can be adapted to regression tasks.The paper reports experiments over different datasets and DNN models.Source: §I · p. 2

II. PRELIMINARIES

The preliminaries distinguish group and individual fairness, then frame individual discrimination through counterfactuals and information-theoretic measures. The examples show that Shannon and min entropy can rank the same cases differently.

Fairness Terminology

  • Group fairness compares outcome statistics across protected groups, while individual fairness asks whether similar individuals, differing only in protected attributes, receive similar outcomes.The passage also describes fairness through unawareness and its potential inadequacy when protected attributes influence predictions through non-protected attributes.Source: §II · p. 2

Individual Discrimination

  • Individual discrimination adapts counterfactual queries to find an individual whose outcome changes favorably under different protected attributes; quantitative measures for this notion were not established in the cited state of the art.The paper proposes information-theoretic measures to quantify it.Source: §II · p. 2

Information-Theoretic Concepts

  • For a deterministic DNN with uniformly distributed inputs, approximate output equivalence classes define the remaining uncertainty after observing the model output.The passage introduces the equivalence relation used to describe those classes.Source: §II · p. 3

Quantitative Notion and Fairness

  • The approach extends individual discrimination with quantitative information flow: Shannon entropy measures expected discrimination across responses, while min entropy measures it for the maximum-likelihood response.The paper focuses on individual fairness rather than group fairness.Source: §II · pp. 2–3

Example 1

  • Shannon entropy assigns 2.0 bits of discrimination to the third individual and 1.875 to the fourth, whereas min entropy assigns 2.0 and 2.3, respectively.Thus, the two measures can prioritize these cases differently.Source: §II · p. 3

III. OVERVIEW

DICE combines quantitative fairness testing with causal debugging to identify discriminatory inputs and influential neurons. Its overview reports stronger instance generation than three baselines, alongside an example of QID measurement and mitigation.

DICE in Nutshell

  • DICE searches globally to increase discrimination, then locally expands promising seeds into discriminatory instances using threshold-based clustering for smooth feedback.Figure 1 depicts the overall testing and causal-debugging workflow.Source: §III · p. 3, Figure 1 · p. 4
  • DICE intervenes on neurons, compares QID across activation states, and reports neurons whose activation increases or reduces discrimination.The authors describe keeping positively influential neurons active or negatively influential neurons inactive as a potential mitigation strategy.Source: §III · p. 4

Test Cases

  • In the adult census example, DICE found 4.05 bits of min-entropy QID and 2.64 bits of Shannon-entropy QID, out of 5.3 protected-attribute bits.After one hour, it generated 230,593 test cases; six had the maximum 5.3-bit QID.Source: §III · p. 4

Localization and Mitigation

  • 20% lower QID discrimination with 3% accuracy loss followed intervention on the second layer’s most influential neurons in the census model.Deactivating neuron 15 reduced discrimination by 19.6%, while activating neuron 19 reduced it by 17.6%.Source: §III · p. 4

Comparison to the State-of-the-art

  • 79.0k instances for DICE versus 10.4k for AEQUITAS on census sex discrimination; DICE generated more ID instances overall but found the first more slowly.The other reported counts are 18.2k for ADF and 21.6k for NEURONFAIR; DICE outperformed the baselines in time to the first 1,000 instances.Source: §III · p. 4

IV. PROBLEM STATEMENT

The problem statement models a DNN as a classifier over protected and non-protected inputs. It defines QID as a quantitative measure of discrimination and frames testing and debugging as optimization problems.

DNN Semantics and Syntax

  • A DNN maps protected and non-protected inputs to class probabilities; its predicted label is the class with the highest score.The formulation assumes finite-domain protected inputs and a known, pre-trained network.Source: §IV · pp. 4–5

Individual Discrimination

  • Individual discrimination occurs when two inputs differing only in protected features receive different outcomes, including a more favorable outcome for one.The section describes the comparison as potentially counterfactual.Source: §IV · p. 5

Quantitative Individual Discrimination

  • QID groups protected values into equivalence classes when their DNN outputs are approximately close, quantifying protected information used in an individual’s outcome.The section uses Shannon and min entropy to quantify QID, with class sizes and the number of classes entering the measures.Source: §IV · p. 5

Quantitative Fairness Testing and Debugging

  • Testing seeks a non-protected input that maximizes QID over protected values; debugging seeks layers and neurons with the largest causal effects on QID.Neuron interventions are evaluated subject to preserving DNN functionality, including accuracy constraints.Source: §IV · p. 5

V. APPROACH

DICE searches for high-QID inputs using gradient-guided clustering, then uses causal interventions to localize influential layers and neurons. Its neuron-level comparisons constrain interventions to preserve model accuracy.

Characterizing Quantitative Individual Discrimination

  • DICE prioritizes worst-case test inputs with min-entropy QID, then searches for inputs that maximize the number of distinguishable output classes.The search also uses the distance between classes to break ties when inputs yield the same class count.Source: §V · p. 5

Search Approach

  • The global phase perturbs non-protected features using gradients from samples in the largest cluster, aiming to split that cluster and increase QID.The local phase starts from promising instances and uses a continuous cluster-based objective to generate more discriminatory instances.Source: §V · pp. 5–6

Search Procedure

  • The search procedure seeds instances from data partitions, clusters predictions for protected-value variants, and enters local search when cluster count or distance increases.The algorithm runs until its timeout and returns the cluster count and generated test cases.Source: §V · p. 6

Debugging Approach

  • DICE first ranks layers by changes in sensitivity to protected attributes, then compares QID under neuron activation and deactivation while constraining accuracy loss.It returns neurons with the largest positive and negative average causal differences; the authors describe opposite activation choices as a potential mitigation.Source: §V · pp. 6–7

Debugging Procedure

  • The debugging procedure selects the layer with the largest sensitivity change, tests neuron values with minimal accuracy impact, and returns neurons with large causal effects.It uses statistics of neuron outputs to choose intervention values before computing average causal difference.Source: §V · p. 7

VI. EXPERIMENTS

The experiments use socially critical datasets and six-layer fully connected networks, and pose questions about QID characterization, instance generation, and causal debugging. The supplied passages describe the setup and questions, not their answers.

Datasets and DNN models

  • The experimental setup uses 10 socially critical datasets and six-layer fully connected networks with 64, 32, 16, 8, 4, and 2 neurons.Training uses 1,000 epochs, batch size 128, and learning rate 0.01.Source: §VI · p. 7

Research Questions

  • The study asks whether DICE characterizes protected information, generates discriminatory instances efficiently, and uses causal debugging to localize and mitigate discrimination.The passage identifies these as the three research questions.Source: §VI · p. 7

Technical Details

  • The paper states that the open-source DICE tool and experimental subjects are publicly accessible.The experiments average results over 10 runs and report specified search and debugging parameters.Source: §VI · p. 7

A. Characterizing QID via Search (RQ1)

DICE uses QID to characterize how much protected information informs DNN decisions and to rank discriminatory cases by severity. In its search experiments, it expanded the number of clusters and found cases using substantial amounts of protected information.

A. Characterizing QID via Search (RQ1)

B. Individual Discriminatory Instances (RQ2)

DICE generated more individual discriminatory instances than the three comparison tools in the reported experiments, though it took longer to find the first instance. Its local-search success rate and time to generate 1,000 instances were also compared.

B. Individual Discriminatory Instances (RQ2)

C. Causal Debugging of DNNs for Fairness (RQ3)

DICE’s causal debugging approach localized neurons with fairness effects and used neuron interventions to mitigate discrimination. The reported mitigation reduced QID discrimination, with accuracy losses below 5%.

C. Causal Debugging of DNNs for Fairness (RQ3)

  • Across localized neurons, average causal differences ranged from 4% to 55% for positive fairness effects and from 0.6% to 18.3% for negative effects.Layer 2 was localized more often than other layers; layers 3, 4, and 5 were each localized once.Source: C. Causal Debugging of DNNs for… · p. 9
  • Activating selected neurons reduced QID discrimination by at least 5% and up to 64.3%, with 3% and 2% accuracy loss, respectively.Source: C. Causal Debugging of DNNs for… · p. 9
  • De-activating selected neurons improved fairness by at least 6% and up to 27%, with 1% and 2% accuracy loss, respectively.Source: C. Causal Debugging of DNNs for… · p. 9
  • After DICE localized neurons, a mitigation strategy could reduce discrimination by at least 6% and up to 64.3%, with less than 5% loss of accuracy.Source: C. Causal Debugging of DNNs for… · p. 9

VII. DISCUSSION

DICE’s evaluation is bounded by counterfactual realism, dynamic-testing coverage, and questions about whether the tested models represent fairness testing broadly. The discussion also presents comparisons of discriminatory-instance generation, causal neuron localization, and mitigation-related measures.

Limitations

  • Unrealistic protected-attribute perturbations may produce counterfactuals that overapproximate discrimination, while domain-specific constraints can mitigate this risk.The authors also note that dynamic testing may miss discriminatory inputs, and that the debugging method does not explain or interpret black-box DNN functions.Source: §VII · p. 9

Evaluation

  • Table III compares discriminatory-instance generation across tools over 900 seconds, reporting instance counts, local-search success, and times to the first and 1,000th instances.Results are averaged over 10 runs; deviations from the mean appear in parentheses.Source: Table III · p. 10

Localization

  • The localization table reports neurons with positive or negative fairness influence and normalized average causal differences for each direction.The table concerns layers and neurons that causally influence fairness.Source: Table IV · p. 10, §VII · p. 10

Mitigation

  • Table V reports accuracy, test-case cluster counts, and computation time after deactivating the most negative-impact neuron or activating the most positive-impact neuron.The caption defines the corresponding A=0, K=0, A>0, K>0, and TI measures.Source: Table V · p. 10

Threat to Validity

  • Whether datasets and DNN models from fairness-testing literature are sufficiently representative remains an open question.The validity discussion reports repeated experiments and evaluation on 10 DNN models from that literature.Source: §VII · pp. 9–10

VIII. RELATED WORK

DICE is positioned against work on fairness testing, DNN debugging, and bias mitigation, while the section also identifies related formal-methods research and future application areas. Its stated distinction from existing fairness testers is quantitative discrimination measurement and support for prioritizing test cases.

Fairness Testing of ML systems

  • Against existing fairness testers,,, DICE quantifies discrimination, supports test-case prioritization, and searches multiple protected attributes at once.The related work describes prior approaches to individual fairness testing, including black-box and gradient-guided methods.Source: §VIII · pp. 10–11

Group Fairness

  • Group-fairness testing and mitigation work,,,,, is outside this paper’s scope.The section describes FAIRWAY for post-discovery configuration mitigation and PARFAIT-ML for bias magnitude across classic-ML hyperparameters.Source: §VIII · p. 11

Debugging of Deep Neural Network

  • DICE adapts layer localization from CRADLE and AUDEE, using causal reasoning for fairness rather than correlation.The comparison notes that AUDEE perturbs neuron values for functional correctness without accounting for the accuracy or severity of neuron contributions to a bug.Source: §VIII · p. 11

In-process Mitigation

  • Unlike parameter- or loss-constraint methods, and exponentiated-gradient metalearning, DICE specializes mitigation to neural networks and individual fairness.The authors propose exploiting layer structure for causal reasoning and mitigation, and suggest extension with in-process techniques.Source: §VIII · p. 11

Formal Methods

  • Formal-methods examples include probabilistic-program fairness certification with FAIRSQUARE and fair decision-tree inference with SFTREE.Source: §VIII · p. 11

Fairness in income, wealth, and taxation

  • The authors leave further study of income, wealth, and taxation fairness to future work, although their approach is geared toward regression problems.Potential applications include examining income and wealth distributions across race and gender and fairness in taxation.Source: §VIII · p. 11

IX. CONCLUSION

DICE characterizes protected information used in DNN-based decisions and applies quantitative fairness to testing and debugging. Experiments show that its search and debugging algorithms discover and localize fairness defects.

Conclusion

  • DICE uses an information-theoretic model to characterize the amounts of protected information used in DNN-based decision making.Source: §IX · p. 11
  • Experiments show that DICE’s quantitative-landscape-based search and debugging algorithms effectively discover and localize fairness defects.Source: §IX · p. 11
  • The conclusion situates the work in socio-critical applications where bugs in DNN-based software may lead to discriminatory behavior.Source: §IX · p. 11
Loading 2304.04199v1…