Source-linked AI summary
A causal framework for discovering and removing direct and indirect discrimination
Lu Zhang, Yongkai Wu, Xintao Wu
TL;DR
The paper asks how to discover and remove direct and indirect discrimination from historical data before predictive analysis. It models the two forms as path-specific effects in a causal network and develops discovery and removal algorithms. Experiments on real datasets show that the modified data and resulting predictive models contain no direct or indirect discrimination while retaining data utility.
Problem
Historical data can contain discrimination that predictive models learn, while prior metrics may conflate direct, indirect, and objectively explainable effects.
Method
The paper uses causal networks and path-specific effects to measure direct and indirect discrimination, then modifies the decision conditional probability table to remove both effects.
Results
Modified Adult data had direct and indirect effects of 0.013 and 0.049, while modified Dutch data had effects of 0.050 and 0.001; both modified datasets contained no direct or indirect discrimination.
Takeaways & Limitations
The approach ensures that predictive models built from modified data are not subject to either type of discrimination, with relatively small data-utility loss and good prediction accuracy.
Takeaways & Limitations
The paper treats causal-network construction and probabilistic-inference performance as outside its scope.
Abstract
from arXiv · showhide
Anti-discrimination is an increasingly important task in data science. In this paper, we investigate the problem of discovering both direct and indirect discrimination from the historical data, and removing the discriminatory effects before the data is used for predictive analysis (e.g., building classifiers). We make use of the causal network to capture the causal structure of the data. Then we model direct and indirect discrimination as the path-specific effects, which explicitly distinguish the two types of discrimination as the causal effects transmitted along different paths in the network. Based on that, we propose an effective algorithm for discovering direct and indirect discrimination, as well as an algorithm for precisely removing both types of discrimination while retaining good data utility. Different from previous works, our approaches can ensure that the predictive models built from the modified data will not incur discrimination in decision making. Experiments using real datasets show the effectiveness of our approaches.
Introduction
The paper addresses discrimination in predictive decision-making by distinguishing direct and indirect discriminatory effects in historical data. It uses causal networks and path-specific effects to discover and remove these effects before predictive analysis.
- Motivation: Historical discrimination can be learned by predictive models and reproduced in employment, credit, and insurance decisions.The paper therefore focuses on preventing discrimination in both input data and assisted decisions.
- Research gap: Existing discrimination metrics often combine direct discrimination, indirect discrimination, and objectively explainable effects.The paper identifies this conflation as a central limitation of prior discovery techniques.
- Approach: A causal network represents causal paths from protected attributes to decisions, allowing direct and indirect discrimination to be modeled as effects transmitted along different paths.Direct discrimination follows the protected-attribute-to-decision path, whereas indirect discrimination follows paths containing unjustified attributes.
- Approach: The framework uses path-specific effects to measure direct and indirect discrimination from observational data.These measurements support separate discovery and removal algorithms.
- Contributions: The proposed removal procedure aims to eliminate both discriminatory effects while retaining good data utility and preventing discrimination in predictive models built from modified data.Experiments on two real datasets are reported as evidence of effectiveness.
Preliminary Concepts
The preliminary framework uses causal graphs and interventions to represent causal effects, then restricts interventions to selected paths through path-specific effects. Identifiability depends on whether the recanting witness criterion is satisfied.
- Causal networks: A causal network is a directed acyclic graph whose nodes are attributes and whose arrows represent direct causal relationships.Each node has a conditional probability table, and the joint distribution is obtained through factorization.
- Causal effects: Do-calculus represents interventions that force attributes to specified values and estimates post-intervention distributions from observational data.Marginalization sums over value combinations of attributes other than the intervened and outcome attributes.
- Causal effects: The total causal effect transmits an intervention along all causal paths from a cause to an effect.This provides the baseline for defining effects restricted to selected paths.
- Path-specific effects: A path-specific effect transmits an intervention only along a subset of causal paths while leaving other paths under the non-intervened value.It extends total causal effect by separating the paths through which an intervention operates.
- Identifiability: The recanting witness criterion identifies a node that lies on both included and excluded path segments between the treatment and outcome.If the criterion is satisfied, the corresponding path-specific effect cannot be estimated from observational data.
- Identifiability: When the criterion is not satisfied, the path-specific effect is computed by partitioning the treatment’s children according to whether their arcs belong to the selected paths.The computation requires the selected and unselected child sets to be disjoint.
Modeling Direct and Indirect Discrimination
The framework represents direct and indirect discrimination as distinct path-specific causal effects of a protected attribute on a decision. It uses these effects to define measurable, threshold-based discrimination criteria.
- Causal Model: The model assumes a causal DAG over attributes, with a protected attribute C having no parent and a binary decision E.The protected attribute is treated as an inherent individual characteristic, and the graph is learned or constructed to represent the dataset’s causal structure.
- Direct Discrimination: Direct discrimination is the path-specific effect transmitted only along the direct path C → E.Its interpretation is the expected change in positive decisions when protected-group membership changes while everything else remains unchanged.
- Indirect Discrimination: Indirect discrimination is the effect transmitted along all paths from C to E that pass through designated redlining attributes R.This corresponds to changing protected-group individuals’ profiles along those paths as if they belonged to the other group.
- Discrimination Criteria: Direct discrimination is claimed when either directional direct path-specific effect exceeds the user-defined threshold τ; the same rule applies to indirect discrimination.The threshold depends on applicable law, with 0.05 given as an example for British sex-discrimination legislation.
- Effect Decomposition: The total causal effect equals the path-specific effect when the path set contains all causal paths, while direct and indirect effects need not sum linearly.The framework therefore extends risk difference by separating effects transmitted along different paths.
- Identification: Direct effects are always measurable from observational data, whereas indirect effects may be unidentifiable when the recanting witness criterion is satisfied.When that criterion is not satisfied, the indirect effect can be computed after partitioning C’s children into path-relevant and non-path-relevant sets.
Discrimination Discovery and Removal
The paper proposes algorithms to discover direct and indirect discrimination and to modify historical data before predictive analysis. Removal changes the decision mechanism while minimizing distributional change, so models trained on the modified data can learn the anti-discriminatory adjustment.
- Discrimination Discovery: PSE-DD builds a causal network and computes direct and indirect path-specific effects to judge discrimination against a user-defined threshold.Its output is a judgment for direct discrimination and a judgment for indirect discrimination.
- Discrimination Discovery: PSE-DD complexity depends on causal-network construction, path-specific-effect computation, and identifying path partitions for indirect discrimination.A straightforward search for all indirect paths may be exponential, although graph-based membership tests are available.
- Discrimination Removal: PSE-DR modifies the decision node’s conditional probability table so both direct and indirect discriminatory effects fall below τ.It then computes the modified joint distribution and generates a new dataset from it.
- Discrimination Removal: PSE-DR minimizes Euclidean distance between the original and modified joint distributions to retain good data utility.The modified conditional probabilities are obtained by solving a quadratic programming problem subject to probability constraints.
- Predictive Models: Changing only the decision node’s conditional probability table lets predictive models learn the modification and avoids biased decision making according to the framework.The authors state that this is intended to ensure nondiscriminatory predictive models, not merely nondiscriminatory released data.
- Nondiscrimination Under Nonidentifiability: When indirect discrimination cannot be accurately measured because of the recanting witness criterion, the data owner may still remove discrimination.This addresses the case where nondiscrimination is required despite nonidentifiability of the indirect path-specific effect.
Experiments
Experiments on Adult and Dutch datasets evaluate causal-network-based discrimination discovery and removal, including effects on predictive models, utility, and accuracy. PSE-based removal eliminates both direct and indirect discrimination in modified data, while comparisons show trade-offs for prior methods.
- Experimental setup: Experiments compare PSE-based methods with LMSG, LPS, and DI on the Adult and Dutch datasets using causal networks.The Adult and Dutch causal graphs represent direct paths and indirect paths through marital status.
- Discrimination discovery: On Adult, direct discrimination is 0.025 and indirect discrimination is 0.175, indicating only significant indirect discrimination at threshold τ = 0.05.Sex is protected, income is the decision, and marital status is the redlining attribute.
- Discrimination removal: After PSE-DR, Adult effects are 0.013 direct and 0.049 indirect, while Dutch effects are 0.050 direct and 0.001 indirect; both modified datasets contain no discrimination.PSE-DD is run after removal to examine whether discrimination remains.
- Predictive models: PSE-DD yields discrimination-free modified training data and testing predictions, with relatively small χ2 utility loss and good prediction accuracy.The comparison evaluates both modified training data and predictions made by classifiers trained on it.
- Comparison: DI removes discrimination from training data but leaves indirect effects of 0.167/0.168 in predictions and has substantially worse data utility than PSE-DR.DI selects λ values near τ = 0.05, but λ has no direct connection with τ.
Related Work
Prior work uses rule-based, similarity-based, conditional, causal, preprocessing, and algorithm-modification approaches to discrimination discovery and removal. The paper distinguishes itself by explicitly separating direct discrimination, indirect discrimination, and explainable effects.
- Discrimination discovery: Rule-based methods such as elift and belift represent particular discrimination patterns, while similarity-based methods address individual discrimination.Similarity-based approaches find groups of similar individuals.
- Discrimination discovery: Conditional discrimination methods allow some discrimination to be explainable by selected attributes, but earlier approaches do not explicitly identify direct and indirect discrimination separately.The cited causal framework uses a Suppes-Bayes causal network but is presented as related work.
- Discrimination removal: Removal methods generally rely on data preprocessing or algorithm tweaking.These categories include several earlier discrimination-removal techniques.
- Discrimination removal: A prior method removes the protected attribute and modifies non-protected attributes so the protected attribute cannot be predicted from them, thereby targeting indirect discrimination.The approach first ensures no direct discrimination and then addresses indirect discrimination.
Conclusions
The paper uses causal networks and path-specific effects to discover and remove direct and indirect discrimination from historical data. Experiments show that its modified datasets and resulting predictive models are not subject to either type of discrimination.
- Contributions: The framework models direct and indirect discrimination as different path-specific effects in a causal network.The causal network captures the data’s causal structure and distinguishes effects transmitted along different paths.
- Contributions: PSE-DD discovers direct and indirect discrimination, while PSE-DR removes both types before predictive analysis.The algorithms are designed for historical data preprocessing before building predictive models.
- Results: Experiments on real datasets show that only the proposed approach ensures predictive models built from modified data are free of both discrimination types.The conclusion specifically concerns discrimination in decision making by the resulting predictive models.