Source-linked AI summary
Fairwashing: the risk of rationalization
Ulrich Aïvodji, Hiromi Arai, Olivier Fortineau, Sébastien Gambs, Satoshi Hara, Alain Tapp
TL;DR
The paper addresses the risk that black-box explanations can create a false perception of fairness for unfair machine-learning models. It proposes rationalization with LaundryML, which enumerates fairer interpretable rule-list surrogates, and demonstrates this risk on real-world data. The results show that such surrogates can retain high fidelity while being considerably less unfair.
Problem
Black-box explanations may be used to promote the false perception that an unfair machine-learning model respects ethical values.
Method
LaundryML uses regularized rule-list enumeration to find interpretable models approximating a black box while satisfying fairness and fidelity constraints.
Results
The experiments demonstrate that rationalization can produce rule lists with high fidelity to unfair black-box models while being considerably less unfair.
Takeaways & Limitations
Fairwashing is feasible through both global model explanation and local outcome explanation, motivating awareness and detection research.
Takeaways & Limitations
The evaluation adopts demographic parity as its group-fairness criterion, while broader fairness metrics and detection approaches remain future extensions.
Abstract
from arXiv · showhide
Black-box explanation is the problem of explaining how a machine learning model -- whose internal logic is hidden to the auditor and generally complex -- produces its outcomes. Current approaches for solving this problem include model explanation, outcome explanation as well as model inspection. While these techniques can be beneficial by providing interpretability, they can be used in a negative manner to perform fairwashing, which we define as promoting the false perception that a machine learning model respects some ethical values. In particular, we demonstrate that it is possible to systematically rationalize decisions taken by an unfair black-box model using the model explanation as well as the outcome explanation approaches with a given fairness metric. Our solution, LaundryML, is based on a regularized rule list enumeration algorithm whose objective is to search for fair rule lists approximating an unfair black-box model. We empirically evaluate our rationalization technique on black-box models trained on real-world datasets and show that one can obtain rule lists with high fidelity to the black-box model while being considerably less unfair at the same time.
1. Introduction
Black-box explanations can make high-stakes machine-learning decisions interpretable, but vague standards for valid explanations create opportunities to conceal unfairness. The paper shows that interpretable surrogates can rationalize unfair black boxes and enable fairwashing.
- Motivation: High-stakes machine-learning use and harmful decisions have increased demand for explanations of algorithmic outcomes.Examples include credit scoring, predictive justice, medical diagnosis, and wrongful parole denial.
- Existing approaches: Explainability approaches include transparent-box design and post-hoc black-box explanation, whose internal logic is hidden from auditors.Black-box explanation reverse-engineers a model to create an interpretable representation without requiring cooperation from its operator.
- Risk: Black-box explanations can be unreliable or misleading, including through model explanation, outcome explanation, and model inspection.Model explanation targets the whole black-box logic, whereas outcome explanation focuses on a specific decision.
- Fairwashing: Regulations require explanations for consequential decisions but do not specify precisely what constitutes a valid explanation, leaving room to conceal unfairness.The paper frames this loophole as an opportunity for dishonest companies to provide misleading explanations.
- Contribution: The paper demonstrates that a fairer explanation can be forged from a genuinely unfair black box through rationalization.This establishes fairwashing as a concrete risk of black-box explanation.
- Contribution: Rationalization systematically produces interpretable models approximating a black box while scoring as fairer under a predefined fairness metric.The technique supports both global model explanation and local outcome explanation and is agnostic to the black-box model and fairness metric.
2. Related work
The paper situates its approach among interpretable models, black-box explanation tasks, fairness metrics, rule lists, and exact rule-list enumeration. These foundations define how explanations, fairness, and regularized rule-list search are used in the paper.
- Interpretability and explanation: Interpretability means explaining or giving a decision process understandable meaning, often through models such as trees, linear models, rule lists, or rule sets.The paper focuses on model explanation and outcome explanation as two black-box explanation tasks.
- Interpretability and explanation: Model explanation seeks a global interpretable predictor derived from a black-box model and instances.A decision tree can serve as an explanation when it approximates the black-box model well.
- Interpretability and explanation: Outcome explanation seeks a local interpretable explanation for a black-box decision on one instance.Linear models approximating the black box near that instance are examples, including LIME and SHAP-style approaches.
- Fairness in machine learning: The paper adopts group fairness, requiring approximate equalization of a statistical property across groups defined by a sensitive attribute.Demographic parity equalizes decision distributions across sensitive-attribute groups, whereas equalized odds equalizes false-positive and false-negative rates.
- Fairness in machine learning: Fairness tools such as FairML inspect attribute importance but cannot directly provide model or outcome explanations.This separates model inspection from the explanatory tasks examined in the paper.
- Rule lists: A rule list applies ordered association rules sequentially and returns the corresponding outcome when a rule triggers, otherwise using a default prediction.Rule lists are equivalent to comb-shaped decision trees.
- Rule-list enumeration: CORELS searches binary-predictor rule lists using regularized empirical risk, penalizing longer lists with λ.It uses branch-and-bound pruning, while Lawler-based enumeration successively computes optimal rule lists and constructs excluded subproblems.
3. Rationalization
The paper formalizes fairwashing as presenting an interpretable surrogate that closely approximates a black-box model while appearing fairer, for both global and local explanations. LaundryML addresses this by enumerating regularized rule lists that balance fidelity, unfairness, and list length.
- Problem formulation: Fairwashing uses an interpretable surrogate that approximates a black-box model while having lower unfairness under a predefined fairness metric.The paper distinguishes global model rationalization from local outcome rationalization around an instance.
- Problem formulation: The paper restricts its formal setting to binary attributes, binary classification, and one sensitive attribute, while noting extensions to multi-valued and multiple sensitive attributes.Multi-valued attributes can be converted to binary attributes using one-hot encoding.
- Performance metrics: The rationalization evaluation jointly considers surrogate fidelity to the black box and unfairness under demographic parity.Global fidelity is accuracy relative to black-box predictions, while local fidelity is whether the surrogate matches the black-box prediction for the considered instance.
- LaundryML: Because optimizing fidelity and unfairness simultaneously is difficult, the approach enumerates multiple high-fidelity, low-unfairness models and selects one for rationalization.Selection may use additional criteria or human inspection, and evaluation is conducted on real-world datasets.
- LaundryML: LaundryML enumerates rule lists using a regularized objective that accounts for both fidelity-related error and unfairness, with regularization controlling list length.The algorithm is presented in model-rationalization and outcome-rationalization versions.
- LaundryML: The model-rationalization version labels instances with black-box predictions before passing the resulting dataset to LaundryML.The local outcome version instead constructs a neighborhood around the considered instance and runs LaundryML on that neighborhood.
4. Experimental evaluation
The evaluation tests whether LaundryML can produce interpretable explanations that remain faithful to unfair black-box models while reducing measured unfairness. Across global and local rationalization scenarios, the experiments show substantial fairness improvements, with a fidelity–fairness trade-off and a scope limitation for suing-group explanations.
- Experimental setting: The experiments evaluate fidelity and unfairness for global model rationalization and local outcome rationalization on Adult Income and ProPublica Recidivism.Random-forest black boxes label suing groups used to train interpretable surrogate models, while test sets evaluate black-box performance.
- S1: Responding to a suing group: As β increases, LaundryML-global produces models with lower unfairness and lower fidelity, while lower λ permits more complex models with higher fidelity.The evaluation visualizes this fidelity–fairness trade-off to select a fairness threshold.
- S1: Responding to a suing group: 0.908 fidelity and 0.058 unfairness were obtained on Adult Income, while ProPublica Recidivism reached 0.748 fidelity and 0.080 unfairness.These are the selected best models among those at least twice less unfair than the original black-box model.
- S1: Responding to a suing group: FairML ranked sensitive attributes substantially lower in the surrogate models: gender moved from second to last on Adult Income, and race from fifth to thirteenth on ProPublica Recidivism.The comparison concerns gender:Male for Adult Income and race:Caucasian for ProPublica Recidivism.
- S2: Responding to an individual claim: With β = 0.9, LaundryML-local found models with unfairness = 0.0 for each rejected Adult Income minority-group user, while ProPublica models were at least twice less unfair than the black boxes.The local scenario rationalizes outcomes independently for individual minority-group users.
- Scope limitation: The suing-group explanation can be customized to the original group: fidelity on the test set tends to be slightly lower, so it may rationalize additional members less effectively.The paper identifies detecting or constructing difficult-to-detect rationalizations as future work.
5. Conclusion
The paper introduces rationalization and fairwashing risks, proposes LaundryML for enumerating fair rule-list explanations, and demonstrates feasibility on real-world data. It also identifies extensions toward broader models and metrics, plus future work on detecting rationalization.
- LaundryML enumerates rule lists that incorporate fairness as a constraint alongside fidelity, enabling rationalization of unfair black-box models.
- The experiments on real-world datasets demonstrate that fairwashing can be achieved through both model explanation and outcome explanation.
- The framework can be extended to other interpretable models and fairness metrics, as shown in additional appendix experiments.
- Future work targets the social implications of fairwashing and methods for estimating whether an explanation is a rationalization.
- FairML audits found surrogate models fairer by ranking sensitive attributes substantially lower than in the corresponding black-box models.
A. Generalization to other fairness metrics
LaundryML-global is evaluated with four fairness metrics while holding the Adult Income dataset and Random Forest black-box model fixed. Across these metrics, it finds explanation models suitable for fairwashing, with the results described as agnostic to the fairness metric.
- The experiments compare overall accuracy equality, statistical parity, conditional procedure accuracy, and demographic parity using the same Random Forest model and Adult Income dataset.
- LaundryML-global finds explanation models for fairwashing under each of the four evaluated fairness metrics.
- When black-box unfairness is high, defined as unfairness ≥0.1, higher unfairness regularization produces the reported performance pattern.
B. Generalization to other black-Box models
LaundryML-global is evaluated across Random Forest, SVM, XGBOOST, and MLP black-box models under demographic parity. It finds explanation models suitable for fairwashing for each model type, indicating model-agnostic feasibility in these experiments.
- The experiments vary the black-box model among Random Forest, SVM, XGBOOST, and MLP while fixing LaundryML-global, demographic parity, and the Adult Income setting.
- LaundryML-global finds explanation models for fairwashing for each evaluated black-box model type.
- Overall, the results confirm that fairwashing is possible across the black-box model types considered.
- The figures report unfairness and fidelity CDFs for SVM and XGBOOST black-box models under demographic parity.
- The MLP experiment likewise reports unfairness and fidelity CDFs for rationalized explanations under demographic parity.