Source-linked AI summary
Towards Realistic Individual Recourse and Actionable Explanations in Black-Box Decision Making Systems
Shalmali Joshi, Oluwasanmi Koyejo, Warut Vijitbenjaronk, Been Kim, Joydeep Ghosh
TL;DR
Automated decisions can leave individuals with undesirable outcomes, creating a need for actionable recourse. The paper models data manifolds to find minimal, realistic changes and applies the approach to supervised classification and causal decision systems. Its causal evaluation is constrained by limited counterfactual information and the suitability of available datasets for recourse.
Problem
Individuals may receive undesirable outcomes from automated decision systems, while recourse has received comparatively little attention in fairness research.
Method
The paper uses a gradient-based algorithm to traverse a generative model’s latent manifold and find minimal changes that improve outcomes, including in causal systems.
Results
The proposed algorithm provides recourse for supervised classification and causal decision-making systems by finding small, manifold-constrained changes toward improved outcomes.
Takeaways & Limitations
Manifold-based recourse is presented as a way to make suggested changes more effective and achievable while supporting broader classes of decision-making systems.
Takeaways & Limitations
Causal recourse evaluation is limited because counterfactual information is scarce and available datasets may contain features that are not amenable to recourse.
Abstract
from arXiv · showhide
Machine learning based decision making systems are increasingly affecting humans. An individual can suffer an undesirable outcome under such decision making systems (e.g. denied credit) irrespective of whether the decision is fair or accurate. Individual recourse pertains to the problem of providing an actionable set of changes a person can undertake in order to improve their outcome. We propose a recourse algorithm that models the underlying data distribution or manifold. We then provide a mechanism to generate the smallest set of changes that will improve an individual's outcome. This mechanism can be easily used to provide recourse for any differentiable machine learning based decision making system. Further, the resulting algorithm is shown to be applicable to both supervised classification and causal decision making systems. Our work attempts to fill gaps in existing fairness literature that have primarily focused on discovering and/or algorithmically enforcing fairness constraints on decision making systems. This work also provides an alternative approach to generating counterfactual explanations.
1 Introduction
The paper frames individual recourse as actionable changes for unfavorable automated decisions and proposes realistic, minimal changes constrained by the data manifold.
- Recourse gives individuals concrete attribute changes that can improve unfavorable outcomes from automated decision systems.The paper presents recourse as a safeguard for individuals affected by decisions such as credit denial.
- The proposed framework models the population’s data manifold and searches its latent representation for high-probability paths near the individual’s original attributes.This is intended to produce changes that are both outcome-improving and realistic.
- The method seeks minimal manifold-constrained changes rather than changes that are merely small but unlikely under the client population’s circumstances.The paper illustrates that increasing monthly payments may be more realistic than changing income substantially.
- Compared with prior actionable-recourse work, the approach addresses broader machine-learning systems while mitigating effects on variables omitted from a proposed change set.The cited prior work focuses on linear models and flipsets, whereas manifold characterization can mitigate such issues.
2 Related Work
Related work distinguishes actionable recourse from counterfactual explanations and motivates manifold-based recourse as realistic without restricting the decision system to linear models.
- Prior recourse work optimizes user-specified costs and prevents changes to immutable variables using linear integer programming.Its scope is linear algorithmic decision-making systems.
- The proposed approach models the entire data distribution and follows shortest manifold paths, so recourses need not be observed training samples or linear-model solutions.It is also presented as applicable beyond supervised classification.
- Counterfactual explanations focus on why a model produced an outcome, whereas recourse focuses on actionable changes to improve that outcome.The paper notes that counterfactual methods may fail to follow realistic sample paths and can produce profiles outside the ambient data domain.
3 A Framework for Individual Recourse
The framework generates individual recourse by finding minimal changes that remain likely under the data distribution, using a generative model and optimization. It extends this approach from differentiable supervised classifiers to causal decision-making systems, including interventions and hidden confounders.
- Supervised classification: Recourse is defined as changing attributes of an individual with an undesirable outcome to improve that outcome.The changes are represented as nonzero attribute differences from the original instance.
- Supervised classification: The desired recourse minimizes distance from the original sample while requiring the new sample to remain sufficiently likely under p(x).The likelihood threshold is parameterized by γ > 0, and the resulting recourse is the set of changed attributes.
- Supervised classification: A generative model maps latent variables to samples from the data distribution, enabling recourse to be searched along the learned data manifold.The paper describes VAEs and GANs as examples and uses an encoder when available to obtain a latent representation.
- Supervised classification: The optimization balances target-label improvement against closeness to the original sample, with λ controlling this trade-off.REVISE starts from the original sample’s latent encoding and takes gradient steps until the classifier crosses into the desired class.
- Causal decision making: Unobserved confounders can make treatment effects difficult or impossible to identify, while model misspecification, approximation, and selection bias can limit causal-model reliability.The paper motivates causal recourse despite these limitations because undesirable outcomes can remain for individuals even under more accurate causal systems.
- Causal decision making: In causal systems, REVISE traverses hidden-confounder space to find minimal attribute changes that improve outcomes, optionally under treatment interventions do(t).The causal formulation re-estimates hidden confounders while keeping the resulting attributes close to the original point.
4 Experiments
The experiments evaluate REVISE for supervised classification and causal decision-making, including immutable variables and confounding. They also use recourse to diagnose how biased attributes can influence black-box decisions.
- Recourse for Classification Systems: REVISE is evaluated on UCI defaultCredit using an HI-VAE generative model to handle heterogeneous attributes.The evaluation targets recourse for clients expected to default under a classification-based decision system.
- Recourse for Classification Systems: The classification experiment compares sample recourses from REVISE with linear softmax, nonlinear MLP, and Ustun et al. methods.Table 1 presents original defaulting features and recourses intended to prevent default; unchanged attributes are marked with '-'.
- Recourse in Causal Models: Causal-model experiments evaluate recourse sparsity, latent-space distance, and input-space distance under randomized and observational treatment assignments.The comparison uses models trained with and without biases reflecting differences in treatment-effect reliability.
- Handling Immutable Variables: Immutable variables are held fixed by conditioning observed attributes on the immutable variables while allowing the remaining attributes to change.The method modifies a causal latent-variable model, CEVAE, to handle immutable variables and recourse.
- Handling Immutable Variables: On TWINS data, confounding significantly changes the nature of recourse, whose main recommendations involve lowering specific risk factors.The study notes that these recourses are illustrative rather than practical because of the dataset itself.
- Recourse under Attribute Confounding: Recourse exposes attribute-confounding effects: biased hair-color classifier f2 changes gender-specific facial attributes more often than unbiased classifier f1 when crossing its decision boundary.The experiment uses hair-color transitions to diagnose whether gender information is entangled with the classifier's predictions.
5 Conclusions
The paper presents algorithmic recourse as minimal, realistic changes derived from data distributions or manifolds, applicable across classification and causal decision-making systems.
- The proposed algorithm samples latent space through a generative model to find minimal changes that improve an individual’s outcome.It supports both classification and causal decision-making systems.
- Restricting recourse to data-manifold paths avoids recommending changes that are small numerically but unrealistic for the client population.The approach follows minimal sample paths along the learned manifold.
- The method can diagnose whether a decision-making algorithm systematically confounds specific attributes.This gives the recourse mechanism an additional diagnostic role beyond generating actionable changes.
6 Appendix
The appendix documents the experimental pipeline, datasets, model settings, and supplementary tables used to evaluate recourse across classifiers, causal data, and image-based diagnostic examples.
- Experiments train generative models, train black-box candidate decision models, and then evaluate the proposed recourse algorithm.This three-step procedure is stated as the common evaluation protocol.
- Table 4 evaluates recourse for a linear softmax classifier with ℓ1 regularization 1e−5 and 79.37% accuracy.The original outcome is credit default, while the recoursed outcome is prevention of default in the next month.
- The UCI defaultCredit appendix compares the proposed method with Ustun et al. by number of changed attributes and qualitative recourse.The sparsest Ustun et al. flipset is used as the baseline, while cost functions differ.
- Additional experiments document HI-VAE processing, classifier settings, TWINS immutable variables, conditional causal-model modifications, and CelebA hair-color diagnostics.The appendix includes supplementary recourse tables and a table measuring gender flips when hair color changes.