PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning Model-Agnostic Counterfactual Explanations for Tabular Data

Martin Pawelczyk, Johannes Haug, Klaus Broelemann, Gjergji Kasneci

arXiv:1910.09398v2cs.LGstat.ML

TL;DR

Counterfactual methods should suggest changes that are not local outliers, remain connected to correctly classified observations, and are attainable for individuals. The paper adds difficulty as a quality criterion and introduces C-CHVAE, which searches latent space to generate high-density counterfactuals for heterogeneous tabular data. Experiments report stronger faithfulness than competing methods, with greater difficulty on the evaluated real-world datasets.

  • Problem

    Existing sparse counterfactual methods do not ensure proximity, connectedness, or sufficiently low difficulty, which are needed for attainable suggestions.

  • Method

    C-CHVAE models heterogeneous tabular data with a conditional autoencoder and searches perturbed latent representations without requiring input-space distance functions or classifier access.

  • Results

    C-CHVAE outperforms competing methods on counterfactual faithfulness across the reported synthetic and real-world evaluations, while its faithful real-world counterfactuals have greater total and maximum percentile shifts.

  • Takeaways & Limitations

    Attainable counterfactuals can be generated for heterogeneous tabular data by searching high-density latent neighborhoods rather than relying on ad-hoc input-space distances.

  • Takeaways & Limitations

    The framework assumes independence between latent mixture components and protected inputs in the CHVAE formulation.

Abstract

from arXiv · show

Counterfactual explanations can be obtained by identifying the smallest change made to a feature vector to qualitatively influence a prediction; for example, from 'loan rejected' to 'awarded' or from 'high risk of cardiovascular disease' to 'low risk'. Previous approaches often emphasized that counterfactuals should be easily interpretable to humans, motivating sparse solutions with few changes to the feature vectors. However, these approaches would not ensure that the produced counterfactuals be proximate (i.e., not local outliers) and connected to regions with substantial data density (i.e., close to correctly classified observations), two requirements known as counterfactual faithfulness. These requirements are fundamental when making suggestions to individuals that are indeed attainable. Our contribution is twofold. On one hand, we suggest to complement the catalogue of counterfactual quality measures [1] using a criterion to quantify the degree of difficulty for a certain counterfactual suggestion. On the other hand, drawing ideas from the manifold learning literature, we develop a framework that generates attainable counterfactuals. We suggest the counterfactual conditional heterogeneous variational autoencoder (C-CHVAE) to identify attainable counterfactuals that lie within regions of high data density.

1 INTRODUCTORY REMARKS

Counterfactual explanations aim to help people change automated decisions, but attainable suggestions must also remain close to the data distribution and correctly classified observations. The paper proposes C-CHVAE, a heterogeneous, classifier-agnostic framework that searches latent space for faithful counterfactuals without ad-hoc input-space distances.

  • 1 INTRODUCTORY REMARKS: Counterfactual explanations describe small input changes intended to influence a classifier’s decision in a person’s favor.They are motivated by requests for meaningful information about automated decisions and by the desire to empower affected individuals.
  • 1 INTRODUCTORY REMARKS: Attainability combines proximity, connectedness, and low total CDF percentile shifts to characterize counterfactual suggestions that are realistic and not too difficult.Proximity excludes local outliers, connectedness measures closeness to correctly classified observations, and percentile shifts quantify difficulty.
  • 1 INTRODUCTORY REMARKS: For heterogeneous tabular data, C-CHVAE avoids relying on expert-defined input-space distance or cost functions by measuring similarity through latent representations.The framework supports concurrent real-valued, positive real-valued, count, categorical, and ordinal inputs through appropriate likelihood models.
  • 1 INTRODUCTORY REMARKS: C-CHVAE searches for counterfactuals by perturbing encoded data in a low-dimensional latent neighborhood, decoding the perturbation, and checking it with a pretrained classifier.The decoder reconstructs potential counterfactuals that are likely to occur, while protected inputs condition the neighborhood search.
  • 1 INTRODUCTORY REMARKS: The framework generates counterfactuals in high-density regions, addressing faithfulness requirements while remaining compatible with heterogeneous tabular data and classifier-agnostic use.Its stated merits include support for interval constraints and no requirement for ad-hoc predefined input distances.

2 RELATED LITERATURE

The related literature includes counterfactual methods that differ in their training requirements, classifier assumptions, similarity functions, and treatment of data-manifold structure. C-CHVAE learns similarity in latent space to generate counterfactuals on the data manifold.

  • The paper distinguishes its approach from related work by learning a separate similarity model that generates counterfactuals lying on the data manifold.
  • Some methods address heterogeneous-feature constraints by specifying feature supports, while C-CHVAE can impose such constraints through feature-specific likelihood functions.
  • C-CHVAE learns similarity from data in latent space rather than relying on task-specific, predefined similarity functions between inputs and counterfactuals.
  • Table 1 contrasts hypothetical recommendations for one denied-credit individual, including whether changes appear reasonable, connected, non-outlying, and attainable.
  • Existing approaches include methods requiring training, methods constrained by classifier families, and methods using predefined input-to-counterfactual distances or costs.
  • Counterfactual explanations are presented as recommendations that alter predictions through impactful changes individuals may exert in the real world.

3 BACKGROUND

The background develops variational autoencoders and conditional VAEs as models that learn latent representations while bounding data or conditional log likelihoods. Different decoder likelihoods accommodate different input types.

  • A simple VAE optimizes the evidence lower bound, which bounds the data log likelihood from below.
  • The encoder and decoder distributions can be chosen according to input type, including Gaussian models for continuous data and Bernoulli models for binary data.
  • Conditioning on immutable or other inputs xp yields a conditional objective that bounds the conditional log likelihood log p(xf |xp).

4 C-CHVAE

C-CHVAE embeds counterfactual search in a conditional heterogeneous variational autoencoder that models free inputs given immutable inputs and searches near latent representations. Its factorized architecture supports heterogeneous data and feature constraints.

  • The C-CHVAE: The objective combines conditional likelihood modelling, a lower-dimensional latent representation, latent regularization, and a classifier-based counterfactual condition.
  • Factorized encoder: The encoder and decoder preserve heterogeneous feature structure while the decoder reconstructs free inputs from perturbed latent representations.
  • Factorized decoder: C-CHVAE uses a hierarchical factorized decoder with a conditional uniform Gaussian mixture prior to generate counterfactuals conditional on immutable attributes.
  • Parameter sharing and likelihood models: The factorized decoder assigns likelihood models per input, jointly representing real-valued, positive real-valued, count, categorical, and ordinal features.
  • Parameter sharing and likelihood models: Appropriate likelihood choices can encode interval constraints, including truncated normal or Beta distributions for interval data.
  • The C-CHVAE: The framework takes a pretrained classifier and CHVAE encoder and decoder, then searches latent perturbations until a counterfactual satisfies the empowerment condition.

5 EVALUATING ATTAINABILITY OF COUNTERFACTUALS

The paper evaluates counterfactual attainability through faithfulness and difficulty. Proximity measures outlier status, connectedness measures links to correctly classified data, and percentile-shift costs measure the effort implied by changes.

  • Faithfulness comprises proximity to non-counterfactual correctly classified observations and connectedness to correctly classified observations from the same class.
  • The proposed attainability view combines non-outlier proximity, similarity to correctly classified observations, and low total CDF percentile shifts.
  • Proximity: Proximity evaluates whether a counterfactual is a local outlier relative to nearby correctly classified observations.
  • Connectedness: Connectedness receives a binary score according to whether an epsilon-chain links the counterfactual to a correctly classified observation from the target class.
  • Individual costs of counterfactuals: Individual difficulty is measured using cumulative-density-function percentiles of free features, so a percentile cost reflects the shift needed to obtain the suggested result.
  • Individual costs of counterfactuals: The total percentile shift provides a baseline attainability measure, while the maximum percentile shift captures the greatest difficulty across mutable features.

6 EXPERIMENTS

Synthetic and real-world experiments show that C-CHVAE produces more faithful counterfactuals than comparison methods, while often requiring larger feature shifts. Its counterfactuals occupy high-density regions and improve faithfulness across GMSC and HELOC data.

  • Synthetic experiments: C-CHVAE generates counterfactuals in high-density regions, unlike AR and GS, whose sparse suggestions can lie in unattainable areas.In the synthetic example, AR and GS change only the second feature axis and suggest counterfactuals in a region where the data-generating process places no observations.
  • Synthetic experiments: The counterfactual latent density concentrates on two of three modes that generate data close to the decision boundary.This corresponds to finding the closest latent code among the relevant modes of the latent distribution.
  • Real-world data sets: On GMSC, C-CHVAE outperforms all other methods on faithfulness, with local-outlier-score differences reaching 20 percentage points.GS and AR have similar faithfulness, while HCLS performs worst; C-CHVAE also gains more on connectedness as ε increases.
  • Real-world data sets: Higher faithfulness comes with greater difficulty: C-CHVAE counterfactuals require higher total and maximum shifts on GMSC and higher maximum shifts on HELOC.The reported trade-off appears in the total-shift versus maximum-shift comparisons for both datasets.
  • Real-world data sets: On HELOC, C-CHVAE outperforms all other methods on both faithfulness measures and across all parameter choices.The comparison reports HCLS as weaker, potentially because its required feature-change directions can restrict suggestions and make them less typical.

7 CONCLUSION AND FUTURE WORK

The framework searches for counterfactuals in latent space using a pretrained classifier and CHVAE encoder-decoder, sampling increasingly distant regions until it finds a label-changing example. It supports uniform sphere sampling, multiple counterfactuals, and heterogeneous likelihood models.

  • Counterfactual search: The algorithm uses a pretrained classifier with the trained CHVAE encoder and decoder to search for counterfactuals in latent space.It seeks the smallest latent representation change that alters the predicted label, rather than the smallest input-space change.
  • Counterfactual search: It samples perturbed latent representations from l_p-spheres around the encoded observation until finding a counterfactual.The search radius expands when needed, and the procedure can return no counterfactual consistent with the data distribution.
  • Counterfactual search: Uniform sphere sampling is implemented by drawing unit-sphere observations and rescaling them with radii sampled uniformly between r1 and r2.The resulting observations are uniformly distributed over the defined spherical search region.
  • Counterfactual search: The procedure can generate multiple counterfactuals, allowing users to choose from a menu of alternatives that best fits their preferences.These collections are also called flip sets.
  • Heterogeneous data: The framework enumerates likelihood models for real-valued, positive real-valued, count, ordinal, and categorical features.The listed models include Gaussian, log-normal, Poisson, ordinal, and multinomial-logit formulations.

C SYNTHETIC EXAMPLE

The synthetic example uses discretized make-moons data with a categorical second feature and fragmented density. C-CHVAE correctly produces counterfactuals across the tested categories, whereas the comparison methods cover different category ranges.

  • Synthetic example: The synthetic dataset contains two half-circles, with x2 treated as categorical across 19 categories and labels determined by whether x1 exceeds zero.The upper and lower halves contain equal numbers of observations.
  • Synthetic example: Finding attainable counterfactuals is nontrivial because the data density is highly fragmented despite the simple class assignment.The figures show the generating process, class distribution, histograms, and test observations.
  • Synthetic example: AR generates counterfactuals in the 1st and 9th categories, while GS generates counterfactuals across all categories.The comparison uses 25 counterfactuals generated by AR, HCLS, and C-CHVAE.
  • Synthetic example: C-CHVAE correctly produces counterfactuals in the 1st, 9th, and 19th categories.This demonstrates coverage of the categories explicitly tested in the example.

D.1 Real world example: “Give Me Some Credit”

The “Give Me Some Credit” experiment uses an 80/20 train-test split and compares AR, GS, HCLS, and C-CHVAE under specified model and feature-change settings. C-CHVAE is configured with heterogeneous likelihood models and an l2-regularized logistic-regression classifier.

  • Experimental setup: The experiment uses 80 percent of the data for training and the remainder as a holdout test set, with the classifier given access to all features.The feature states are reported in Table 3.
  • Baselines: AR requires an action set, mutable and immutable features, feature directions, and a shifted decision boundary.DebtRatio is constrained to move downward, and the adjusted threshold is pAR = 0.50.
  • Baselines: GS uses a 0.1 step size for generating observations around the input and an l2-regularized logistic-regression classifier.Its search is based on a version of the YPHL algorithm.
  • Baselines: HCLS is configured with a budget of 10, unit feature-change costs, and an SVM with a Gaussian kernel.The implementation uses the authors’ baseline MATLAB implementation.
  • C-CHVAE: C-CHVAE uses latent dimensions 5 for s and 6 for z, trains for 50 epochs, and models count features with Poisson likelihoods and positive real-valued features with log-normal distributions.Its classifier is l2-regularized logistic regression.

D.2 Real world example: HELOC

The HELOC experiment predicts two-year repayment from homeowners’ credit-report information. Its comparison methods and C-CHVAE use distinct feature-direction, indirect-feature, latent-dimension, and training settings.

  • Dataset and task: The HELOC dataset contains U.S. homeowners’ credit applications, with the task of predicting repayment within two years.Available features and assumed likelihood models are summarized in Table 4.
  • Baselines: AR and GS are used without specifying how features must move.This differs from the directional constraints specified for HCLS.
  • Baselines: HCLS uses a linear-kernel SVM and designates NumTrades90Ever/DerogPubRec as the indirect feature.Feature movement directions are specified in Table 4.
  • C-CHVAE: C-CHVAE uses latent dimensions 1 for s and 10 for z and trains for 60 epochs.The remaining configuration follows the preceding setup, with feature likelihood details given in Table 4.
Loading 1910.09398v2…