PaperlayerView exact revision on arXiv

Source-linked AI summary

CARLA: A Python Library to Benchmark Algorithmic Recourse and Counterfactual Explanation Algorithms

Martin Pawelczyk, Sascha Bielawski, Johannes van den Heuvel, Tobias Richter, Gjergji Kasneci

arXiv:2108.00783v1cs.LGcs.AI

TL;DR

Counterfactual explanations propose actionable changes toward favorable outcomes, but the growing range of methods calls for meaningful comparisons. CARLA provides a benchmarking library with methods and evaluation measures. Across the benchmark, methods' strengths vary by measure rather than one method dominating all comparisons.

  • Problem

    As counterfactual methods proliferate, meaningful comparison is needed for actionable recommendations in consequential settings.

    Source: §1 · pp. 1–2

  • Method

    CARLA benchmarks counterfactual methods with integrated evaluation measures across three real-world data sets and supports evaluations using different models.

    Source: §1 · p. 2, §4.2 · p. 7, §5 · p. 8

  • Results

    Across independence-based methods, no single method outperformed all competitors; dependence-based results were mixed, with strengths varying by measure.

    Source: §5 · p. 9

  • Takeaways & Limitations

    CARLA has the potential to help researchers and practitioners derive realistic, use-case-driven recourse strategies and assess their quality through comparative evaluations.

    Source: §6 · p. 9

    Causality-based methods were not included because they usually require a true causal graph, which is a practical limiting factor.

    Source: §2.2 · p. 3

Abstract

from arXiv · show

Counterfactual explanations provide means for prescriptive model explanations by suggesting actionable feature changes (e.g., increase income) that allow individuals to achieve favorable outcomes in the future (e.g., insurance approval). Choosing an appropriate method is a crucial aspect for meaningful counterfactual explanations. As documented in recent reviews, there exists a quickly growing literature with available methods. Yet, in the absence of widely available opensource implementations, the decision in favor of certain models is primarily based on what is readily available. Going forward - to guarantee meaningful comparisons across explanation methods - we present CARLA (Counterfactual And Recourse LibrAry), a python library for benchmarking counterfactual explanation methods across both different data sets and different machine learning models. In summary, our work provides the following contributions: (i) an extensive benchmark of 11 popular counterfactual explanation methods, (ii) a benchmarking framework for research on future counterfactual explanation methods, and (iii) a standardized set of integrated evaluation measures and data sets for transparent and extensive comparisons of these methods. We have open-sourced CARLA and our experimental results on Github, making them available as competitive baselines. We welcome contributions from other research groups and practitioners.

1 Introduction

Counterfactual explanations offer actionable changes toward favorable model outcomes, while algorithmic recourse adds feasibility constraints. CARLA addresses the need for transparent comparison by providing a shared benchmarking library for researchers and practitioners.

1 Introduction

  • Machine-learning predictions in sensitive domains can affect personal lives, including through loan rejection or denial of parole.The paper names disease diagnostics, criminal justice, and credit-risk scoring as examples of such domains.Source: §1 · p. 1
  • Counterfactual explanations suggest actionable feature changes intended to help individuals obtain favorable future outcomes.An example is increasing income to obtain insurance approval.Source: §1 · p. 2
  • Algorithmic recourse frames such suggestions as minimal changes subject to feasibility constraints, such as keeping an individual's sex unchanged.The paper situates recourse in a rapidly growing literature on available methods.Source: §1 · p. 2
  • Because the method shapes the counterfactual explanation an individual receives, transparent benchmarking helps researchers compare methods and practitioners choose a suitable recourse mechanism.The paper presents standardized comparison and quality assurance as essential prerequisites.Source: §1 · p. 2
  • CARLA is a Python library offering competitive baselines and standardized comparisons of counterfactual methods across integrated data sets.It also supports custom predictive models, evaluation measures, and integration of new methods.Source: §1 · p. 2

2 Related Work

The related work distinguishes feature-highlighting explanations from counterfactual approaches, which recommend interventions. Counterfactual methods are grouped by their assumptions about feature independence, dependence, or causal structure.

2 Related Work

  • Explainability research follows distinct paradigms for explaining complex machine-learning models.The paper frames the related work around these different streams.Source: §2 · p. 2

2.1 Feature Highlighting Explanations

  • Feature-highlighting methods explain individual predictions by showing how inputs are used, with model-agnostic approaches using interpretable surrogates for black-box models.Other approaches include gradient-based methods for white-box neural networks and rule-based or probabilistic methods for tree ensembles.Source: §2.1 · p. 3

2.2 Counterfactual Explanations

  • Counterfactual explanations recommend constructive input interventions, and the paper groups them as independence-based, dependence-based, or causality-based.These approaches emphasize both feature importance and recommendations.Source: §2.2 · p. 3
  • Independence-based methods assume independent input features, but their omission of correlations can make estimated intervention costs too optimistic when features depend on one another.The paper gives income and tenure as an example of dependent features.Source: §2.2 · p. 3
  • Causality-based methods use Pearl's causal framework and usually require structural equations or a causal graph; the paper excludes them because access to the true graph is a practical limitation.The authors note that minimum-cost guarantees can be vacuous when a causal model is misspecified.Source: §2.2 · p. 3
  • Dependence-based methods use generative models to represent input dependencies in a lower-dimensional latent intervention space, commonly using variational autoencoders.The paper also describes graph-based shortest paths and integer programming as alternatives in this line of work.Source: §2.2 · p. 3

3 Preliminaries

The preliminaries frame algorithmic recourse through two observational formulations: one assumes independent inputs, while the other uses a generative model to capture dependencies.

3 Preliminaries

  • The section reviews algorithmic recourse as an observational, non-causal problem and distinguishes two approaches.Source: §3 · p. 3

3.1 Counterfactual Explanations for Independent Inputs

  • Under pairwise input independence, recourse minimizes input-space change subject to admissible perturbations that yield a classifier score above the threshold.Admissible changes can exclude sensitive attributes such as age or sex.Source: §3.1 · p. 4

3.2 Recourse for Correlated Inputs

  • For correlated inputs, recourse perturbs a latent code and uses a generative model to produce an admissible counterfactual satisfying the desired classifier outcome.The latent-space admissibility set can constrain counterfactual codes to remain within the range of the factual code.Source: §3.2 · p. 4

4 Benchmarking Process

The benchmarking process surveys counterfactual methods and evaluates them with measures spanning cost, feasibility, data support, redundancy, success, and runtime. It covers methods with different model assumptions and algorithms, alongside the measures used to compare their recourse suggestions.

4.1 Counterfactual Explanation Methods

  • AR finds minimum-cost actions for linear classifiers through discrete optimization, with user constraints or immutable features available to restrict actions.It uses linear-model coefficients and integer programming solvers such as CPLEX or CBC.Source: §4.1 · p. 4
  • The method overview organizes approaches by assumptions, model type, algorithm, immutable- and categorical-feature support, and other characteristics.The benchmark evaluates methods under this shared comparison framework.Source: §4 · p. 4, Table 1 · p. 6
  • AR–LIME applies AR to nonlinear models using LIME’s local linear approximation, while CEM uses an elastic-net objective and supports a VAE-type regularizer.CEM has no immutable-feature handling.Source: §4.1 · p. 6
  • DICE trades off proximity and diversity; Growing Spheres searches around the factual input, while CLUE and REVISE use VAE-based generative recourse.FACE instead finds counterfactuals from high-density regions with a graph shortest-path algorithm.Source: §4.1 · p. 6

4.2 Evaluation Measures for Counterfactual Explanation Methods

  • The paper introduces six baseline evaluation measures, including normalized ℓ0/ℓ1 costs, user-constraint violations, yNN neighborhood support from positively classified instances, and redundancy; it does not identify all six here.ℓ0 restricts the number of feature changes, while ℓ1 restricts the average change. The authors use k := 5 for yNN.Source: §4.2 · pp. 7–8
  • Redundancy counts changes that can be reverted without changing the counterfactual label; success rate tracks correctly generated counterfactuals, and average time measures generation speed.The redundancy procedure flips feature values back one at a time; lower redundancy indicates fewer unnecessary changes.Source: §4.2 · p. 8

5 Experimental Evaluation

Across the benchmark, no single counterfactual method dominates, and strengths differ by method family and evaluation measure. The experiments use three real-world datasets and compare logistic regression with a multilayer perceptron.

5 Experimental Evaluation

  • The benchmark uses three real-world datasets, logistic regression and a multilayer perceptron; Adult categorical features are binarized, while specified features are immutable.Adult and Give Me Some Credit are described in the supplied passages; COMPAS results are relegated to Appendix B.Source: §5 · p. 8, Table 3 · p. 8
  • The redundancy check flips feature changes back one at a time and does not consider every possible subset of changes.Thus, the reported measure is based on a sequential check rather than exhaustive subset testing.Source: §5 · p. 8
  • No single method dominates: independence-based methods lead on different measures, while dependence-based methods also trade off cost, neighborhood support, redundancy, and runtime.The findings vary across datasets and classifiers.Source: §5 · p. 9
  • AR, AR–LIME, and DICE perform strongest on ℓ0 and have the lowest redundancy, CEM performs strongest on ℓ1, and Growing Spheres is fastest among independence-based methods.The authors expect more sophisticated sampling could improve Growing Spheres’ cost performance.Source: §5 · p. 9
  • CLUE and REVISE lead dependence-based methods on ℓ1, FACE-EPS leads on yNN, and generative approaches perform best on time.CLUE and REVISE perform worst on ℓ0, likely due to imprecise decoder reconstruction; their average redundancy reaches up to twice FACE’s.Source: §5 · p. 9

6 Conclusion and Broader Impact of CARLA

CARLA offers standardized comparisons of counterfactual explanation methods, while its current implementations remain tied to the original research groups’ framework and data choices. The authors identify integration hurdles and anticipate community contributions to the library.

6 Conclusion and Broader Impact of CARLA

  • Existing recourse implementations are based on their original research groups’ code and can be restricted to particular ML frameworks and data sets.The authors plan to decouple methods from these constraints.Source: §6 · p. 9
  • Many method repositories provide installation notes and result-reproduction scripts but omit data-set or black-box-model interface information, complicating integration.CARLA provides a step-by-step guide for integrating new methods and data sets, and the authors welcome contributions.Source: §6 · p. 9
  • CARLA is presented as a platform for transparent comparisons across data sets, with potential to help develop use-case-driven recourse strategies and assess their quality.The authors describe the library’s potential contribution to explainability research.Source: §6 · p. 9

Checklist

The checklist points readers to implementation and reproducibility resources, notes a framework-specific limitation, and introduces CARLA’s software interface.

Checklist

  • Some counterfactual methods are restricted to particular ML frameworks, such as TensorFlow, and are not applicable to PyTorch models.Source: Checklist · p. 14
  • Implementation details and instructions for reproducing results are referenced in the appendices and the project’s GitHub repository.The checklist also directs readers to appendices for training details.Source: Checklist · pp. 14–15
  • The checklist states that the work used publicly available data without personally identifying information and did not use crowdsourcing or human subjects.Source: Checklist · p. 15

A CARLA’s Software Interface

A.1 CARLA’s High Level Software Architecture

CARLA provides shared interfaces for applying and comparing recourse methods across data sets and black-box models. It supports custom methods and user-defined inputs alongside its catalogs, with built-in evaluation measures for benchmarking.

A.1 CARLA’s High Level Software Architecture

  • CARLA’s Data, MLModel, and RecourseMethod components provide common interfaces for data access, model details, and counterfactual generation.Figure 4 depicts catalog and custom implementations, along with dependencies among components.Source: §A.1 · p. 16, Figure 4 · p. 16

A.2 CARLA for Research Groups

  • Research groups can implement custom counterfactual methods through the RecourseMethod wrapper and compare them with existing methods.The library allows researchers to use catalog data sets and models or their own.Source: §A.2 · pp. 16–17, Figure 5 · p. 17

A.3 CARLA as a Recourse Library

  • Users can generate counterfactuals with catalog data and models or wrap their own data sets and black-box models through CARLA’s interfaces.The examples describe loading catalog assets and defining custom data and model classes.Source: §A.3 · pp. 17–18, Figure 6 · p. 17, Figure 7 · p. 18

A.4 Benchmarking Recourse Methods

  • CARLA’s focus includes benchmarking recourse methods with evaluation measures that support qualitative judgments about usability and applicability.The Benchmarking class makes the measures available for wrapped methods.Source: §A.4 · p. 18

B Additional Experimental Results

The appendix presents additional COMPAS experiments and reports that they underline trends highlighted earlier.

B Additional Experimental Results

C ML Classifiers

CARLA fits black-box classifiers while accounting for the different libraries and binary-feature encodings required by explanation methods. It evaluates counterfactual methods using multilayer perceptron and linear-model classifier categories.

C ML Classifiers

  • CARLA supports classifiers fitted with different machine-learning libraries, while explanation methods may require distinct binary-feature encodings.For example, DICE requires one-hot binary inputs, whereas FACE uses a single column; when needed, the authors fitted two models with the same hyperparameters and generated explanations for the same samples.Source: C ML Classifiers · pp. 18–19
  • Classifier types were designed to have the same structure and training parameters across supported machine-learning libraries and encoding variations.This includes matching hidden-layer and neuron counts, learning rate, and epochs.Source: C ML Classifiers · p. 19
  • The multilayer perceptron has three hidden layers of 18, 9, and 3 neurons, uses ReLU and binary cross entropy, and is optimized with RMSProp at a 0.002 learning rate.Training used 25 epochs for COMPAS and 10 for Adult and GMC; further epochs brought only marginal performance increases.Source: C ML Classifiers · p. 19
  • Linear models form the second black-box classifier category and use RMSProp with binary cross entropy, with dataset-specific epoch and batch-size settings.Adult uses 100 epochs and batch size 2048; COMPAS uses 25 epochs and batch size 128; GMC uses 10 epochs.Source: C ML Classifiers · p. 19

D COMPAS Data Set Description

The COMPAS dataset contains records for more than 10,000 Florida criminal defendants and is used to score likelihood of reoffending. The classification task predicts high recidivism risk, with age, sex, and race designated immutable.

D COMPAS Data Set Description

  • COMPAS contains data for more than 10,000 criminal defendants in Florida and is used by the jurisdiction to score likelihood of reoffending.The classification target is high risk of recidivism; race is represented as white or non-white, and age, sex, and race are immutable features.Source: D COMPAS Data Set Description · p. 21
  • The task classifies defendants as high risk of recidivism when score_text is high.The feature race is converted to white and non-white to keep the categorical input binary.Source: D COMPAS Data Set Description · p. 21
  • Age, sex, and race are treated as immutable features in the COMPAS dataset.The raw data was reduced by dropping features such as names, IDs, case numbers, and date-time fields.Source: D COMPAS Data Set Description · p. 21

E Hyperparameter Search for the Counterfactual Explanation and Recourse Methods

The authors generated counterfactual explanations for negatively predicted instances and tuned method-specific settings. For AR, the search addresses actions that may not flip the classifier; other methods use grid searches or stated defaults and parameter choices.

E Hyperparameter Search for the Counterfactual Explanation and Recourse Methods

  • Counterfactual explanations were generated for factual instances with negative class predictions.Source: E Hyperparameter Search for the… · p. 21
  • AR searches 150 actions per instance for low-cost counterfactuals because the lowest-cost action often failed to flip the black-box prediction.AR-LIME uses LIME and samples around the instance to ensure its coefficients are local.Source: E Hyperparameter Search for the… · p. 21
  • For CEM, grid search selected ℓ1 weight 0.9 and ℓ2 weight 0.1; CEM-VAE used ℓ2 weight 0.1 and VAE weight 0.9.The CEM settings yielded the strongest performance in the search.Source: E Hyperparameter Search for the… · p. 21
  • CLUE uses default hyperparameters from [3], while DICE generates one counterfactual per instance and grid-searches its proximity and diversity weights.For CLUE, hyperparameter search did not improve distances while maintaining the same success rate.Source: E Hyperparameter Search for the… · p. 21
  • FACE grid search selected kFACE = 50 for balancing success rate and costs, and an epsilon-graph radius of 0.25 to balance yNN and cost.Source: E Hyperparameter Search for the… · p. 21
  • GS uses a 0.02 sphere-growth step; REVISE uses η = 0.1 and λ = 0.5 for about 1500 iterations, while Wachter uses binary cross entropy and ℓ1 distance.Lower GS step sizes had similar results but higher computational time, and higher values gave worse results; Wachter uses learning rate 0.01 and initial λ 0.01.Source: E Hyperparameter Search for the… · p. 22
Loading 2108.00783v1…