PaperlayerView exact revision on arXiv

Source-linked AI summary

AVCG: A Generalized Variational Framework for Counterfactual Generation under Hypothesis Distributions

Jamie Duell, Alejandro Jimenez Rodriguez, Mahault Albarracin

arXiv:2609.07917v1cs.LGcs.AI

TL;DR

Counterfactuals optimized for a single point-estimate predictor may fail after retraining or under predictive uncertainty. AVCG optimizes over arbitrary distributions of plausible hypotheses, including Bayesian and Rashomon cases. Across four datasets, it maintains cross-model validity and Rashomon validity near 1.000.

Abstract

from arXiv · show

Counterfactual explanations formalize "what-if" scenarios by identifying modifications to an input instance that obtain a desired alternative prediction. Traditionally, whether generated via instance-specific optimization or amortized single pass models, these approaches rely on a single, deterministic point-estimate predictor. However, this ignores predictive uncertainty and hypothesis variability, leading to brittle explanations that frequently become invalid if the underlying model is retrained or updated. To address this fragility, we propose the Amortized Variational Counterfactual Generator (AVCG), a generalized optimization framework that formulates counterfactual generation as optimization over an arbitrary distribution of plausible predictive hypotheses rather than a single deterministic predictor. This formulation naturally accommodates Bayesian posteriors, Rashomon-restricted hypothesis spaces, and other uncertainty representations within a unified optimization framework. Evaluation across multiple benchmark datasets demonstrates that the AVCG framework produces counterfactual explanations that remain highly valid under predictive uncertainty and model changes, while maintaining competitive plausibility and single-pass runtime performance.

Introduction

Counterfactual explanations optimized for a single fixed predictor can become invalid after retraining, data shifts, or under epistemic uncertainty. AVCG addresses the limits of specialized uncertainty-aware methods by optimizing over arbitrary distributions of plausible hypotheses.

Introduction

  • Single-model counterfactuals can become invalid after retraining, minor data shifts, or amid epistemic uncertainty.The vulnerability arises from optimizing explanations against a deterministic decision boundary.Source: Introduction · p. 1
  • Bayesian predictive models and empirically near-optimal hypotheses, such as the Rashomon set, are distinct sources of hypothesis distributions.The introduction identifies Bayesian Neural Networks and Monte Carlo Dropout as examples of Bayesian predictive models.Source: Introduction · p. 1
  • Existing uncertainty-aware approaches are tied to specific representations and often use costly post-hoc optimization, leaving no unified framework for arbitrary hypothesis distributions.Bayesian posterior optimization does not directly support robustness over constrained spaces such as the Rashomon set.Source: Introduction · p. 1
  • AVCG optimizes expected validity over arbitrary hypothesis distributions using a single amortized generator that produces counterfactuals in one forward pass.The objective is decoupled from the mechanism representing uncertainty.Source: Introduction · p. 1

1. We formulate counterfactual generation as optimiza-

The contributions present AVCG as a variational framework over arbitrary hypothesis distributions, with Bayesian and Rashomon instantiations and theoretical and empirical evaluation.

1. We formulate counterfactual generation as optimiza-

  • AVCG formulates counterfactual generation over arbitrary hypothesis distributions, independent of a specific uncertainty representation.The framework derives a generalized variational lower bound whose predictive objective subsumes Bayesian formulations as special cases.Source: §1 · p. 2
  • AVCG instantiates Bayesian uncertainty and Rashomon model multiplicity within the same optimization framework.These represent two distinct robustness paradigms.Source: §1 · p. 2
  • The paper provides theoretical and empirical evaluations of robustness, validity, and inference speed.Source: §1 · p. 2

Preliminaries

The preliminaries formalize a predictive space over model parameters and define hypothesis distributions as probability distributions over plausible predictors. They introduce predictive distributions by marginalizing over hypotheses and set up counterfactual and robust-counterfactual concepts.

Preliminaries

  • A generalized predictive space represents hypothesis variability for a parametric model fθ mapping inputs to probability outputs.PΦ(θ) denotes a generalized probability distribution over model parameters θ.Source: Preliminaries · p. 2
  • A hypothesis distribution PΦ(θ) is a distribution over model parameters representing a practitioner’s prior beliefs or constraints on plausible predictors.The hypothesis space consists of models parameterized by θ.Source: Preliminaries · p. 2
  • The generalized predictive distribution is defined by marginalizing a model’s predictive distribution over the hypothesis space.The text also states that this can equivalently be written as an expectation.Source: Preliminaries · p. 2
  • The preliminaries introduce counterfactual instances as generator outputs for factual inputs and then define robustness over hypothesis distributions.The robust-counterfactual definition is framed for a factual instance x and a generator g with output x′.Source: Preliminaries · p. 2

Methodology

AVCG formulates counterfactual generation as expected-risk optimization over hypothesis distributions, with Bayesian and Rashomon instantiations. Its objective combines latent-variable prediction with proximity regularization, while theoretical analysis examines stability and validity.

The Generalized AVCG Framework

  • AVCG optimizes counterfactual generation over arbitrary hypothesis distributions, instantiated with either a Bayesian posterior or a Rashomon-restricted distribution.The objective takes an expectation over PΦ(θ).Source: Methodology · p. 2
  • The generator maximizes target-label probability by marginalizing over latent variables and predictor parameters, using an encoder, generator, and target predictor.The variational posterior qψ(z | y′, x) supports the lower bound; qψ is the encoder, gϕ the generative model, and Pθ the target predictor.Source: The Generalized AVCG Framework · p. 2
  • A distance-based proximity constraint is added to keep generated interventions realistically actionable within the dataset-level objective.The objective is minimized to obtain the encoder and generator parameters.Source: Proximity Regularization · pp. 2–3

AVCG-B: The Bayesian Instantiation

  • AVCG-B uses the full Bayesian posterior and trains against dynamically sampled dropout masks to optimize validity in expectation across weight uncertainty.The paper describes this instantiation as suited to average-case generalization.Source: AVCG-B: The Bayesian Instantiation · p. 3

AVCG-R: The Rashomon-Restricted Instantiation

  • AVCG-R restricts hypotheses to models within ε of optimal validation loss, retaining qualifying frozen masks and optimizing uniformly over them.The Rashomon-restricted distribution targets robustness across an empirically constrained hypothesis space.Source: AVCG-R: The Rashomon-Restricted… · p. 3

Theoretical Analysis

  • The analysis bounds objective changes under distribution shifts and gives high-probability validity; an Adult Income example estimates 99% validity for a new Rashomon model.The example uses a Rashomon-set predictive variance of approximately 0.002, expected score 0.95, and β = 0.45.Source: Theoretical Analysis · p. 4, Rashomon Example · p. 4

Experimental Setup

Experiments use four tabular benchmarks, compare AVCG with Bayesian counterfactual explainers and QUCE, and assess validity, plausibility, and stability. The setup also defines cross-model and Rashomon-specific evaluation metrics.

Datasets and protocol

  • Experiments use Adult Income, Heart Disease, Wisconsin Breast Cancer, and Spambase, with means and standard deviations recorded across five seeds and 100 instances per dataset.Data are split into 80% training and 20% test, with 15% of training data held out for validation.Source: Datasets · p. 4, Experimental Setup · p. 4

Baselines

  • The comparison includes two Bayesian counterfactual explainers and QUCE evaluated over expected predictions rather than a single point estimate.The Bayesian baselines are Schut et al. and Batten et al.; QUCE is Duell et al.Source: Baselines · p. 4

Evaluation environments

  • Table 1 covers Adult Income and Breast Cancer, grouping methods by strict ε = 0.0 and relaxed ε = 0.8 empirical bounds.Reported values are mean ± standard deviation over five seeds.Source: Table 1 · p. 5

Metrics

  • Evaluation covers validity, plausibility, stability, diversity, and inference time, alongside two proposed robustness metrics.The listed measures include Val, IM1, Imp, NE, RIC, CMV, and RVR.Source: Metrics · p. 5
  • CMV tests counterfactual transferability across surrogate models trained on the same data distribution, while RVR measures the valid fraction across an empirically derived Rashomon set.RVR is defined for the Rashomon set selected through validation-set evaluation.Source: Metrics · p. 5

Results

AVCG-B and AVCG-R maintain cross-model validity and Rashomon validity ratios near 1.000 across four datasets, while several post-hoc methods lose validity under model multiplicity. The results also report diversity, fast single-pass inference, and stability across Rashomon set sizes.

Robustness Under Predictive Uncertainty

  • Both AVCG-B and AVCG-R maintain cross-model validity and Rashomon Validity Ratio near 1.000 across four datasets, unlike post-hoc methods that lose validity under model multiplicity.On Spambase, Schut et al. has CMV 0.773 and Batten et al. has CMV 0.755.Source: Robustness Under Predictive Uncertainty · p. 5, Table 2 · p. 6

Amortization: Diversity and Inference Efficiency

  • AVCG-R records Diversity of 1.078 on Breast Cancer and 0.747 on Spambase, while post-hoc baselines record Diversity 0.000 across datasets.The passage attributes post-hoc collapse to per-instance deterministic optimization and describes AVCG's latent sampling as producing diverse valid counterfactuals.Source: Amortization: Diversity and Inference… · pp. 5–6
  • AVCG generates counterfactuals in about 10−3 seconds with one forward pass, versus Schut et al.'s 0.04 seconds per instance.The passage says AVCG shifts the computational burden to training and reports an efficiency improvement of over an order of magnitude.Source: Amortization: Diversity and Inference… · pp. 5–6

ϵ-Sweep Results

  • AVCG-R remains stable across Rashomon set sizes as ϵ varies, and diversity commonly increases across datasets as ϵ decreases.The reported trends are based on means and deviations across five runs; the authors suggest stricter validity constraints may limit the counterfactual solution space.Source: ϵ Sweep · p. 6

Conclusion

AVCG is a generalized amortized framework for counterfactual generation over arbitrary hypothesis distributions, with Bayesian and Rashomon-restricted spaces as special cases. The conclusion reports superior performance across key metrics at decreased computational cost, while identifying the amortization gap as an open direction.

Conclusion

  • AVCG generalizes amortized counterfactual generation to arbitrary hypothesis distributions, with Bayesian posteriors and Rashomon-restricted spaces as special cases.The conclusion reports superior performance across key metrics and decreased computational cost.Source: Conclusion · p. 6

Future Work

  • Future work could characterize the amortization gap between instance-optimal latent representations and the shared encoder as the Rashomon set evolves.The authors suggest adaptive refinement strategies when hypothesis-space variability induces large deviations.Source: Conclusion · p. 6
  • The framework could be extended to further hypothesis spaces, such as a Laplace approximation around the maximum a posteriori solution.Source: Conclusion · p. 6

Appendix

The appendix provides variational-model background, evaluation-metric definitions, implementation details, and supplementary experiments. It also includes proof material for theoretical results.

Proofs

Variational Background

Metrics

  • The evaluation metrics cover target-label validity, target-class distance, autoencoder manifold alignment, posterior-predictive smoothness, and generator stability under input perturbations.These are respectively described as Validity, Implausibility, IM1, Robustness to Noisy Executions, and Robustness to Input Changes.Source: Metrics · p. 11

Implementation Details

  • Implementation approximates Rashomon-restricted expectations with 50 Monte Carlo dropout passes and latent expectations with five samples.The retained predictors form an empirical uniform distribution subject to a validation-loss constraint under a deterministic ϵ.Source: Implementation Details · p. 11

Supplementary Experiments

Ablations

The ablation studies examine each term across each metric in Figures 6 and 7.

Ablations

  • Figures 6 and 7 present ablation studies on each term across each metric.Source: Ablations · p. 11

Additional Methods

This section identifies a standard CVAE inference comparison and presents additional evaluation scopes for image and tabular datasets, alongside ε-sweep and ablation figures.

Additional Methods

  • The comparison includes standard Conditional VAE inference p(x | y′, z), using a learned p(x | y, z), and Wachter et al. (Wachter, Mittelstadt, and Russell 2018).Source: Additional Methods · p. 11
  • Table 3 reports amortized counterfactual evaluation on high-dimensional image datasets as mean ± standard deviation over five seeds.Source: Table 3 · p. 12
  • Table 4 reports amortized counterfactual evaluation on Adult Income and Breast Cancer as mean ± standard deviation over five seeds.Source: Table 4 · p. 12
  • Figure 3 examines the impact of the ε hyperparameter for the AVCG-R Rashomon set on Adult Income.Source: Figure 3 · p. 13
  • Table 5 reports amortized counterfactual evaluation on Heart Disease and Spambase as mean ± standard deviation over five seeds.Source: Table 5 · p. 13
  • Figures 4 and 5 examine ε-sweeps for the AVCG-R Rashomon set on Breast Cancer and Heart Disease, respectively.Source: Figure 4 · p. 14, Figure 5 · p. 15
  • Figures 6 and 7 present ablation studies across key metrics on Heart Disease and Spambase, respectively.Source: Figure 6 · p. 16, Figure 7 · p. 16

Related papers

Loading 2609.07917v1…