Source-linked AI summary
Explaining Classifiers with Causal Concept Effect (CaCE)
Yash Goyal, Amir Feder, Uri Shalit, Been Kim
TL;DR
Existing interpretability methods can confuse correlated concepts with causes, making it difficult to understand classification decisions reliably. The paper defines CaCE and uses conditional VAEs to approximate concept interventions. Across multiple datasets, including high-dimensional images, VAE-CaCE estimates the true causal effect and closely matches GT-CaCE.
Problem
Existing explanation methods may reflect correlations confounded in the data rather than the causal effect of a human-interpretable concept on a classifier’s prediction.
Method
The paper defines CaCE and uses conditional VAEs to generate counterfactuals by intervening on concept values, approximating otherwise difficult image interventions.
Results
The approach estimates the true causal effect across multiple datasets, including high-dimensional images, and closely matches GT-CaCE.
Takeaways & Limitations
CaCE provides a quantitative causal explanation of how a concept’s presence or absence affects a classifier’s output, rather than merely identifying correlated concepts.
Takeaways & Limitations
Real-world CaCE estimation is constrained by incomplete knowledge of natural-image causal graphs and unaccounted-for confounding factors.
Abstract
from arXiv · showhide
How can we understand classification decisions made by deep neural networks? Many existing explainability methods rely solely on correlations and fail to account for confounding, which may result in potentially misleading explanations. To overcome this problem, we define the Causal Concept Effect (CaCE) as the causal effect of (the presence or absence of) a human-interpretable concept on a deep neural net's predictions. We show that the CaCE measure can avoid errors stemming from confounding. Estimating CaCE is difficult in situations where we cannot easily simulate the do-operator. To mitigate this problem, we use a generative model, specifically a Variational AutoEncoder (VAE), to measure VAE-CaCE. In an extensive experimental analysis, we show that the VAE-CaCE is able to estimate the true concept causal effect, compared to baselines for a number of datasets including high dimensional images.
1. Introduction
The paper targets misleading concept explanations caused by confounding and introduces CaCE to measure whether a high-level concept causally changes a classifier’s output. It approximates this effect with conditional generative models and demonstrates effectiveness across datasets, including high-dimensional images.
- Motivation: Interpretability methods can mistake correlated concepts for causes, as cars may correlate with bicycles even when removing cars leaves the bicycle prediction unchanged.The example motivates distinguishing observational correlation from the effect of intervening on a concept.
- Causal Concept Effect: CaCE measures the causal effect of a concept’s presence or absence on a deep model’s prediction rather than its mere correlation with that prediction.It is intended for high-level concepts and summarizes predictions for an entire class.
- Causal Concept Effect: Global concept explanations are especially vulnerable to confounding because they summarize all data points and concepts are often correlated in datasets.The framework seeks concepts whose isolated presence or absence causally affects the model output.
- Estimation challenge: Directly adding or removing concepts in high-dimensional images is difficult, making exact intervention-based estimation of CaCE challenging.Examples include replacing male doctors with otherwise identical female doctors or editing cars out of bicycle images.
- Estimation approach: Conditional VAEs generate counterfactuals by intervening on concept values, approximating the natural generation process and thereby estimating CaCE.The approach is trained on the classifier’s training data and uses different concept inputs to the generative network.
- Empirical evaluation: The proposed framework estimates the true causal effect of concept explanations across multiple datasets, including high-dimensional images.This is presented as an effectiveness demonstration of the approach.
2. Related Work
The related work distinguishes global from local explanations and identifies confounded concepts as a key weakness of global concept methods. The paper addresses this weakness by formalizing causal effects for model outputs, while leaving some local confounding cases for future work.
- Causal explanations: CaCE formalizes the causal effect of a concept explanation on a trained model’s output as a focused notion of explanation.This connects the paper to broader work on causality and explainability.
- Global and local explanations: Global methods explain predictions for an entire class, whereas local methods explain a single test instance.Global explanations are described as more informative for evaluating overall robustness and deployment decisions.
- Confounding in global explanations: Global concept methods such as TCAV can provide misleading explanations when concepts like cars and roads are highly correlated in the data.The paper frames confounding as a limitation of global explanation methods.
- Scope boundary: Most local interpretation methods do not face this confounding problem because pixels deterministically cause a given model output.The paper notes that data-dependent pixel perturbations can still introduce confounding and leaves those cases for future work.
3. CaCE: Causal Concept Effect
CaCE explains classifier predictions through the causal effect of interpretable concepts, formalized within a structural causal model and estimated using interventions or generative approximations. Synthetic experiments show that CaCE remains zero for irrelevant concepts even when correlations mislead TCAV.
- Causal Concept Effect: CaCE measures how changing a human-interpretable concept’s presence or absence affects a classifier’s output, rather than merely correlating with it.The framework targets global explanations for an entire class and defines concepts as causes of images, including objects and abstract properties.
- Structural causal model: The image-generation model treats binary concepts as causes of images, while the trained classifier maps images mechanistically to outputs.The structural causal model includes concept-generation noise and image-generation noise; interventions set one concept to a chosen value.
- Structural causal model: CaCE assumes an atomic intervention can change one concept while keeping all other concepts fixed, which may fail for mutually exclusive concepts.Expectations under these interventions are represented with the do-operator.
- Causal Concept Effect: For categorical concepts, CaCE compares two concept values and uses one value as a base while marginalizing over the alternatives.The binary case recovers the comparison between presence and absence.
- Why do we need CaCE?: 90% versus 10% color-label associations left CaCE at zero but raised TCAV’s color score to 1.0, showing confounding can mislead correlation-based explanations.The classifier made identical predictions for images differing only in bar color, despite the biased dataset.
4. Measuring CaCE
CaCE measures concept effects through interventions rather than correlations. Because direct image interventions are difficult, the paper approximates them with conditional VAEs, using decoder-only or encoder-decoder counterfactual generation.
- Challenges: Real-world CaCE estimation is difficult because the causal graph of natural images is unknown, preventing reliable adjustment for all confounders.Image pixels are downstream from concepts, so conditioning on pixels cannot solve the confounding problem.
- Approaches: The paper evaluates controlled interventions for exact GT-CaCE and generative models for approximate CaCE, comparing estimates with ground truth and other explanation methods.It also proposes diagnostic tests to increase confidence in VAE-based estimates.
- Conditional VAE: A conditional VAE models p(I|C0 = a, L = l), allowing concept interventions while conditioning on the class label.The class label supports class-conditional CaCE and improves the generative model compared with omitting it.
- Dec-CaCE: Dec-CaCE generates counterfactual image pairs by fixing the latent vector and class label while changing the concept value.The paired images are intended to differ only in the concept.
- EncDec-CaCE: EncDec-CaCE infers a latent distribution for each image, decodes a counterfactual with the flipped concept, and averages prediction differences over image sets.This enables CaCE estimation for particular images or subsets, unlike the generative-network-only approach.
- Diagnostics: The VAE approach assumes limited hidden confounding and successful disentanglement, so the paper uses positive-effect and null-effect diagnostics.Passing either diagnostic does not establish that the estimated CaCE is correct.
5. Results
Across synthetic and high-dimensional image datasets, the proposed CaCE estimators generally track causal effects better than correlation-based baselines, while model complexity and data availability affect estimates.
- Datasets and evaluation: The experiments cover BARS, colored-MNIST, COCO-Miniplaces, and CelebA, using ground-truth causal effects where the data-generation process is controllable.CelebA instead uses StarGAN-based approximations for the blonde-hair concept because its image-generation process is uncontrolled.
- Metric and baselines: CaCE outputs a vector matching the classifier’s output dimension; multiclass colored-MNIST reports mean absolute probability differences across classes.For binary tasks, results report the class-0 score without loss of information.
- BARS, colored-MNIST, and COCO-Miniplaces: Correlation-based CONEXP and TCAV overestimate concept importance when concepts correlate with labels but do not causally affect predictions.This pattern appears in BARS, colored-MNIST, and COCO-Miniplaces, whereas the proposed estimators remain close to ground truth where reported.
- BARS, colored-MNIST, and COCO-Miniplaces: When color and class are highly correlated in BARS, GT-CaCE is 0.58, and Dec-CaCE estimates it correctly while CONEXP and TCAV reflect correlation instead.Among the proposed methods, Dec-CaCE performs better than EncDec-CaCE in this experiment.
- Model complexity: More complex classifiers tend to produce higher CaCE values under color-class correlation, while convolutional DC-VAEs estimate CaCE 8-49% closer to GT-CaCE than fully connected alternatives.The classifier comparison contrasts ResNet-100 with simpler models; both DC-VAE variants still estimate CaCE reasonably well.
- Diagnostics: The colored-MNIST diagnostics observe average GT-CaCE of 0.152 for class-label interventions and CaCE of 0.003 for an independent dummy concept.The authors interpret these results as passing both diagnostic tests.
6. Conclusions
The paper proposes CaCE to explain classifier predictions through the causal effect of a concept’s presence or absence, rather than correlation alone. It estimates CaCE with conditional VAEs and reports close agreement with GT-CaCE across datasets.
- CaCE measures the causal effect of a concept’s presence or absence on a classifier’s output.
- Conditional VAEs provide a way to estimate CaCE when direct interventions are difficult to perform.
- The method closely matches GT-CaCE across a number of datasets.