Source-linked AI summary
Counterfactual Visual Explanations
Yash Goyal, Ziyan Wu, Jan Ernst, Dhruv Batra, Devi Parikh, Stefan Lee
TL;DR
The paper addresses the challenge of explaining why a vision system chose one class instead of a specified alternative. It generates counterfactual visual explanations by transferring discriminative regions from a distractor image, and finds that these explanations improve human performance in fine-grained bird classification when added to training examples.
Problem
Vision systems need explanations that clarify why an observed class was chosen instead of a specific unselected outcome.
Method
The method selects a distractor image predicted as class c′ and identifies spatial regions whose replacement in query image I would push the model toward c′.
Results
Users trained for fine-grained bird classification perform better with counterfactual explanations in addition to training examples.
Takeaways & Limitations
Counterfactual explanations provide guidance about which bird parts distinguish species and can support machine teaching.
Takeaways & Limitations
The minimum-edit search can become computationally large, with over a million candidate subsets for k = 2 when h = w = 16.
Abstract
from arXiv · showhide
In this work, we develop a technique to produce counterfactual visual explanations. Given a 'query' image $I$ for which a vision system predicts class $c$, a counterfactual visual explanation identifies how $I$ could change such that the system would output a different specified class $c'$. To do this, we select a 'distractor' image $I'$ that the system predicts as class $c'$ and identify spatial regions in $I$ and $I'$ such that replacing the identified region in $I$ with the identified region in $I'$ would push the system towards classifying $I$ as $c'$. We apply our approach to multiple image classification datasets generating qualitative results showcasing the interpretability and discriminativeness of our counterfactual explanations. To explore the effectiveness of our explanations in teaching humans, we present machine teaching experiments for the task of fine-grained bird classification. We find that users trained to distinguish bird species fare better when given access to counterfactual explanations in addition to training examples.
1. Introduction
The paper introduces counterfactual visual explanations that show how changing regions of an image could make a vision model choose a specified alternative class. It also argues that these explanations support human teaching by highlighting discriminative differences between similar bird species.
- Motivation: Counterfactual explanations answer why a model selected class c rather than a specific alternative c′ by describing changes that would produce the alternative decision.This frames explanations around likely unselected outcomes rather than only the observed prediction.
- Approach: The approach identifies regions in a query image and a distractor image whose replacement would move the model toward the distractor’s class.For example, a Crested Auklet query can be compared with a Red Faced Cormorant distractor.
- Interpretability: Unlike standard feature attribution, the method seeks discriminative regions that distinguish visually similar categories rather than regions shared across both classes.The paper illustrates this contrast using bird features such as crowns, necks, and beaks.
- Evaluation: The authors apply the method across multiple image-classification datasets and analyze CUB explanations using segmentation and keypoint annotations to show that they highlight discriminative bird parts.The explanations are presented as both interpretable and discriminative.
- Machine Teaching: Users learning fine-grained bird classification perform better when given counterfactual explanations alongside training examples.The proposed teaching format tells learners where to look and how a bird would need to differ to resemble another species.
2. Approach
The approach finds minimal spatial feature edits that transform a query image toward a distractor class, using either exhaustive greedy search or a continuous relaxation. It decomposes the CNN into spatial extraction and decision components, then identifies corresponding query and distractor cells whose replacement changes the prediction.
- 2.1. Minimum-Edit Counterfactual Problem: The CNN is decomposed into a spatial feature extractor f(I) and a decision network g(f(I)), enabling edits in spatial feature space.The feature extractor produces spatial cells, while the decision network maps them to class outputs.
- 2.1. Minimum-Edit Counterfactual Problem: The method seeks the minimum number of region replacements from query image I using distractor image I′ so the model predicts class c′.The distractor is already classified as c′, avoiding direct pixel optimization and trivial full-image replacement.
- 2.1. Minimum-Edit Counterfactual Problem: The optimized edit set identifies discriminative attention maps in both images: edited query cells and the corresponding source cells in the distractor.The number of edits grows combinatorially, with O((h × w)^(2+k)) candidate subsets and over a million candidates for h = w = 16 and k = 2.
- 2.1. Minimum-Edit Counterfactual Problem: A binary gating vector a selects query cells to replace, while permutation matrix P maps selected cells from distractor features f(I′) into f(I).The resulting transformation is f(I*) = (1 − a) ◦ f(I) + a ◦ P f(I′).
- 2.2. Greedy Sequential Exhaustive Search: Greedy exhaustive search evaluates every query–distractor cell pair, selects the pair maximizing distractor-class log-probability, and repeats until the decision changes.Previously selected edits are excluded in subsequent sequential searches.
- 2.3. Continuous Relaxation: A continuous relaxation replaces binary a and permutation P with softmax-parameterized distributions optimized by gradient descent, with entropy losses encouraging nearly binary solutions.This relaxation is used as an alternative best-edit procedure and requires fewer evaluation calls on average.
3. Related Work
The paper situates counterfactual visual explanations among attribution-based and counterfactual explanation methods, and connects them to machine teaching with visual explanations.
- Visual Explanations: Feature attribution methods highlight image regions considered important for a model’s prediction, often using gradient-based techniques and upsampling.
- Counterfactual Explanations: Counterfactual explanation methods address why a model predicted class c instead of class c′, but differ in output modality and generation process.
- Counterfactual Explanations: This paper’s visual explanations require no additional attribute annotations and are generated directly from the target model, supporting faithfulness by design.
- Machine Teaching: Machine teaching research studies how examples can be selected or ordered to improve human learning, including applications where models teach tasks they perform superhumanly.
- Machine Teaching: Prior visual machine-teaching work used saliency maps with heuristics, while this paper compares against a GradCAM-based explanation baseline.
4. Experiments
The approach is evaluated across four image-classification datasets, producing counterfactual edits that expose discriminative visual regions and can shift model predictions toward distractor classes.
- The study evaluates counterfactual explanations on SHAPES, MNIST, Omniglot, and CUB 2011, using qualitative results to assess interpretability and discriminativeness.
- MNIST: MNIST visualizations replace highlighted query and distractor regions to create composite digits, with edits such as transforming a 3 toward a 5.
- MNIST: 2.67 edits on average change MNIST predictions from c to c′, while the approach runs at 15 µs per image on a Titan XP GPU.
- Omniglot: 1.46 edits on average change Omniglot predictions from c to c′, and the method identifies appropriate edits despite the characters’ complex shapes.
- Caltech-UCSD Birds (CUB): On CUB, explanations highlight bird attributes including head plumage, yellow wing spots, and wing texture, while some outputs are less interpretable because of pose misalignment.
- Caltech-UCSD Birds (CUB): 75% of query-image edits and 80% of distractor-image edits occur near bird keypoints; agreement is 78% across distractors of one class but 42% across distractor classes.
5. Machine Teaching
The machine-teaching study uses counterfactual feedback to guide humans toward bird features distinguishing two categories. Participants train with examples and feedback, then classify test images without examples or feedback.
- The interface trains and tests subjects on CUB bird classification using two category options, Alpha and Bravo.
- During training, incorrect answers trigger counterfactual attention maps based on a nearby distractor image selected by keypoint similarity.
- During testing, subjects classify images without example images or feedback, enabling comparison of learned bird-category distinctions.
- 78.77% mean test accuracy with counterfactual explanations exceeded 74.29% with GradCAM and 71.09% without explanations.
- Among nine participants without ML knowledge, accuracy was 72.4% with counterfactual explanations versus 61.7% without explanations.
6. Conclusion
The paper presents counterfactual visual explanations that specify how an image should change for a model to predict a target class, and reports informative explanations across datasets plus guidance for human bird classification.
- The approach answers how an image should differ for a model to predict a specified class c′ and produces informative explanations across multiple datasets.
- In fine-grained bird classification, counterfactual explanations provide guidance that helps humans perform better on the task.
Appendix
The appendix provides additional results on the SHAPES dataset.
- Additional SHAPES-dataset results are provided in Appendix I.
7. Experiments on SHAPES
The SHAPES experiment evaluates counterfactual attention maps on simple 3×3 grid images containing one colored shape. Because the true shape locations are known, the approach achieves 100% accurate attention maps.
- Dataset: SHAPES images are 30×30-pixel 3×3 grids with one cell containing a colored circle, square, or triangle that determines the label.Shapes vary across blue, green, and red, with small random perturbations in size and pixel values.
- Classification model: A one-convolutional-layer CNN with two fully connected layers achieves 100% test accuracy on the three-class task.The authors attribute this performance to the simplicity of the dataset.
- Experimental settings: For each query image, the experiment randomly selects a distractor class different from the prediction and a distractor image predicted as that class.Spatial features have size 3 × 3 × 100.
- Results: 100% accurate attention maps are recovered because the known shape locations provide ground truth for each query–distractor pair.Figure 11 illustrates the approach identifying the query region and corresponding distractor region for a counterfactual replacement.