Explanations based on the Missing: Towards Contrastive Explanations with Pertinent Negatives

Amit Dhurandhar; Pin-Yu Chen; Ronny Luss; Chun-Chen Tu; Paishun Ting; Karthikeyan Shanmugam; Payel Das

Source-linked AI summary

Explanations based on the Missing: Towards Contrastive Explanations with Pertinent Negatives

Amit Dhurandhar, Pin-Yu Chen, Ronny Luss, Chun-Chen Tu, Paishun Ting, Karthikeyan Shanmugam, Payel Das

arXiv:1802.07623v2Feb 21, 2018cs.AIcs.CVcs.LG

TL;DR

Black-box explanation methods have not explicitly identified features that must be absent to preserve a classification. CEM finds minimally sufficient present features and critically absent contrastive features, producing precise explanations across three real-world datasets that match human intuition.

Problem
Existing neural-network explanation methods do not explicitly identify minimally and critically absent features needed to justify a classification.
Method
CEM identifies minimally sufficient pertinent positives and contrastive perturbations representing pertinent negatives that should be absent to preserve the classification.
Results
Across MNIST, procurement fraud, and fMRI datasets, CEM generated precise explanations that matched human intuition and were easy for experts to evaluate.
Takeaways & Limitations
Including what must be absent yields more complete and well-rounded explanations of black-box classifications.
Takeaways & Limitations
Autoencoder-based representations may be infeasible when data are insufficient or low quality.

Abstract

from arXiv · show

In this paper we propose a novel method that provides contrastive explanations justifying the classification of an input by a black box classifier such as a deep neural network. Given an input we find what should be %necessarily and minimally and sufficiently present (viz. important object pixels in an image) to justify its classification and analogously what should be minimally and necessarily \emph{absent} (viz. certain background pixels). We argue that such explanations are natural for humans and are used commonly in domains such as health care and criminology. What is minimally but critically \emph{absent} is an important part of an explanation, which to the best of our knowledge, has not been explicitly identified by current explanation methods that explain predictions of neural networks. We validate our approach on three real datasets obtained from diverse domains; namely, a handwritten digits dataset MNIST, a large procurement fraud dataset and a brain activity strength dataset. In all three cases, we witness the power of our approach in generating precise explanations that are also easy for human experts to understand and evaluate.

1 Introduction

The paper introduces contrastive explanations for neural-network classifications by identifying both minimally sufficient present features and minimally critical absent features. It proposes CEM, which seeks realistic explanations near the data manifold and validates them across MNIST, procurement fraud, and fMRI datasets.

Motivation: Human explanations commonly combine present characteristics with minimally absent characteristics to distinguish an input from its closest alternative class.The absence of glasses distinguishes Steve from a similar person who wears them.
CEM: CEM identifies pertinent positives that are minimally sufficient for the current classification and pertinent negatives whose presence would change it.For an MNIST digit classified as 3, CEM highlights 3-shaped pixels as present and a top horizontal line associated with 5 as absent.
Contrastive explanations: Unlike LRP and LIME, CEM explicitly highlights pertinent negatives as evidence needed to distinguish the predicted class from a close alternative.The paper frames pertinent positives and negatives together as a more complete explanation of the classification hypothesis.
Method: CEM searches for minimal present and absent features near the data manifold using a convolutional autoencoder to produce more realistic, human-intuitive explanations.The autoencoder may be unsuitable when data are insufficient or poor quality, or unnecessary when all feature-value combinations have domain semantics.
Validation: The approaches are validated on three real-world datasets: MNIST, a large-corporation procurement fraud dataset, and an ABIDE I resting-state fMRI dataset.MNIST explanations are generated both with and without an autoencoder.

2 Related Work

Prior interpretability work includes rule-based, prototype-based, psychometric, human-consumable, and formal approaches, while neural-network explanations commonly use class prototypes or highlight relevant input parts. The paper distinguishes its per-input pertinent positives and negatives from present-feature-focused methods, global anchors, and adversarial attacks.

Interpretable modeling: Interpretability research spans rule or decision lists, prototype exploration, psychometric methods, human-consumable models, and formal measures.These approaches include efforts to devise interpretable models and to formalize or quantify interpretability.
Neural-network explanations: A survey groups neural-network understanding methods into class prototypes and explanations that highlight relevant parts of an input.Related applications include vision and natural-language processing.
Neural-network explanations: Most existing explanation methods emphasize present features and do not identify features necessarily and sufficiently present or absent for an individual classification.Some perturbation methods remove features, but the passage distinguishes them from identifying pertinent positives and negatives.
Anchors: Anchors provide global feature-value rules sufficient to predict a class, whereas pertinent positives and negatives are customized per input and can usually be found without dataset-wide anchors.The passage states that a dataset may not always possess anchors, while pertinent positives and negatives can almost always be found.
Adversarial examples: Unlike largely unconstrained untargeted adversarial attacks, pertinent positives allow only deletions and pertinent negatives only additions, with a distinct minimally sufficient objective for pertinent positives.The passage identifies simultaneous additions and deletions as a key difference from the proposed constraints.

3 Contrastive Explanations Method

The Contrastive Explanations Method formulates pertinent-positive and pertinent-negative explanations as constrained perturbation optimizations around an input. It combines prediction-based losses, elastic-net sparsity regularization, and optional autoencoder reconstruction to produce interpretable present and absent features.

Contrastive Explanations Method: CEM optimizes a perturbation δ added to input x0 over feasible missing or existing components to explain the neural network’s prediction.The modified example is x = x0 + δ, with pertinent negatives drawn from X/x0 and pertinent positives from X ∩ x0.
Pertinent Negative Analysis: A pertinent negative identifies minimally added features that change x0’s top-1 predicted class from t0.Its loss encourages a different class and uses κ ≥0 to control the separation between the original-class score and the highest competing score.
Objective and Data-Manifold Constraint: The optimization uses an elastic-net regularizer for feature selection and an L2 autoencoder reconstruction error to keep modified examples near the natural data manifold.The reconstruction term is usable when a well-trained domain autoencoder is available; otherwise γ is set to zero.
Pertinent Positive Analysis: A pertinent positive identifies existing features whose retained perturbation δ preserves x0’s top-1 class, making δ representative of the prediction.The positive loss is minimized when [Pred(δ)]t0 exceeds the highest competing class score by at least κ.
Optimization: Projected FISTA solves the pertinent-positive and pertinent-negative optimization problems, with L1 shrinkage supporting efficient optimization and sparse perturbations.The method projects iterates onto the feasible perturbation set and applies an analogous procedure to both analyses.

4 Experiments

Experiments on MNIST, procurement fraud, and brain-imaging datasets show that CEM produces informative contrastive explanations, including pertinent positives and negatives, across diverse domains. Comparisons with LRP and LIME, expert evaluations, and class-preservation tests support the method’s explanatory quality and predictive effectiveness.

Datasets: CEM was evaluated on MNIST, a procurement fraud dataset, and an fMRI brain-imaging dataset spanning normal and autistic individuals.The procurement data included millions of invoices and tens of thousands of vendors, while the brain dataset contained activity patterns for both populations.
MNIST: On MNIST, the convolutional autoencoder improved CEM’s pertinent positives and negatives in many cases, although explanations without it were already convincing.The classifier used a feed-forward CNN trained on 60,000 images and achieved 99.4% test accuracy.
MNIST: Unlike LRP and LIME, CEM illustrates what is necessary and sufficient for classifier results through contrastive explanations.LRP uses pixel-level relevance and LIME uses segmented superpixels, but neither identifies necessary and sufficient information in the same way.
Procurement fraud: Expert evaluations on 45 procurement invoices found that CEM’s pertinent positives and negatives closely matched human judgment and made explanations more complete and intuitive.The evaluation sampled 15 invoices from each of the low-, medium-, and high-risk classes; Table 2 provided expert-validated examples.
Brain imaging: In brain connectome data, CEM outperformed LRP in robustly identifying pertinent positive information and could help reduce diagnostic false positives and false negatives.The comparison used ten subjects, five classified as autistic and five as neuro-typical, focusing on atypical functional connectivity patterns.
Cross-dataset results: 100% of CEM pertinent positives and negatives maintained or switched classes, respectively, across both cases and all three datasets.The result supports producing informative, potentially sparse, and predictive explanations across diverse domains.

5 Discussion

The discussion presents CEM as a novel method that identifies minimally present features and necessarily absent contrastive additions for black-box classification explanations. It highlights the method’s usefulness across domains, while noting that pertinent negatives are most valuable for distinguishing similar inputs and may overwhelm users when classes differ greatly.

Limitations: Pertinent negatives are most useful when inputs from different classes are close, such as distinguishing flu from pneumonia.When classes are extremely different, pertinent positives may be sufficient to characterize the input.
Applications: CEM could support model selection, debugging, bias detection, and model improvement beyond generating explanations.The authors suggest preferring models with better explanations when test accuracy is equal because they may be more robust.
Summary: CEM identifies minimally present input features and necessarily absent contrastive additions to justify black-box classifications.The authors describe this as, to their knowledge, the first explanation method to achieve both goals.
Summary: The approach was validated on multiple datasets from different domains and produced explanations that matched human intuition.The authors characterize these explanations as more complete and well-rounded.

A Experiments: FISTA details

The projected FISTA implementation uses fixed regularization coefficients and an adaptive nine-step search for c, with each search run for 1000 SGD iterations.

Projected FISTA sets β = 0.1 and γ = {0, 100} for finding pertinent negatives and pertinent positives.
The parameter c starts at 0.1 and is searched 9 times using run-time information.If fκ never reaches 0, the next c is multiplied by 10; otherwise, it is averaged with the current value.
Each c search runs I = 1000 iterations using the SGD solver.

B MNIST · B.1 Setup

The MNIST setup uses a convolutional neural network trained on 60,000 images and a separate convolutional autoencoder that compresses 28×28 inputs to 14×14 feature maps. Both architectures use convolutional layers, ReLU activations, and max-pooling operations.

B.1 Setup: The classifier is a feed-forward convolutional neural network trained on 60,000 MNIST training images.
B.1 Setup: The CNN contains two convolution-convolution-pooling layer sets followed by three fully connected layers.
B.1 Setup: All CNN convolutional layers use ReLU activation, while pooling uses a 2 × 2 max-pooling kernel for downsampling.
B.1 Setup: The convolutional autoencoder consists of an encoder and a decoder.
B.1 Setup: Its encoder compresses each 28×28 input image to a 14×14 feature map using convolution-convolution-pooling-convolution.
B.1 Setup: The autoencoder’s first two convolutional layers each contain 16 filters with 3 × 3 × D kernels and ReLU activation.
B.1 Setup: The autoencoder uses a 2 × 2 max-pooling layer.

B.2 Results · C Procurement Fraud

On MNIST, CEM explanations using a CAE are visually more interpretable for both pertinent positives and pertinent negatives. The CAE makes digit outlines more pronounced and uses a few extra pixels to show a 6-to-4 transformation.

B.2 Results: CAE-based explanations are visually more interpretable for pertinent positives on MNIST.This is illustrated in Figure 4.
B.2 Results: CAE-based explanations are also visually more interpretable for pertinent negatives on MNIST.
B.2 Results: In the third row, the outline of the 5 is much more pronounced with a CAE.
B.2 Results: In the seventh row, the outline of the 3 is similarly more pronounced with a CAE.
B.2 Results: The same interpretability trend holds across pertinent-positive and pertinent-negative explanations.
B.2 Results: In the second row, a few extra pixels transform the 6 to a 4 and clearly show the transformation.

C.1 Dataset Details · D Brain Functional Imaging

The supplied dataset details describe procurement-fraud data assembled from vendor, commodity, financial, government, country-risk, and business-registration sources. These sources provide identifying, behavioral, risk, and authenticity-related information about businesses.

C.1 Dataset Details: The VMF records vendor names, addresses, account numbers, and registration dates.These fields describe registered vendors and their company-associated information.
C.1 Dataset Details: The RVL and RCL list potentially fraudulent vendors and manipulable commodities.The passage identifies both vendors and commodities as potentially vulnerable to manipulation.
C.1 Dataset Details: The FI contains vendor maturity information and stock trends.These variables capture vendor maturity and financial-market behavior.
C.1 Dataset Details: The annual US-government FPL provides two lists of suspect businesses.The FPL is described as a recurring government-released source.
C.1 Dataset Details: The CPI publicly scores country risk on a 0-100 scale, with lower scores indicating worse perception and higher risk.The supplied passage also connects country risk to tax havens, where taxes are minimal and financial privacy is maintained.
C.1 Dataset Details: Dun & Bradstreet assigns each registered business a unique DUNS number and DUNS name.A DUNS ID provides a certain level of authenticity to the business.

D.1 Dataset Details

The study uses ABIDE I, a large public resting-state fMRI dataset containing individuals with autism spectrum disorder and neurotypical individuals. Resting-state fMRI measures functional relationships between brain regions and supports investigation of clinical populations.

D.1 Dataset Details: ABIDE I provides resting-state fMRI acquisitions from individuals diagnosed with autism spectrum disorder and neurotypical individuals.The dataset is publicly available and large.
D.1 Dataset Details: Resting-state fMRI captures neural measurements of functional relationships between brain regions.These measurements are particularly useful for investigating clinical populations.