PaperlayerView exact revision on arXiv

Source-linked AI summary

Explaining Image Classifiers by Counterfactual Generation

Chun-Hao Chang, Elliot Creager, Anna Goldenberg, David Duvenaud

arXiv:1807.08024v3cs.CV

TL;DR

Image saliency requires identifying which regions would most change a classifier’s decision when unobserved, without relying on unrealistic replacements. FIDO marginalizes masked regions using conditional generative in-filling and optimizes Bernoulli dropout masks, producing compact, relevant maps with fewer artifacts than previous methods. Its explanations remain bounded by the generative model’s ability to capture conditional image densities.

  • Problem

    Existing saliency methods use heuristic references that can produce unnatural inputs and ignore relationships between surrounding image regions.

  • Method

    FIDO optimizes a Bernoulli dropout distribution while sampling masked-region replacements from a conditional generative model.

  • Results

    FIDO produces more parsimonious saliency maps that highlight contextual information relevant to predictions and consistent with the training distribution.

  • Takeaways & Limitations

    Generative in-filling mitigates artifacts and encourages counterfactual inputs closer to the natural image manifold.

  • Takeaways & Limitations

    FIDO’s performance is upper-bounded by how well its generative model captures the conditional input density.

Abstract

from arXiv · show

When an image classifier makes a prediction, which parts of the image are relevant and why? We can rephrase this question to ask: which parts of the image, if they were not seen by the classifier, would most change its decision? Producing an answer requires marginalizing over images that could have been seen but weren't. We can sample plausible image in-fills by conditioning a generative model on the rest of the image. We then optimize to find the image regions that most change the classifier's decision after in-fill. Our approach contrasts with ad-hoc in-filling approaches, such as blurring or injecting noise, which generate inputs far from the data distribution, and ignore informative relationships between different parts of the image. Our method produces more compact and relevant saliency maps, with fewer artifacts compared to previous methods.

1 INTRODUCTION

The paper reframes saliency as finding regions whose replacement by plausible alternatives most changes classifier output. Its generative in-filling framework produces more concentrated, relevant ImageNet saliency maps than existing methods.

  • Heuristic references such as blur or random colors can create unnatural in-filled images because they ignore surrounding-pixel context.
  • The proposed framework replaces masked regions with plausible alternatives sampled by conditioning a generative model on the visible image.
  • The method is model-agnostic for differentiable classifiers and uses variational Bernoulli dropout to compute feature importance.
  • On ImageNet, the resulting saliency maps identify more relevant and concentrated pixels than existing methods.

2 RELATED WORK

Prior saliency methods use gradients, reference substitutions, or region-wise marginalization, each with limitations in locality, nonlinear interactions, computational cost, or data-distribution fidelity.

  • Gradient-based methods compute sensitivity using local classifier gradients, but gradient saturation, discontinuities, and architectural inductive biases can affect their explanations.
  • The graphical-model formulation partitions an image into an unobserved region and its complement before replacing the unobserved region with a reference value.
  • Reference-based methods substitute pixels with uninformative values, while some address gradient discontinuity but ignore nonlinear interactions between inputs.
  • Zintgraf et al. marginalize each pixel or patch, whereas the proposed variational Bernoulli approach searches efficiently for sparse regions and models interactions between disjoint regions.
  • Fong and Vedaldi optimize perturbations using heuristic references including mean pixels, Gaussian noise, or blur; Dabkowski and Gal amortize perturbation estimation with an auxiliary network.

3 PROPOSED METHOD

FIDO formulates saliency as optimizing masks while marginalizing unobserved pixels with conditional generative in-filling. A factorized Bernoulli mask supports efficient, sparse searches over regions and models interactions among disjoint pixels.

  • Proposed Method: FIDO replaces heuristic reference values with samples from a strong conditional generative model, marginalizing pixels inside or outside the mask.The model conditions in-fills on the observed image context, preserving relationships between masked and unmasked regions.
  • Proposed Method: Conditional generative in-filling produces texture consistent with image surroundings, ameliorating artifacts observed with heuristic in-fills.The method is intended to make saliency explanations more plausible under the natural-image distribution.
  • 3.1 OBJECTIVE FUNCTIONS: SDR minimizes classifier confidence after replacing a small masked region with reference pixels, whereas SSR maximizes confidence while penalizing mask size.Both objectives trade off classifier score against the number of selected pixels.
  • Proposed Method: The method represents saliency with a factorized Bernoulli distribution over binary pixel masks and optimizes each pixel’s dropout probability.The resulting parameter map indicates pixels marginalized for SDR or retained for SSR.
  • 3.2 COMPARISON TO FONG & VEDALDI (2017): FIDO uses stochastic mini-batch mask sampling to explore binary masks efficiently and obtain uncertainty estimates, unlike BBMP’s local continuous-mask search.BBMP also uses heuristic in-filling, while FIDO samples reference values from a learned generative model.

4 EXPERIMENTS

Experiments evaluate FIDO’s infilling strategies, objective functions, classifier dependence, and quantitative explanation quality. Generative infilling and LSSR generally produce more focused, less artifact-prone, and more parsimonious saliency maps than heuristic alternatives.

  • 4.1 INFILLING METHODS: FIDO uses heuristic and generative infilling methods, including Mean, Blur, Random, Local, VAE, and CA.Generative methods condition in-filling on non-masked image content, while heuristic methods use fixed or ad-hoc references.
  • 4.1 INFILLING METHODS: Generative Contextual Attention infilling produces texture more consistent with surrounding content than heuristic in-fills.Figure 3 compares the methods using a centered mask and reports that CA ameliorates the distribution mismatch visible with heuristic in-fills.
  • 4.2 COMPARING THE SDR AND SSR OBJECTIVE FUNCTIONS: LSSR produces fewer artifacts than LSDR, particularly when weak Mean infilling is used.The authors attribute this difference to the relative ease of optimizing LSDR and use LSSR in subsequent experiments.
  • 4.3 COMPARING INFILLING METHODS: Generative infilling methods produce more focused explanations and fewer artifacts than Mean, Blur, and Random heuristics.FIDO-CA produces the fewest artifacts according to the proportion of salient-map pixels outside the ground-truth bounding box.
  • 4.4 COMPARING CLASSIFIER ARCHITECTURES: FIDO-CA yields qualitatively different saliency regions across AlexNet, VGG, and ResNet despite correct classification by all three architectures.For one bird image, AlexNet emphasizes the body, whereas VGG and ResNet emphasize head features.
  • 4.5 QUANTITATIVE EVALUATION: FIDO evaluates compactness by successively replacing salient pixels with generative in-fills and measuring how many altered pixels change the normalized classification score.The lower the required number of salient pixels, the better; the quantitative evaluation also includes WSL error and the Saliency Metric on ImageNet.
  • 4.6 ABLATION STUDY: The ablation study indicates that both Bernoulli-mask search and generative in-filling are needed for the strongest explanations.It compares FIDO-CA with BBMP-CA and with FIDO variants using Blur or Random infilling.

5 SCOPE AND LIMITATIONS

FIDO’s explanations depend on modeling the input distribution with an expressive generative model. Its performance is therefore bounded by how well that model captures conditional image densities.

  • 5 SCOPE AND LIMITATIONS: FIDO’s performance is upper-bounded by the generative model’s ability to capture the conditional input density.The method seeks counterfactual inputs close to the natural image manifold because classifier behavior on out-of-distribution inputs is ill-defined.

6 CONCLUSION

The paper concludes that FIDO combines adaptive Bernoulli dropout with strong generative infilling to explain differentiable classifiers. Quantitative comparisons indicate more parsimonious, concentrated, and training-distribution-consistent explanations than existing methods.

  • 6 CONCLUSION: FIDO explains differentiable classifiers by marginalizing over plausible alternative inputs using adaptive Bernoulli dropout and strong generative infilling.The framework highlights contextual information relevant to predictions while preserving label information.
  • 6 CONCLUSION: Quantitative comparisons find that FIDO produces more parsimonious explanations than existing methods.The conclusion describes the resulting saliency maps as concentrated and consistent with the classifier’s training distribution.

A.1 COMPARISONS OF BBMP AND FIDO

The comparison examines how sparsity, upsampling, random initialization, and total-variation regularization affect BBMP and FIDO saliency maps. FIDO with CA in-filling generally concentrates saliency more effectively while reducing artifacts, though overly strong regularization can remove signal.

  • BBMP and FIDO comparison: FIDO with CA in-filling produces fewer artifacts and concentrates saliency on smaller regions than BBMP with Blur or Random in-filling.BBMP Blur tends to produce small patches and pixels, while BBMP Random can produce off-object lines.
  • Sparsity penalty: Low sparsity penalties make all methods prone to artifacts, so selecting λ appropriately is important.The compared methods use different λ values, and BBMP requires roughly five times higher sparsity penalty for visually comparable maps.
  • Upsampling effect: Upsampling regularization removes artifacts, especially for the weaker Mean in-filling strategy.Mean requires stronger regularization than CA to avoid artifacts, although CA can still leave small artifacts outside objects.
  • Stability: The method produces similar saliency maps across four random seeds when using CA in-filling with ResNet and λ = 10^-3.This tests stability under different random initializations.
  • Total variation effect: Total-variation regularization reduces adversarial artifacts but risks losing signal when its penalty is too strong.The regularizer therefore trades artifact suppression against preservation of salient information.

A.5 ANALYSIS OF GENERATIVE MODEL INFILLING

This analysis evaluates generative in-filling and optimization settings for recovering classifier predictions after pixels are removed. Generative models preserve classifier probability better than several baselines, while adequate batch size is important for satisfactory saliency outputs.

  • Generative model infilling: VAE and CA generative in-filling methods consistently achieve higher classifier probabilities than four baseline in-filling strategies.The comparison uses ResNet classification probability on 1,000 validation images after random pixel removal.
  • Regularization: Total-variation regularization can reduce artifacts but may remove signal when applied too strongly.This regularization trade-off is shown in the accompanying analysis.
  • Batch size effects: Batch sizes below 4 produce unsatisfactory saliency maps.The authors attribute this result to high variance in the resulting gradient estimates.

A.7 ABLATION STUDY

The ablation study compares FIDO-CA with BBMP-CA across thresholds using localization and pixel-removal evaluations. FIDO-CA is substantially more effective and less brittle across the tested thresholds.

  • Threshold ablation: BBMP-CA is relatively brittle across different threshold values τ in WSL and SM evaluations.The study evaluates both metrics on a subset of 1,000 images.
  • Pixel-removal evaluation: FIDO-CA substantially outperforms BBMP-CA in the successive pixel-removal experiment across varying thresholds.Pixels are sorted by saliency and successively replaced with CA-GAN in-filled values; lower is better.
  • Threshold ablation: At τ = 0.3, BBMP-CA performs slightly better than BBMP and FIDO with heuristic in-filling but remains substantially inferior to FIDO-CA.Both WSL error and Saliency Metric use lower-is-better comparisons.

B MORE EXAMPLES

Additional examples compare FIDO’s in-filling variants with a prior method and show classifier confidence for counterfactual images. The examples illustrate how FIDO-CA can isolate compact contextual regions, such as net pixels around a fish.

  • Additional examples: FIDO-CA counterfactual examples are compared with the realtime method of Dabkowski and Gal, alongside additional FIDO in-filling variants.Figures 20 and 21 provide further comparisons across in-filling approaches.
  • Confidence comparisons: The examples report classifier confidence p(c|x̂) below the input and each infilled image.The figure uses FIDO-CA with CA-GAN in-filling and includes several counterfactual image comparisons.
  • Contextual information: FIDO-CA is hypothesized to isolate compact contextual information, such as net pixels surrounding a fish.The highlighted net region is described as context information missing from the counterfactual.
  • Saliency-map examples: Additional saliency-map examples include varied in-filling methods and a prior method in the right-most column.The examples are shown under total variation 0.01.
  • Ablation examples: Further ablation examples are provided to complement the quantitative and qualitative comparisons.These examples are presented in Figure 22.
Loading 1807.08024v3…