PaperlayerView exact revision on arXiv

Source-linked AI summary

Explanation by Progressive Exaggeration

Sumedha Singla, Brian Pollack, Junxiang Chen, Kayhan Batmanghelich

arXiv:1911.00483v3cs.LGcs.AIcs.CV

TL;DR

Feature-importance approaches may not explain why a classifier makes a prediction, and local perturbations may yield implausible inputs. This paper generates gradual, plausible counterfactual variations to shift classifier outputs; reported evaluations include realistic-looking samples and medical-data outcomes, with bias detection as an additional use.

  • Problem

    Feature-importance methods such as saliency maps do not explain how and why image regions matter to a prediction, while local perturbations may not produce plausible inputs.

    Source: §1 · p. 1

  • Method

    The method generates plausible progressive variations of a query, using the predictor’s output and input gradient to traverse its decision boundary.

    Source: §1 · p. 2, §2 · pp. 2–3

  • Results

    The paper reports high-quality explanations that preserve query identity, align with target-disease definitions without explicitly using them, and can identify plausible classifier confounding biases.

    Source: §5 · pp. 12–13

  • Takeaways & Limitations

    The method can provide a visual explanation as a gradual tuning knob and can also be used to detect bias in a classifier.

    Source: §1 · pp. 1–2, §5 · p. 13

    The method assumes a differentiable classifier and access to its output and input gradient.

    Source: §2 · p. 2

Abstract

from arXiv · show

As machine learning methods see greater adoption and implementation in high stakes applications such as medical image diagnosis, the need for model interpretability and explanation has become more critical. Classical approaches that assess feature importance (e.g. saliency maps) do not explain how and why a particular region of an image is relevant to the prediction. We propose a method that explains the outcome of a classification black-box by gradually exaggerating the semantic effect of a given class. Given a query input to a classifier, our method produces a progressive set of plausible variations of that query, which gradually changes the posterior probability from its original class to its negation. These counter-factually generated samples preserve features unrelated to the classification decision, such that a user can employ our method as a "tuning knob" to traverse a data manifold while crossing the decision boundary. Our method is model agnostic and only requires the output value and gradient of the predictor with respect to its input.

1 INTRODUCTION

The paper motivates explanations that show how a classifier’s decision changes, not just which features are important. It proposes plausible, progressive counterfactual variations that let users traverse a decision boundary.

1 INTRODUCTION

  • Feature-importance heatmaps may be uninformative when disease effects span large image regions and do not explain why a predictor returns an outcome.Local perturbations can also fail to produce realistic or plausible inputs, reducing their usefulness to end users such as physicians.Source: §1 · pp. 1–2
  • The proposed local explanation method generates plausible, progressive query variations that change the black box’s output and traverse its decision boundary.It is model agnostic and requires predictor values and input gradients; the paper evaluates it on benchmarks and real medical imaging data.Source: §1 · p. 2
  • The method’s counterfactual samples are reported as realistic-looking, externally evaluated on a real medical application, and useful for detecting predictor-training bias.The paper also describes explanations as a tuning knob for progressively transforming inputs across a decision boundary.Source: §1 · pp. 1–2

2 METHOD

The method generates plausible, gradual perturbations that shift a black-box classifier’s posterior while traversing the data manifold. Its design combines conditional generation with compatibility and self-consistency constraints.

2 METHOD

  • The explainer If(x, δ) generates a perturbed input xδ whose classifier output should differ from f(x) by the requested δ.The method assumes a differentiable predictor and access to its output and input gradient.Source: §2 · pp. 2–3

2.1 DATA CONSISTENCY

  • A GAN framework is used to model the data distribution, while the conditional generator produces perturbations intended to remain plausible and realistic.The method uses the encoder output rather than random noise as the generator input.Source: §2.1 · p. 3
  • A fixed step size discretizes the classifier’s posterior range into bins, and the query’s encoded representation and target bin condition the generator.The encoder maps data-manifold inputs to a low-dimensional embedding; the generator maps the embedding and step count back to the data manifold.Source: §2.1 · p. 3, §2 · p. 3

2.2 COMPATIBILITY WITH THE BLACK BOX

  • The compatibility objective combines a conditional-ratio term with KL divergence to encourage generated inputs to yield the desired classifier output.The condition is ordered with perturbation size, and the method adapts the discriminator term for ordinal multi-class regression using binary classification terms.Source: §2.2 · p. 4

2.3 SELF CONSISTENCY

  • Reconstruction and cycle-consistency losses enforce identity at zero perturbation and reversibility under the opposite perturbation.The reverse step maps a perturbed input from its current bin back to the original bin.Source: §2.3 · p. 4

2.4 OBJECTIVE FUNCTIONS

  • The overall objective balances the conditional GAN, black-box compatibility, and reconstruction losses with hyperparameters.The conditional GAN uses a hinge-version adversarial loss.Source: §2.4 · p. 4

3 RELATED WORK

The related-work section contrasts pixel- or patch-level perturbations and saliency maps with generative explanations. The paper positions its approach as a gradual, data-consistent counterfactual method evaluated on medical and curated vision data.

Perturbation-based methods

  • Pixel- and patch-level perturbations may produce unnatural images, whereas this method enforces consistency with unperturbed data and is not restricted to imaging.Prior approaches include patch removal, occlusion, optimal perturbation, and counterfactual patch replacement.Source: §3 · pp. 4–5

Saliency map-based methods

  • Saliency methods highlight relevant image regions, but may require intermediate layers, be vulnerable to adversarial attacks, or highlight large areas when class effects are distributed.The paper notes that broad highlighting can reduce interpretive usefulness in radiology images.Source: §3 · p. 5

Generative explanation-based methods

  • Generative explanation methods produce visual explanations; this approach distinguishes itself through gradual class-effect changes and consistency criteria for feasible perturbations.The paper reports evaluation on real medical imaging applications as well as curated computer-vision datasets.Source: §3 · p. 5

4 EXPERIMENTS

The experiments evaluate the explainer’s criteria across CelebA and CheXpert, including medical counterfactuals, bias detection, class discrimination, saliency maps, and human evaluations.

4 EXPERIMENTS

  • The experiments cover CelebA face attributes and CheXpert chest X-rays, with Cardiomegaly used as the medical target class.The reported datasets contain 200K celebrity face images and 224K chest X-rays from 65K patients.Source: §4 · p. 5

4.1 EVALUATING THE CRITERIA OF THE EXPLAINER

  • The classifier’s actual responses generally track the requested perturbations, nearly perfectly for Young/Old but less closely for Smiling and Cardiomegaly.The plots also show generated perturbations across the full classification range [0, 1].Source: §4.1 · pp. 6–7, Figure 3 · p. 7
  • The explainer significantly outperforms xGEM on image quality, producing crisper, more realistic-looking explanations across three prediction tasks.The reported comparison uses FID for the top and bottom 10% of the decision interval.Source: §4.1 · p. 6, Table 1 · p. 7

4.2 COUNTERFACTUAL EVALUATION ON MEDICAL DATA

  • Generated explanations capture Cardiomegaly-related heart-size changes: paired distributions differ significantly, while healthy and abnormal populations show no confidently detectable difference in average distributions.The paired-sample test reports p-value ≪ 0.001; the independent comparison reports p-value < 0.01, with the authors stating they cannot reject identical average distributions with high confidence.Source: §4.2 · p. 8

4.3 SALIENCY MAP

  • The method’s derived saliency maps are localized and comparable to popular gradient-based methods, and significantly better than random maps under pixel-perturbation evaluation.All methods showed declining classifier accuracy as the fraction of perturbed relevant pixels increased; the authors did not optimize their method for the best saliency map.Source: §4.3 · p. 9, Figure 5 · p. 9

4.4 BIAS DETECTION

  • For a classifier trained on gender-confounded smiling data, explanations often change gender as smile increases, unlike explanations from the classifier trained on gender-balanced data.The comparison uses a proxy oracle classifier to identify gender changes.Source: §4.4 · pp. 9–10, Figure 6 · p. 10

4.5 EVALUATING CLASS DISCRIMINATION

  • The multi-label evaluation measures how often generated explanations change non-target attributes, with rows representing target attributes and columns representing source attributes.The supplied passages describe the evaluation and figure encoding but do not report its results.Source: §4.5 · pp. 10–11, Figure 7 · p. 11

4.6 HUMAN EVALUATION

  • Human annotators generally identified the target class and successfully ordered explanations, though subtle differences made the hard ordering pairs more difficult.“Bangs” was the most difficult target to identify; annotators selected “Smile and Gender” 12.5% of the time for the biased-classifier explanations.Source: §4.6 · p. 11

5 CONCLUSION

The method explains black-box decisions through natural-looking, gradual query-image perturbations that change classifier output. It also supports model-agnostic saliency maps and identification of plausible confounding biases.

5 CONCLUSION

  • The method produces natural-looking, gradual query-image perturbations that change the black-box classifier’s output.The paper reports evaluation on two different datasets, including medical imaging, and says explanations preserve query identity and align with the target disease definition without explicitly using that information.Source: §5 · pp. 12–13
  • The approach can generate saliency maps in a model-agnostic setting.Source: §5 · p. 13
  • The method can identify plausible confounding biases in a classifier.Source: §5 · p. 13

A.1 IMPLEMENTATION DETAILS

The generator and discriminator use an architecture adapted from Miyato & Koyama (2018), with conditional information supplied to the generator’s residual blocks.

A.1 IMPLEMENTATION DETAILS

  • The generator and discriminator architecture is adapted from Miyato & Koyama (2018).Source: §A.1 · p. 15
  • The generator receives the image encoding and applies the condition through conditional batch normalization in each of five residual blocks.Each block uses BN-ReLU-Conv3-BN-ReLU-Conv3; the encoder shares the structure but downsamples the image.Source: §A.1 · p. 15
  • The discriminator comprises five residual blocks, each using ReLU-Conv3-ReLU-Conv3.Source: §A.1 · p. 15

A.2 XGEM IMPLEMENTATION

The xGEM baseline is implemented by extending a pretrained VAE to generate 128×128 face images and incorporate classifier-based label-flipping loss.

A.2 XGEM IMPLEMENTATION

  • The VAE-based xGEM implementation was extended from 64×64 to 128×128 image output.The VAE was trained to generate face images, using default settings and architectures before extension.Source: §A.2 · p. 16
  • The pretrained VAE was extended with cross-entropy loss to flip the query image’s label through the classifier.Source: §A.2 · p. 16
  • The paper presents qualitative comparisons between its explanations and xGEM’s in Figure 9.Source: §A.2 · p. 16

A.3 EXTENDED RESULTS FOR EVALUATING THE CRITERIA OF THE EXPLAINER

The appendix extends CelebA evaluation to four additional prediction tasks and assesses explanation quality and consistency across six tasks.

A.3 EXTENDED RESULTS FOR EVALUATING THE CRITERIA OF THE EXPLAINER

  • The extended CelebA evaluation covers six prediction tasks, including beard, makeup, black hair, and bangs attributes.The four additional tasks are no-beard or beard, heavy makeup or light makeup, black hair or not, and bangs or no-bangs.Source: §A.3 · p. 16, Table 5 · p. 16
  • The extended results include qualitative examples and evaluate compatibility with the black-box classifier.Figure 10 presents qualitative results, while Figure 11 evaluates compatibility.Source: §A.3 · p. 16
  • The six-task results also assess data consistency and self-consistency using FID, LSC, and FVA measures.Lower FID indicates better image quality; higher LSC and FVA are better.Source: §A.3 · p. 16, Table 5 · p. 16

A.4 ABLATION STUDY

The ablation study evaluates three loss components against data consistency, compatibility with the classifier, and self-consistency. It also presents generated explanations for CelebA tasks and reports a positive expected-versus-actual classifier-output trend.

A.4 ABLATION STUDY

  • The cGAN adversarial, KL, and reconstruction losses respectively enforce data consistency, compatibility with f, and self-consistency.Source: §A.4 · p. 16
  • The ablation varies one hyper-parameter at a time, holding λcGAN = 1, λf = 1, and λrec = 100, for young-versus-old prediction on CelebA.Source: §A.4 · p. 16
  • Data consistency is assessed with FID, compatibility by comparing f(x)+δ with f(xδ), and self-consistency using LSC and face verification accuracy.FID is evaluated for high-confidence target-label presence, target-label absence, and overall; the compatibility ablation is shown in Figure 12.Source: §A.4 · p. 16, Figure 12 · p. 20
  • Figures 9 and 10 show generated visual explanations for three and six CelebA prediction tasks, respectively.Figure 10 also displays classifier outputs above each image.Source: Figure 9 · p. 18, Figure 10 · p. 19
  • The expected classifier output f(x)+δ and actual output f(xδ) show a monotonically increasing trend, indicating consistency with the expected condition.Source: Figure 11 · p. 20
Loading 1911.00483v3…