Source-linked AI summary
Inverse Classification for Comparison-based Interpretability in Machine Learning
Thibault Laugel, Marie-Jeanne Lesot, Christophe Marsala, Xavier Renard, Marcin Detyniecki
TL;DR
When classifier and data information are unavailable, explaining a classifier’s prediction is challenging. The paper finds a nearby observation assigned to another class, using sparsity-aware costs and Growing Spheres to identify the change. Two applications illustrate how the approach can reveal classifier behavior, while digit examples also show that classifier-focused explanations need not match human concepts.
Problem
The paper addresses how to explain a classifier’s prediction when neither the classifier nor existing data are available to the explainer.
Source: Explaining by Comparing · p. 2, Principle of the Proposed Approach · p. 3, Finding the Closest Ennemy · p. 3Method
It finds a nearby observation classified differently and represents the explanation as the change from the original observation, using Growing Spheres to search without existing data.
Source: Introduction · p. 1, Principle of the Proposed Approach · p. 3, Finding the Closest Ennemy · p. 3Results
Two applications showed insights into classifier behavior, including locally relevant news-popularity features and strengths and weaknesses of the digit-classification SVM.
Source: Conclusion and Future Works · p. 6Takeaways & Limitations
In the news application, Growing Spheres provided information about features locally relevant to popularity prediction.
Source: Sparsity Evaluation · p. 5, Conclusion and Future Works · p. 6Digit explanations may not resemble recognizable digits, and the method explains the classifier’s decision rather than the reality it approximates.
Source: Applications to Digit Classification · p. 6
Abstract
from arXiv · showhide
In the context of post-hoc interpretability, this paper addresses the task of explaining the prediction of a classifier, considering the case where no information is available, neither on the classifier itself, nor on the processed data (neither the training nor the test data). It proposes an instance-based approach whose principle consists in determining the minimal changes needed to alter a prediction: given a data point whose classification must be explained, the proposed method consists in identifying a close neighbour classified differently, where the closeness definition integrates a sparsity constraint. This principle is implemented using observation generation in the Growing Spheres algorithm. Experimental results on two datasets illustrate the relevance of the proposed approach that can be used to gain knowledge about the classifier.
Introduction
Interpretability is difficult to pin down amid complex high-performing models and disagreement about what counts as an explanation. The paper proposes a post-hoc, comparison-based approach that finds a nearby observation classified differently, using Growing Spheres without requiring existing data.
Introduction
- The paper frames interpretability as difficult because leading models are complex and the meaning of interpretability is subjective and unsettled.Source: Introduction · p. 1
- The approach explains an individual prediction by comparing its observation with the closest possible observation from a different class.It is post-hoc and focuses on a single prediction.Source: Introduction · p. 1
- The motivation is that end users may lack access to both the original classifier and its data, while complex decisions may resist other explanation methods.The paper also notes that examples can support learning.Source: Introduction · p. 1
- Growing Spheres locally explores a classifier’s input space to find its decision boundary and the minimal change needed to alter a prediction.The method uses no existing data beyond the observation being interpreted.Source: Introduction · p. 1
Post-hoc Interpretability
Post-hoc interpretability uses classifier outputs to explain particular predictions, with methods differing in their assumed inputs and explanation forms. Instance-based explanations can be useful when feature-based or other representations are unsuitable, but existing approaches may depend on prior model or data knowledge.
Post-hoc Interpretability
- Post-hoc methods explain classifier behavior around particular observations, generally without explaining the model’s learning process.The discussion organizes approaches by available inputs and forms of explanation.Source: Post-hoc Interpretability · p. 1
Available Inputs
- Some explainers require knowledge of the prediction algorithm, while model-agnostic sensitivity methods examine local classifier responses to perturbations.Baehrens et al. (2010) use Parzen windows to approximate a local gradient.Source: Available Inputs · p. 2
Forms of Explanations
- Explanations aim to represent classifier behavior accurately and be understandable, but complex cases can make those goals conflict.Comparing examples may be easier and more accurate than forcing a potentially incomplete or misleading representation.Source: Forms of Explanations · p. 2
- For predictions near training-data boundaries, Baehrens et al. (2010) report that gradient explanations can point opposite the decision boundary, where comparison with another-class observations may be more useful.Source: Forms of Explanations · p. 2
- Existing instance-based approaches may rely on prior model, training-data, or labeled-instance knowledge; Kabra et al. (2015) identify influential training observations.Source: Forms of Explanations · p. 2
Comparison-based Interpretability
The proposed approach explains a prediction by finding a nearby observation assigned to another class, even when the classifier and existing data are unavailable. It treats the resulting change as local evidence about classifier behavior, not as causal knowledge or a description of reality.
Explaining by Comparing
- When users lack classifier or data knowledge, comparison-based explanations can use generated examples rather than relying on prior information.The paper also motivates examples as a way to support learning.Source: Explaining by Comparing · p. 2
Principle of the Proposed Approach
- The explanation task asks for the minimal change to an observation that changes its predicted class, applying inverse classification to interpretability.Source: Principle of the Proposed Approach · p. 3
- The approach offers local insight into what a classifier treats as important but does not claim causal knowledge or characterize the model’s global behavior.Generated examples may be unrealistic; the goal is to understand the classifier regardless of whether its prediction is right.Source: Principle of the Proposed Approach · p. 3
Finding the Closest Ennemy
- The explanation is the difference vector e − x between the observation and a different-class observation, with “ally” and “ennemy” denoting same-class and other-class observations.The formalization assumes binary classification for simplicity but can also apply to multiclass classification.Source: Finding the Closest Ennemy · p. 3
- The cost combines Euclidean proximity with a sparsity term so the explanation can favor small changes affecting fewer features.The l0 norm measures sparsity and is combined with the l2 norm as a weighted average.Source: Finding the Closest Ennemy · p. 3
- Because the cost is discontinuous and the setting assumes a black-box classifier with no existing data, Growing Spheres sequentially approximates the minimization with a two-step heuristic.Source: Finding the Closest Ennemy · p. 3
Growing Spheres
Growing Spheres searches around an observation for a nearby example classified differently, then sparsifies the difference to form an explanation. Its generation and feature-selection stages respectively seek a close opposing-class observation and reduce the features changed.
Growing Spheres
- Growing Spheres generates instances in all directions until crossing the decision boundary, then sparsifies the difference to produce the final solution e∗.Figure 1 illustrates the two steps, with the white plus marking the final enemy used to generate explanations.Source: Growing Spheres · p. 3, Figure 1 · p. 4
Generation
- Generation samples uniformly within an initial l2-ball, shrinks its radius if enemies appear, then expands through spherical layers until finding one.The algorithm returns the l2-closest generated enemy to x.Source: Generation · p. 4
Feature Selection
- Feature selection seeks a sparser difference by removing the smallest coordinate changes first; the final explanation is the vector x−e∗.This heuristic treats the smallest changes as less locally relevant to the classifier boundary.Source: Feature Selection · p. 4
Experiments
The experiments illustrate Growing Spheres on news popularity and digit classification, using local explanations to inspect classifier behavior and explanation sparsity. The examples show both interpretable prediction changes and cases where classifier-boundary examples do not match human-recognizable digits.
Experiments
- The news experiment evaluates illustrative explanations as a way to gain insight into popularity prediction and checks their sparsity as a proxy for readability.It uses 58 features from 39,644 articles and reports a 0.70 final AUC score for the random forest.Source: Experiments · p. 5, Application for News Popularity… · p. 5
Application for News Popularity Prediction
- For the not-popular article, two changed coordinates concern reference and keyword popularity: the least-popular referenced article needs 2,016 more shares, and the most-popular articles using each keyword need 913 more.With those changes, the article would be predicted popular by the random forest.Source: Illustrative Example · p. 5
- For a popular article, reducing the same features and lowering text subjectivity by 0.03 would change the prediction to not popular.The paper describes the subjectivity change as a slightly more objective point of view.Source: Illustrative Example · p. 5
- 80% of news test observations need changes in 9 directions or fewer, and none need more than 17; explanations may use different features across observations.The authors report a maximum of 17 changed coordinates and caution that this does not imply a shared set of 17 features explains every observation.Source: Sparsity Evaluation · p. 5
Applications to Digit Classification
- For digit 8/9 examples, many changed pixels lie at the bottom left, where the closest opposite-class examples open or close the digits’ bottom portions.Some influential pixels are harder to interpret and could indicate an inaccurate learned boundary; the closest examples are not proper 8s or 9s, and the classifier fails to capture the concepts that make digits recognizable to humans.Source: Applications to Digit Classification · p. 6
- Across 3,528 digit-test instances, every explanation uses at most 62 features, or 7.9% of the 784 total features.The experiment uses a binary classifier for digits 8 and 9.Source: Applications to Digit Classification · pp. 5–6
Conclusion and Future Works
The paper uses sparse, comparison-based post-hoc explanations to learn about classifiers, and reports findings from news-popularity and digit-classification applications. It identifies constraints on usable features and input regions as future work.
Conclusion and Future Works
- Growing Spheres compares an observation with its closest differently classified neighbour, using a sparsity-aware cost without requiring classifier or existing-data information.The method aims to explain a single prediction and identify the minimal change needed to alter it.Source: Conclusion and Future Works · p. 6
- The news application found locally relevant features for popularity prediction, while the digit application revealed strengths and weaknesses of the support vector machine.The digit application illustrated concepts learned by the classifier.Source: Conclusion and Future Works · p. 6
- The authors checked that the proposed explanations are sparse.Source: Conclusion and Future Works · p. 6
- Future work could constrain input regions or features so explanations use features an end-user can leverage.The paper gives a business analyst seeking actionable purchase-prediction explanations as an example.Source: Conclusion and Future Works · p. 6