PaperlayerView exact revision on arXiv

Source-linked AI summary

Post-hoc explanation of black-box classifiers using confident itemsets

Milad Moradi, Matthias Samwald

arXiv:2005.01992v2cs.AIcs.LG

TL;DR

Black-box classifiers can make accurate predictions while concealing their decision logic, and common post-hoc approaches have limitations in how they represent local data patterns. CIE uses confident itemsets to represent decision boundaries in smaller subspaces; the paper reports stronger descriptive accuracy and interpretability than comparison methods, including favorable user evaluations.

  • Problem

    Black-box classifiers conceal their decision logic, while perturbation-based explainers can use unrealistic feature values and decision sets can miss infrequent but discriminative subspace patterns.

    Source: §1 · pp. 3–4

  • Method

    CIE is a model-agnostic post-hoc method that uses confident itemsets associated with class labels to discretize the input space and construct instance- and class-wise explanations.

    Source: §1 · pp. 4–5, §5 · p. 26

  • Results

    CIE mimicked black-box behavior more accurately than other XAI methods, achieved higher descriptive accuracy with fewer decision itemsets and items, and received better user ratings for understandability and interpretability.

    Source: §5 · p. 27

  • Takeaways & Limitations

    CIE’s explanations can represent decision boundaries and expose alternative predictions, uncertainty, and ambiguity in overlapping subspaces.

    Source: §1 · pp. 4–5

    The motivating comparison is scoped to perturbation-based and decision-set explainers, with cited weaknesses involving unrealistic samples and missed infrequent subspace patterns.

    Source: §1 · p. 4

Abstract

from arXiv · show

Black-box Artificial Intelligence (AI) methods, e.g. deep neural networks, have been widely utilized to build predictive models that can extract complex relationships in a dataset and make predictions for new unseen data records. However, it is difficult to trust decisions made by such methods since their inner working and decision logic is hidden from the user. Explainable Artificial Intelligence (XAI) refers to systems that try to explain how a black-box AI model produces its outcomes. Post-hoc XAI methods approximate the behavior of a black-box by extracting relationships between feature values and the predictions. Perturbation-based and decision set methods are among commonly used post-hoc XAI systems. The former explanators rely on random perturbations of data records to build local or global linear models that explain individual predictions or the whole model. The latter explanators use those feature values that appear more frequently to construct a set of decision rules that produces the same outcomes as the target black-box. However, these two classes of XAI methods have some limitations. Random perturbations do not take into account the distribution of feature values in different subspaces, leading to misleading approximations. Decision sets only pay attention to frequent feature values and miss many important correlations between features and class labels that appear less frequently but accurately represent decision boundaries of the model. In this paper, we address the above challenges by proposing an explanation method named Confident Itemsets Explanation (CIE). We introduce confident itemsets, a set of feature values that are highly correlated to a specific class label. CIE utilizes confident itemsets to discretize the whole decision space of a model to smaller subspaces.

1. Introduction

Black-box models can capture complex relationships, but their hidden decision logic makes their predictions difficult to understand. CIE uses confident itemsets to represent decision boundaries and address shortcomings of common post-hoc explainers.

1. Introduction

  • Black-box models can achieve high predictive accuracy on complex or nonlinear data, yet their hidden inner workings make individual predictions difficult to explain.XAI methods aim to help users understand predictions, failures, error patterns, and model or data biases.Source: §1 · p. 3
  • Post-hoc methods approximate a black-box’s learned relationships, while model-based explanations use transparent models suited to less complex data.The paper motivates post-hoc explanations when complex or nonlinear relationships call for intricate black-box models.Source: §1 · p. 3
  • Perturbation-based explainers can approximate behavior using feature values that do not occur in real samples because they ignore local feature distributions and class-label density.The cited example, LIME, perturbs real samples and fits a local simple model to the resulting neighborhood.Source: §1 · p. 4
  • Decision-set methods may sacrifice instance- or class-wise interpretability, omit confidence measures, and discard infrequent values that discriminate within particular subspaces.Their global interpretability objective may also filter overlapping decision sets, despite the potential value of revealing uncertain boundaries.Source: §1 · p. 4
  • CIE uses confident itemsets to represent instance- and class-wise decision boundaries, with reported improvements in descriptive accuracy and interpretability over comparison explainers.The method is model-agnostic and can represent alternative predictions in overlapping subspaces, exposing uncertainty and ambiguity.Source: §1 · pp. 4–5

2. Related work

Related work contrasts transparent models with post-hoc approaches and describes global and local explanation methods. It positions CIE against perturbation-based explanations and frequent-itemset approaches, emphasizing confident itemsets’ ability to capture local patterns.

2. Related work

  • Predictive accuracy metrics alone may be insufficient for judging a model’s usefulness, motivating explanations of its learned relationships, decisions, and feature-space behavior.The cited account also identifies understanding possible biases and collective feature influence as relevant goals.Source: §2 · p. 5
  • Transparent model-based methods include rule-based partitions, prototype clusters, and combinations of local classifiers, but simple models may fit complex nonlinear data poorly.The passage describes a tradeoff between model complexity and explainability.Source: §2 · p. 5
  • Post-hoc explanation reconstructs explanations for black-box decisions; global methods describe overall behavior, whereas local methods relate a record’s features to an outcome.Examples of global approaches include decision trees, prototypes, feature-importance scores, and minimal feature adjustments.Source: §2 · pp. 5–6
  • LIME approximates local behavior with perturbed records but ignores feature-value distributions, whereas CIE uses real dataset values and black-box predictions.The passage characterizes CIE as model-agnostic and applicable to various black-box models.Source: §2 · p. 6
  • Frequent itemsets can miss infrequent patterns that accurately describe small subspaces, a concern for complex relationships such as text classification.CIE instead uses confident itemsets to represent local model behavior in different input-space regions.Source: §2 · p. 6
  • Confident itemsets represent feature–class relationships within small subspaces and can capture associations involving multiple feature values.Compared with decision sets based on frequent itemsets, CIE is reported to produce concise explanations without reducing descriptive accuracy.Source: §2 · p. 7

3. Confident itemsets explanation method

CIE partitions a black-box classifier’s decision space by predicted class and uses confident itemsets to represent local feature–class relationships. It builds instance-wise explanations from matching itemsets and class-wise explanations from selected subsets.

3. Confident itemsets explanation method

  • CIE discretizes the feature space into subspaces containing instances assigned to each black-box-predicted class, then discovers class-correlated itemsets within them.The method uses these smaller subspaces to represent instance- and class-wise decision boundaries.Source: §3.1 · p. 7, §3 · p. 7

3.1. Extracting confident itemsets

  • Tabular records use feature–operator–value triples, while text records use individual words as items.Numerical feature values are assigned to user-defined ranges before representation.Source: §3.1 · p. 8, Figure 1 · p. 9
  • A confident K-itemset meets the class-specific minimum confidence threshold, and every subset of that itemset must also be confident.Mining proceeds iteratively from 1-itemsets, adding larger itemsets until a predefined maximum is reached or an iteration finds none.Source: §3.1 · pp. 8–9

3.2. Instance-wise explanations

  • For an instance, CIE gathers extracted confident itemsets that occur in the record and scores each represented class by the confidence values of its matching itemsets.It assigns the instance the class with the highest score; overall and class-wise support measure itemset frequency and help rank itemsets in explanations.Source: §3.2 · pp. 12–13

3.3. Class-wise explanations

  • A class-wise explanation is a subset of that class’s confident itemsets selected to optimize fidelity, interpretability, and coverage.Fidelity measures how accurately the explanation mimics the black-box’s class assignments.Source: §3.3 · pp. 13–14
  • Class-wise interpretability is quantified using explanation size, total item count, maximum itemset length, and overlap between itemsets.Overlap counts itemset pairs sharing at least one item.Source: §3.3 · pp. 14–15
  • CIE combines six reward functions in an objective whose weights are selected by cross-validation, while end users specify the values of θ1, θ2, and θ3.The stated optimization uses approximate local search for the objective’s submodular, non-monotone, non-negative, and non-normal form.Source: §3.3 · pp. 15–16

4. Experimental results and discussion

Experiments across tabular and text datasets compare CIE with other explanation methods on fidelity, interpretability, and user ratings. CIE generally performs well, with a particularly strong comparison on global explanation size for text data.

4. Experimental results and discussion

  • Experiments covered three tabular and three text datasets and compared CIE with LIME, MUSE, greedy, and random explanation approaches.The tabular datasets were Adult, Compas, and Thyroid Disease; the text datasets were TREC-QC, SST, and SMS-SC.Source: §4 · pp. 17–18

4.1. Fidelity to the black-box’s predictions

  • CIE achieved the highest instance-wise fidelity scores on two tabular datasets and all three text datasets; LIME scored highest on the TD dataset.The authors suggest LIME may benefit from continuous features and uniform random perturbations on TD.Source: Table 1 · p. 19, §4.1 · p. 19, Table 2 · p. 20
  • CIE achieved the highest class-wise fidelity scores on two tabular datasets and all three text datasets, though class-wise explanations were less accurate than instance-wise ones.The authors describe class-wise explanations as useful for distinguishing classes through important feature values and correlations.Source: Table 3 · p. 21, §4.1 · p. 21

4.2. Interpretability

  • At the same number of decision itemsets, CIE generally achieved higher descriptive accuracy than the other methods, except on one tabular dataset.The study treats explanations with fewer decision elements as more interpretable.Source: §4.2 · p. 22, Figure 6 · p. 23
  • CIE reached 76% descriptive accuracy on TREC question classification with 40 decision itemsets, versus 70 for LIME and 80 for MUSE.The authors report that CIE produced more interpretable global explanations; its advantage over other methods was larger on text than tabular datasets.Source: §4.2 · p. 22, Figure 7 · p. 24, Figure 8 · p. 25

4.3. User study

  • Users gave CIE the highest scores for both textual and tabular explanations, citing confidence scores, multi-feature itemsets, compactness, and easier inference about model behavior.The study involved 20 participants who rated explanations on understandability, model inference, size, and task relevance.Source: §4.3 · pp. 25–26, Table 4 · p. 26

5. Conclusion

CIE explains black-box classifiers by discretizing input space into subspaces using confident itemsets, then producing instance-wise and class-wise explanations. Experiments report more accurate and interpretable explanations than comparison methods, with user evaluations also favoring CIE.

5. Conclusion

  • CIE’s fidelity experiments found more accurate instance-wise and class-wise approximations of black-box behavior across text and tabular datasets.The authors report that confident itemset mining efficiently approximates decision boundaries by identifying feature values highly correlated with class labels.Source: §5 · pp. 26–27
  • CIE produces more interpretable global explanations with fewer decision itemsets and items, and users rated its explanations more understandable and interpretable.The user comparison was subjective; the supplied conclusion gives no numeric scores.Source: §5 · p. 27
  • Confident itemsets capture local feature–label relationships across subspaces, while optimizing fidelity, interpretability, and coverage selects minimal class-wise subsets.The conclusion presents these relationships and objectives as the basis for high-quality class-wise explanations.Source: §5 · pp. 26–27
  • CIE addresses limitations of perturbation-based and decision-set explanations by using real feature values, capturing correlations in subspaces, and optimizing fidelity, interpretability, and coverage.The paper also says its confidence measure quantifies correlation strength and can reveal ambiguity and uncertainty in the model’s decision space or data.Source: §5 · p. 27
  • Future work includes adapting CIE to domain-specific tasks with domain knowledge and adding feature modifications that could change a black-box prediction.The authors identify biomedical NLP, decision support, and recommender systems as examples of these directions.Source: §5 · pp. 27–28
Loading 2005.01992v2…