PaperlayerView exact revision on arXiv

Source-linked AI summary

Interpreting random forest classification models using a feature contribution method

Anna Palczewska, Jan Palczewski, Richard Marchese Robinson, Daniel Neagu

arXiv:1312.1121v1cs.LG

TL;DR

Random forests hide how features relate to predictions, and common importance measures can be biased or insufficient. This paper extends instance-level feature contributions to random-forest classification and uses contribution patterns to interpret predictions and assess reliability. The proposed methodology is reported as robust on UCI benchmark datasets.

  • Problem

    Random forests hide feature–prediction relationships, while common importance measures can be biased toward continuous or many-category variables and may not fully explain those relationships.

    Source: §1 · p. 2

  • Method

    The paper extends instance-level feature contributions to random-forest classification and uses median, cluster, and log-likelihood analyses to find class-specific patterns.

    Source: §1 · p. 3

  • Results

    The authors report that the methodology was robust on UCI benchmark datasets and demonstrate its use for interpreting predicted classifications and assessing prediction reliability.

    Source: §7 · p. 28

  • Takeaways & Limitations

    For a new instance, tree-vote fraction and feature-contribution cluster membership can inform reliability assessment; core-cluster proximity and high decisiveness suggest the prediction should be trusted.

    Source: §5.2 · p. 13

    Feature contributions retrieve forest predictions only when the unanimity condition holds; otherwise, they support interpretation only.

    Source: §3 · p. 6

Abstract

from arXiv · show

Model interpretation is one of the key aspects of the model evaluation process. The explanation of the relationship between model variables and outputs is relatively easy for statistical models, such as linear regressions, thanks to the availability of model parameters and their statistical significance. For "black box" models, such as random forest, this information is hidden inside the model structure. This work presents an approach for computing feature contributions for random forest classification models. It allows for the determination of the influence of each variable on the model prediction for an individual instance. By analysing feature contributions for a training dataset, the most significant variables can be determined and their typical contribution towards predictions made for individual classes, i.e., class-specific feature contribution "patterns", are discovered. These patterns represent a standard behaviour of the model and allow for an additional assessment of the model reliability for a new data. Interpretation of feature contributions for two UCI benchmark datasets shows the potential of the proposed methodology. The robustness of results is demonstrated through an extensive analysis of feature contributions calculated for a large number of generated random forest models.

1 Introduction

The introduction motivates interpreting random forests by the need to understand how features relate to predictions, beyond aggregate importance measures. It extends instance-level feature contributions to classification and proposes class-specific patterns to aid interpretation and reliability assessment.

1 Introduction

  • Random forests are black-box models, and understanding how features influence individual predictions matters for evaluating model outcomes and supporting decisions.The introduction frames interpretation as a way to examine relationships between predictions and training data, and to identify which features contribute to a specific classification.Source: §1 · p. 2
  • Common random-forest importance measures can favor continuous or many-category variables, and their general representation may not fully explain input–prediction relationships.Source: §1 · p. 2
  • Prior feature contributions measure each variable’s positive or negative influence on an individual prediction, but the cited method applies to numerical outcomes.The approach was positively tested on a chemical-compound QSAR model.Source: §1 · pp. 2–3
  • This paper extends feature contributions to categorical random-forest predictions and develops median, cluster, and log-likelihood techniques to find class-specific contribution patterns.The patterns support interpretation of predictions and more detailed analysis of reliability for unseen data.Source: §1 · p. 3

2 Random Forest

A random forest combines tree predictors built from randomized training samples and feature choices. Its class prediction is based on tree votes, which can also be represented as a distribution over classes.

2 Random Forest

  • Each tree uses a bootstrap sample as its local training set, with the remaining training instances forming an out-of-bag set for estimating goodness-of-fit.Source: §2 · p. 3
  • Trees are grown by CART splits using a variable from a randomly selected subset, are not pruned, and use random quantities assumed independent and identically distributed across trees.Source: §2 · pp. 3–4
  • For prediction, each tree votes for a class; the forest returns the class with the most votes, resolving ties randomly.Source: §2 · p. 4
  • The forest’s tree predictions can be represented as a probability distribution over classes, with the predicted class given by the largest coordinate.The forest prediction is obtained by averaging tree-level class indicators across all trees.Source: §2 · p. 4

3 Feature Contributions for Binary Classifiers

For binary classifiers, feature contributions sum changes in class-1 probability along an instance’s paths through the trees. Under unanimity condition (U), they recover the forest prediction; otherwise, they support interpretation only.

3 Feature Contributions for Binary Classifiers

  • Each feature’s local increment is its change in class-1 probability from parent to child when that feature splits the parent, and forest contributions sum increments along tree paths.The resulting vector contains a contribution for each feature.Source: §3 · p. 5
  • Feature contributions recover the forest prediction only when condition (U) holds; otherwise, they support model interpretation without retrieving the prediction.The condition requires unanimity across trees’ terminal-node training instances.Source: §3 · p. 6

3.1 Example

  • For x1 and x3, f2 raises class-1 probability by 0.125, but f3’s large negative contribution is decisive for assigning class 0.In the toy Iris forest, f1 and f4 contribute zero for x1 because the trees do not use them.Source: §3.1 · p. 7
  • For x6, x7, and x9, the toy forest’s decision depends only on f3.Source: §3.1 · p. 7
  • For x8 and x10, the forest’s prediction is 0.5 because one tree votes for each class, leaving the evidence for the two classes counterbalanced.Source: §3.1 · p. 8

4 Feature Contributions for General Classifiers

The method extends feature contributions to classifiers with multiple classes by using class-probability vectors at tree nodes. For each instance, it attributes changes in the predicted class’s probability along its paths through the forest to the features used at those splits.

  • For K > 2, node predictions are represented by vectors whose coordinates correspond to classes, rather than a single scalar.A coordinate gives the probability of a class among the training instances in that node.Source: §4 · pp. 8–9
  • For an instance, the method focuses on the coordinate for the class predicted by the forest to explain the prediction’s feature contributions.The algorithm first identifies the predicted class and initializes a contribution vector indexed by features.Source: §4 · p. 9
  • As the instance traverses each tree, the contribution for a split feature is updated by the change in the predicted class’s node probability between parent and child.The resulting feature contributions are averaged across the forest’s trees.Source: §4 · p. 10

5 Analysis of Feature Contributions

The section proposes three ways to find patterns in feature contributions and use them to assess prediction reliability. These approaches summarize typical class contributions with medians, identify contribution-based clusters, and compare instances with clusters while accounting for within-cluster variation.

5.1 Median

  • Class-specific medians of contributions from correctly classified training instances provide a standard level for comparing new predictions with typical model behaviour.A high vote fraction combined with contributions close to the class median supports relying on a prediction; distant training examples may indicate multiple standard levels.Source: §5.1 · pp. 11–12

5.2 Cluster Analysis

  • Contribution-based clusters can reveal multiple standard patterns within a class; in examined examples, clustering feature values did not yield useful results.Treat information criteria or the Elbow method as an upper guide to cluster count, then inspect cluster size, voting decisiveness, and distance from the centre. For a new instance, a nearby core cluster and high vote fraction together support trusting its prediction; otherwise, misclassification risk may be increased.Source: §5.2 · pp. 12–13

5.3 Log-likelihood

  • The log-likelihood approach weights contribution deviations by within-cluster variability, addressing the limitation that Euclidean distance treats such deviations alike.It models feature contributions as base values plus normally distributed perturbations, with feature-specific sample variances and an assumption of independence between features; equivalently, the log-likelihood ranks instances by negative squared weighted Euclidean distance.Source: §5.3 · pp. 14–15

6 Applications

The applications use the BCW and Iris datasets to demonstrate feature-contribution analysis, cluster-based interpretation, and robustness assessment. Across these examples, contributions reveal both common class patterns and instance-specific evidence.

6 Applications

  • A covariance matrix of feature contributions has F(F + 1)/2 distinct entries, often more than a cluster’s size, limiting useful recovery of dependence structure.More advanced methods such as principal component analysis are left for future research.Source: §6 · p. 15

6.1 Breast Cancer Wisconsin Dataset

  • BCW medians identify F4, F7, F14, F23 and F28 as significant, in agreement with permutation and Gini importance results.The approach also exposes why the forest made a particular instance’s prediction.Source: §6.1 · p. 16, Figure 4 · p. 17, Figure 5 · p. 17
  • Instances 194 and 537 show atypical contributions for F23, F4 and F14, while F2, F22 and F27 help classify them correctly despite moderate global importance.This illustrates how features of limited general importance can matter for specific predictions.Source: Figure 6 · p. 18, §6.1 · p. 19

6.2 Cluster Analysis and Log-likelihood

  • BCW class-specific core clusters have more decisive forest votes and lower contribution variability than the smaller clusters, whose features show greater variability.The largest cluster for each class is identified as its core cluster.Source: §6.2 · pp. 19–20, Figure 7 · p. 21, Figure 8 · p. 22
  • BCW test instances’ log-likelihoods for the two class cores separate the classes well, but likelihoods are comparable only within a single cluster.Figure 9 plots class-0 instances as circles and class-1 instances as triangles.Source: §6.2 · pp. 20, 23, Figure 9 · p. 23

6.3 Iris Dataset

  • For Iris, the model correctly classified 47 of 50 test instances; feature 3’s unusually low contribution flagged a misclassified instance despite 88% of trees voting for its predicted class.Another instance was correctly classified with 86% of trees voting for its predicted class; its contributions were more typical.Source: §6.3 · pp. 23–25, Table 5 · p. 24
  • Iris feature-contribution clusters group instances of the same class, consistent with coherent patterns in the model’s reasoning.The analysis uses two clusters per class, selecting the larger cluster as the core cluster.Source: §6.3 · pp. 25–26, Figure 11 · p. 25

6.4 Robustness Analysis

  • Across 100 BCW random forests, training-set median contributions varied moderately, while test-set contributions varied slightly more but remained qualitatively similar.The average model accuracy was 0.963, and the authors report reliable contribution information for unseen instances.Source: §6.4 · pp. 26, 28, Figure 12 · p. 27

7 Conclusions

The paper extends feature contributions to random forest classification and proposes three techniques for finding patterns in how features are used. UCI benchmark analyses demonstrate the methodology’s robustness and its use in understanding predictions and assessing their reliability.

7 Conclusions

  • The paper extends feature contributions to random forest classification and proposes median, cluster analysis, and log-likelihood techniques for finding feature-use patterns.Source: §7 · p. 28
  • UCI benchmark analyses show the methodology’s robustness and demonstrate its use in understanding dependence between instance characteristics and predicted classifications.Source: §7 · p. 28
  • Feature contributions can also be used to assess prediction reliability, and the paper investigates their relation to standard variable-importance measures.Source: §7 · p. 28
Loading 1312.1121v1…