PaperlayerView exact revision on arXiv

Source-linked AI summary

Visualizing the Effects of Predictor Variables in Black Box Supervised Learning Models

Daniel W. Apley, Jingyu Zhu

arXiv:1612.08468v2stat.ME

TL;DR

Black-box models need interpretable predictor-effect visualizations, but predictor dependence can make PD plots extrapolate and M plots biased. The paper introduces ALE plots, which use conditional local effects and accumulation to avoid these problems while reducing computation. ALE also has an important interpretation boundary: its curves should not be read as predictions from varying a correlated predictor across its entire range while holding another fixed.

  • Problem

    Predictor dependence limits existing effect visualizations because PD plots can extrapolate beyond observed combinations, whereas M plots can exhibit omitted-variable bias.

  • Method

    ALE plots accumulate conditional local effects to avoid extrapolation and omitted-variable bias when visualizing main and low-order interaction effects.

  • Results

    ALE plots recover additive predictor effects up to an additive constant and require fewer model evaluations than PD plots.

  • Takeaways & Limitations

    ALE plots provide a computationally efficient alternative for visualizing predictor effects when predictors are dependent, within their supported interpretation.

  • Takeaways & Limitations

    ALE curves should not be interpreted as predictions from varying one highly correlated predictor across its full range while holding another predictor fixed.

Abstract

from arXiv · show

When fitting black box supervised learning models (e.g., complex trees, neural networks, boosted trees, random forests, nearest neighbors, local kernel-weighted methods, etc.), visualizing the main effects of the individual predictor variables and their low-order interaction effects is often important, and partial dependence (PD) plots are the most popular approach for accomplishing this. However, PD plots involve a serious pitfall if the predictor variables are far from independent, which is quite common with large observational data sets. Namely, PD plots require extrapolation of the response at predictor values that are far outside the multivariate envelope of the training data, which can render the PD plots unreliable. Although marginal plots (M plots) do not require such extrapolation, they produce substantially biased and misleading results when the predictors are dependent, analogous to the omitted variable bias in regression. We present a new visualization approach that we term accumulated local effects (ALE) plots, which inherits the desirable characteristics of PD and M plots, without inheriting their preceding shortcomings. Like M plots, ALE plots do not require extrapolation; and like PD plots, they are not biased by the omitted variable phenomenon. Moreover, ALE plots are far less computationally expensive than PD plots.

1. Introduction

Black-box models are difficult to interpret, while predictor dependence makes both PD and M plots problematic: PD plots may extrapolate beyond the training-data envelope, whereas M plots can reflect omitted-variable bias. ALE plots are introduced to avoid both problems by accumulating conditional local effects.

  • Motivation: Black-box supervised models capture nonlinear phenomena but are difficult to interpret in terms of individual predictor effects.Understanding predictor effects is critically important in many predictive-modeling applications.
  • Problems with existing plots: PD plots average predictions over a predictor’s marginal distribution, which can require severe extrapolation when predictors are strongly dependent.In the toy example, the averaging occurs over a vertical line extending far beyond the training-data envelope.
  • Problems with existing plots: M plots avoid extrapolation by using conditional distributions, but when predictors are dependent they can reflect both the target predictor’s effect and a nuisance predictor’s effect.The paper relates this distortion to omitted-variable bias in regression.
  • ALE approach: ALE plots average local effects conditionally and accumulate them over the target predictor, avoiding PD extrapolation while avoiding the omitted-variable bias of M plots.The conditional density keeps averaging near observed predictor combinations; accumulation distinguishes ALE from direct M-plot averaging.
  • ALE approach: The paper defines ALE as a method for assessing both main effects and low-order interaction effects in black-box supervised learning models.The introduction focuses on individual predictors and pairwise interactions, with higher-order effects deferred to later sections and appendices.

2. Definition of ALE Main and Second-Order Effects

The paper defines ALE effects by accumulating conditional local changes for individual predictors and by using two-dimensional local differences for predictor pairs, followed by centering and removal of lower-order effects.

  • Main effects: ALE main effects are defined for general predictors by replacing derivatives and integrals with limiting finite differences and summations over increasingly fine partitions.The definitions accommodate fitted functions that are not necessarily differentiable.
  • Main effects: For differentiable models, the ALE main effect integrates the conditional expectation of the predictor’s partial derivative given that predictor’s value.The conditional averaging is over the remaining predictors at the target predictor value.
  • Centering: Centered ALE main effects subtract their zero-order effect so that the resulting effect has mean zero with respect to the predictor’s marginal distribution.The centering changes vertical location without changing the effect’s shape.
  • Second-order effects: ALE second-order effects use two-dimensional finite differences on the Cartesian-product grid formed by partitions of two predictors.The resulting effect is doubly centered by removing both predictor main effects and then the zero-order effect.
  • Higher-order effects: ALE higher-order effects are constructed analogously by sequentially subtracting all lower-order effects from an uncentered higher-order effect.This produces effects whose lower-order components are identically zero.
  • Scope and assumptions: For nominal categorical predictors, ALE estimation requires choosing an ordering of categories before differencing neighboring categories.The paper discusses a practical ordering strategy in an appendix rather than developing the categorical case fully here.

3. Estimation of fj,ALE(xj) and f{j,l},ALE(xj, xl)

ALE estimators replace population partitions and conditional expectations with fixed sample partitions and within-cell averages, yielding consistent main- and second-order effect estimates with computational advantages over PD.

  • Main-effect estimation: ALE estimation replaces limiting partitions with fixed partitions of the observed sample range and conditional expectations with sample averages among observations in each interval or cell.This construction applies to both single-predictor and pairwise effects.
  • Main-effect estimation: Main-effect estimation partitions a predictor’s sample range into intervals, computes local finite differences within intervals, averages them, and cumulatively sums them across intervals.The estimator is then centered by subtracting an estimate of its expected uncentered effect.
  • Second-order estimation: Second-order estimation partitions two predictor ranges and forms a Cartesian grid of rectangular cells for averaging two-dimensional finite differences.The uncentered estimate is subsequently adjusted by subtracting estimated main effects and a final mean.
  • Illustrations: Figure 3 illustrates five intervals for one predictor and averages finite differences across observations in each interval before accumulation.Its example uses n = 30 observations, K = 5 intervals, and interval counts of 4, 6, 6, 5, and 9.
  • Illustrations: Figure 4 illustrates the pairwise estimator using a 5 × 5 grid formed from two five-interval partitions.Each cell contains observations whose two predictor values fall into the corresponding interval pair.
  • Properties: Under mild conditions, the proposed main- and second-order estimators are consistent for their corresponding ALE effects.The paper presents this result in Theorems 3 and 4.
  • Computational properties: ALE estimators require 2^|J| × n model evaluations for an effect over a grid, compared with K^|J| × n evaluations for PD.Thus ALE is substantially more computationally efficient, while also producing reliable estimates where PD can break down.

4. Toy Examples Illustrating when ALE Plots are Reliable but PD Plots Break Down

Toy examples with strongly correlated predictors show ALE plots recovering the true effects for both tree and neural-network models, while PD and M plots can be substantially misleading.

  • Example 1: The Example 1 predictors lie near x2 = x1, creating a narrow data envelope for the fitted 100-leaf tree.The response is deterministic, Y = X1 + X2, and the tree was pruned to approximately the cross-validation-selected size.
  • Example 1: PD estimates fail because tree predictions are extrapolated into sparsely supported regions outside the correlated training-data envelope.For small x1, PD is too high because of overestimated upper-left predictions; for small x2, it is substantially underestimated because of lower-right extrapolation.
  • Example 1: For both predictors, ALE estimates are much more accurate than PD or M estimates for the tree model.ALE is close to the true linear effect for X1 and quadratic effect for X2, whereas PD and M perform poorly.
  • Example 1: M plots combine the true effects of X1 and X2 under strong correlation, making them poor representations of either individual effect.The authors therefore do not consider M plots in subsequent examples.
  • Example 2: Across 50 neural-network replicates in Example 2, ALE estimates remain close to the true linear and quadratic effects, unlike PD estimates.The neural network uses noisy responses with ε ∼ N(0, 0.1^2) and achieves cross-validation r2 values between 0.965 and 0.975.

5. Discussion

The bike-sharing examples show that ALE plots provide interpretable main and interaction effects, while additive-recovery results clarify what these effects represent. ALE also offers a substantial computational advantage over PD plots, but highly correlated predictors require careful interpretation.

  • Bike-sharing main effects: ALE main-effect plots clearly interpret month, hour, weather situation, wind speed, and feeling temperature effects on hourly bike rentals.Rental counts peak in September–October and at morning and evening rush hours, decline with worsening weather and increasing wind speed, and peak near 26°C feeling temperature.
  • Bike-sharing interactions: The hour–weather interaction spans about 110 rental-count units and shows that worsening weather reduces rentals more strongly during rush-hour peaks.Including the main effects clarifies the joint effect and prevents misleading interpretation of the interaction surface alone.
  • Additive recovery: ALE effects recover additive component functions up to an additive constant and recover k-order interactions up to strictly lower-order functions.When component functions are adjusted to have no lower-order ALE effects, the ALE effect equals the corresponding component function.
  • Dependent predictors: For dependent predictors with strong interactions, second-order ALE effects can recover the interaction while the meaning of individual main effects remains ambiguous.In the example f(x)=x1x2, the ALE interaction is x1x2 − 1/2ρ(x1^2 + x2^2), whereas the ALE main effects are nonzero and the PD main effects are zero.
  • Computational cost: ALE computation requires O(n) model evaluations versus O(n^2) for PD when grid fineness grows with sample size.The ALE evaluation count does not depend on the number of grid locations K.
  • Effect decomposition: ALE operators have an orthogonality-like property: applying the same operator preserves an effect, while applying a different-order operator yields zero.Consequently, each ALE interaction effect has identically zero lower-order ALE effects, and vice versa.

6. Conclusions

ALE plots address key weaknesses of PD plots by avoiding extrapolation under correlated predictors and requiring substantially fewer model evaluations.

  • ALE plots avoid extrapolation that can make PD plots unreliable when predictors are highly correlated.
  • ALE plots require 2^|J| × n model evaluations for each effect, compared with K^|J| × n for PD plots.
  • The authors suggest adopting ALE plots as a standard visualization component in supervised learning software.

Appendix A Statements and Proofs of Theorems A.1 and 1 - 4

The appendix states conditions under which ALE main- and second-order effects and their estimators are well defined and consistent. It establishes convergence results for increasingly fine partitions and suitable sample sizes.

  • Theorem A.1: Theorem A.1 gives sufficient conditions for the ALE limit to exist independently of the partition sequence.
  • Theorem 1: Under differentiability, continuity, and related regularity assumptions, the uncentered ALE main effect is represented by an integrated conditional expected derivative.
  • Theorem 2 and Theorem 4: Under analogous regularity conditions, the second-order ALE effect and its estimator satisfy corresponding existence and consistency results.
  • Theorem 3: The ALE main-effect estimator is strongly consistent for each fixed partition resolution under the stated assumptions.
  • Theorem 3: There are sample-size sequences for which the estimator converges to the ALE main effect in probability and in mean as partition resolution increases.
  • Theorem 4: The second-order estimator converges both in probability and in mean for each pair of predictor values.

Appendix B ALE Plot Definition for Higher-Order Effects

The appendix generalizes ALE effects to arbitrary predictor subsets using finite differences on partitioned predictor grids and operators that remove lower-order effects. Higher-order definitions support decomposition and additive recovery, although the authors do not expect frequent visualization beyond second order.

  • Higher-order ALE effects are introduced to derive the ALE decomposition theorem and additive recovery properties.
  • The authors do not envision ALE plots being commonly used for third- and higher-order effects.
  • For a predictor subset J, the method partitions the X_J-space into rectangular grid cells and computes |J|-order finite differences across each cell.
  • The operator L_J aggregates these finite differences using conditional expectations over the complementary predictors.
  • The centered ALE effect f_J,ALE is obtained by removing lower-order effects from L_J(f), including the zero-order effect.
  • For first- and second-order subsets, the general operator reduces to the main- and second-order ALE definitions given earlier.

Appendix C ALE Decomposition Theorem and Some Properties of LJ and HJ

The ALE decomposition represents a fitted function as a sum of component functions, each obtained by applying an operator HJ that isolates a corresponding |J|-order effect. These operators satisfy projection-like and orthogonality-like properties, yielding a unique decomposition.

  • The operators LJ and HJ are characterized by idempotence, linearity, annihilation under index mismatch, and related composition properties.These properties underpin the construction and proof of the ALE decomposition theorem.
  • The proof explains that lower-order effects are iteratively subtracted from LJ(f) when forming fJ,ALE.The composite operators remove effects of progressively smaller subsets before subtracting the zero-order effect.
  • The ALE decomposition theorem expresses f(x) as the sum of ALE component functions fJ,ALE(xJ).Each component represents the |J|-order effect of XJ and is constructed as fJ,ALE = HJ(f).
  • HJ leaves its matching ALE component unchanged, while Hu removes it for every u ≠ J.Thus, the component contains its own order-specific effect and no other effects under the corresponding operators.
  • The ALE decomposition is unique among decompositions whose components satisfy the same orthogonality-like operator property.Any alternative component fJ with HJ(fJ) = fJ and Hu(fJ) = 0 for u ≠ J must equal fJ,ALE.

Appendix D Estimation of fJ,ALE(xJ) for Higher-Order Effects

Higher-order ALE effects are estimated by replacing the operators' integrals with finite-difference estimates over a multidimensional grid. The resulting general estimator includes the first- and second-order estimators as special cases.

  • The general estimator f̂J,ALE is computed by estimating the quantities in the composite ALE expression from right to left.Although the formal notation is lengthy, the estimation concept is described as straightforward.
  • LJ(g)(xJ) is estimated by replacing the integrals in its definition with finite differences over grid cells.The finite-difference estimator uses the partition grid and observations outside J.
  • The |J|-dimensional predictor range is partitioned into K^|J| rectangular cells formed from one-dimensional partitions.Each multidimensional cell is the Cartesian product of the corresponding individual predictor intervals.
  • The estimator f̂J,ALE substitutes the finite-difference estimator for every LJ(g)(xJ) term in the composite expression.The main- and second-order estimators in Sections 3 are special cases for |J| = 1 and |J| = 2.

Appendix E Some Implementation Details

The implementation details address categorical-predictor ordering, computational acceleration, and empty cells in second-order ALE grids. Empty-cell substitutions are unavoidable but, outside the convex hull, do not affect ALE values where training data exist.

  • Categorical predictor levels are ordered by dissimilarity among their conditional distributions for the other predictors, using multidimensional scaling.The ordering is designed to reduce function extrapolation between neighboring levels.
  • Figure 13 illustrates how reordering categorical levels can avoid extrapolation by placing levels with similar conditional predictor distributions nearby.The example compares alphabetical ordering with MDS-based ordering for a categorical X1 and numerical X2.
  • ALE predictions can be vectorized by constructing a 2^|J| × n predictor array and calling the model's predict command once.The paper states this is typically orders of magnitude faster than repeatedly calling predict in a loop.
  • Correlated predictors can create empty or sparsely populated cells in the rectangular grid used for second-order interaction ALE plots.Scatterplot bullets or black rectangles can indicate the locations or counts of such cells.
  • Empty cells require substituting an estimated local effect, using a weighted average of the M nearest nonempty cells.The recommended M is min{10, M0.1}, where those cells together contain at least 10 percent of training observations.
  • The substitution value is somewhat arbitrary and can alter the estimated interaction surface in general.For empty cells outside the bivariate training convex hull, Theorem 5 states that the chosen value does not alter the ALE function in nonempty cells where it matters.
  • Theorem 5 analyzes an empty cell outside the convex hull by comparing ALE functions before and after perturbing its substituted finite difference.The proof sketch tracks the resulting changes through uncentered effects, main effects, and centered interaction effects.
Loading 1612.08468v2…