PaperlayerView exact revision on arXiv

Source-linked AI summary

Relating the Partial Dependence Plot and Permutation Feature Importance to the Data Generating Process

Christoph Molnar, Timo Freiesleben, Gunnar König, Giuseppe Casalicchio, Marvin N. Wright, Bernd Bischl

arXiv:2109.01433v1stat.MLcs.LG

TL;DR

PD and PFI describe model predictions but lack DGP-grounded estimands. The paper defines such estimands and develops learner-level estimates and uncertainty methods based on model refits. It identifies bias, model variance, and Monte Carlo integration as sources of deviation from ground truth.

  • Problem

    PD and PFI describe model predictions but lack a theory connecting them to the DGP or ground-truth estimands they should retrieve.

    Source: §1 · p. 2

  • Method

    The paper defines DGP-grounded PD and PFI estimands and proposes learner-level estimates, variance estimators, and confidence intervals using model refits.

    Source: §1 · p. 3, §4 · p. 7

  • Results

    PD and PFI estimates can deviate from their DGP ground truths through bias, model variance, and Monte Carlo integration error.

    Source: §5 · p. 8

  • Takeaways & Limitations

    Learner-PD and learner-PFI use model refits to account for model variance, with a correction proposed for variance underestimation from overlapping resampled data.

    Source: §1 · p. 3

    PFI estimation assumes a per-instance-computable loss, excluding AUC, and requires unseen data for unbiased loss estimates.

    Source: §3.2 · p. 6

Abstract

from arXiv · show

Scientists and practitioners increasingly rely on machine learning to model data and draw conclusions. Compared to statistical modeling approaches, machine learning makes fewer explicit assumptions about data structures, such as linearity. However, their model parameters usually cannot be easily related to the data generating process. To learn about the modeled relationships, partial dependence (PD) plots and permutation feature importance (PFI) are often used as interpretation methods. However, PD and PFI lack a theory that relates them to the data generating process. We formalize PD and PFI as statistical estimators of ground truth estimands rooted in the data generating process. We show that PD and PFI estimates deviate from this ground truth due to statistical biases, model variance and Monte Carlo approximation errors. To account for model variance in PD and PFI estimation, we propose the learner-PD and the learner-PFI based on model refits, and propose corrected variance and confidence interval estimators.

1 Introduction

Machine-learning models can capture nonlinear relationships but their parameters often do not map readily to properties of the data-generating process (DGP). Although PD and PFI describe model behavior, they lack DGP-grounded estimands and uncertainty quantification; the paper frames them as statistical estimators and develops variance-aware inference.

1 Introduction

  • Machine-learning models can address nonlinear relationships, but their parameters often lack a mapping to DGP properties sought in scientific applications.In these applications, the model is a means to better understand the DGP, particularly how the conditional expectation of targets varies with features.Source: §1 · p. 2
  • PD plots and PFI describe model predictions but lack a theory connecting them to the DGP or a ground-truth estimand they should retrieve.Inference also requires uncertainty quantification, while many machine-learning approaches do not provide variance estimates for predictions or parameters.Source: §1 · pp. 2–3
  • The paper treats PD and PFI as estimators of DGP-grounded truths and separates fixed-model interpretation from learner-level interpretation.It proposes learner-PD and learner-PFI, variance estimators, and confidence intervals that account for model variance through model refits.Source: §1 · p. 3
  • Resampling-based refits can underestimate variance when training samples overlap, producing confidence intervals that are too narrow.The paper applies a variance-correction approach and analyzes interval coverage with and without correction.Source: §1 · p. 3

2 Related Work

Prior work has proposed confidence intervals and variance estimates for PD and PFI, but the approaches described here leave gaps in uncertainty accounting or grounding in DGP estimands. This paper adds a correction for variance underestimation from resampling and relates its estimators to a proposed ground-truth PFI.

2 Related Work

  • Existing PD confidence-band approaches may quantify only Monte Carlo error or omit correction for variance underestimation when accounting for model variance.The passage presents this gap as motivating a theoretical underpinning for the inferential tool.Source: §2 · p. 3
  • Several PFI interval and variance-estimation proposals exist, including methods specific to random-forest PFI and a null-importance test.The cited null-importance test is by Altmann et al. (2010).Source: §2 · pp. 3–4
  • For model-agnostic PFI intervals, the paper adds a correction for resampling-related variance underestimation and relates estimators to its proposed ground-truth PFI.Alternative work bounds PFI using Rashomon sets, while model-free inference seeks properties of data without an intermediary ML model.Source: §2 · p. 4

3 Background and Notation

The section establishes notation for the data-generating process, model and learner, then defines PD and PFI and distinguishes their variants. It also states the evaluation and loss conditions used for estimation.

3 Background and Notation

  • The learner maps datasets and hyperparameters to models, while model randomness arises from sampled training data and possibly from the training process itself.The real-world setting assumes one dataset of size n, so obtaining multiple model fits requires resampling techniques such as bootstrapping, cross-validation, or repeated subsampling.Source: §3 · pp. 4–5
  • A fixed-model interpretation omits variance from learning, whereas learner-level interpretation extends across the distribution of models produced by the learner.The section illustrates the distinction with a decision tree whose feature importance may change when trained on a different sample.Source: §3 · p. 5

3.1 Partial Dependence (PD)

  • Partial dependence describes a feature's expected effect on a model prediction after marginalizing over the other features.A PD plot connects estimated values across grid points, which are usually equidistant or quantiles of the feature distribution.Source: §3.1 · p. 5

3.2 Permutation Feature Importance (PFI)

  • PFI is the increase in loss when a feature set is permuted, and its estimates use Monte Carlo integration.The PFI estimate requires unseen data for unbiased loss estimates, and the assumed loss must be computable per instance, excluding losses such as AUC.Source: §3.2 · pp. 5–6
  • Marginal PFI breaks associations between the permuted features and both the target and remaining features, while conditional PFI samples features given the remaining features.Conditional PFI therefore describes a feature's additional importance given the other feature values; marginal PFI can evaluate combinations unlike the original joint distribution.Source: §3.2 · p. 6

4 Relating Model to Data Generating Process

The paper defines DGP-based counterparts for PD and PFI by applying the interpretation methods to the true function rather than a fitted model. These estimands support comparisons in simulations and treating model-based PD and PFI as statistical estimators, with error decomposed into bias, model variance, and Monte Carlo variance.

4 Relating Model to Data Generating Process

  • DGP-PD and DGP-PFI apply PD and PFI to the true DGP function f instead of the fitted model.For DGP-PFI, the losses under the permuted-feature distribution and the joint distribution are compared using f.Source: §4 · p. 7
  • Because f is usually unknown, the DGP-ground-truth definitions enable comparison with model PD/PFI in simulations and frame those methods as estimators of DGP properties.The definitions also permit study of statistical bias.Source: §4 · p. 7
  • The estimation error can be decomposed into statistical bias, model variance, and variance from Monte Carlo integration.Figure 1 depicts deviations of the model and estimated PD/PFI from their ground-truth counterparts and identifies these error sources.Source: Figure 1 · p. 8
  • The proposed variance and confidence-interval results also apply to conditional variants of PD and PFI, unless otherwise stated.The paper notes that dependent features can make PD and PFI misleading and that extrapolation may be a major issue.Source: §4 · pp. 7–8

5 Bias-Variance Decomposition

PD and PFI estimation error can be decomposed into bias and model variance, while Monte Carlo integration adds another source of variance. Averaging interpretations across model refits captures learner-level variability that fixed-model interpretations omit.

5 Bias-Variance Decomposition

  • PD and PFI mean-squared error decomposes into squared bias and model variance, with Monte Carlo integration adding another variance source.Source: §5 · p. 8, Appendix B · p. 9
  • For PD, bias is the expected model-PD's deviation from DGP-PD, while model variance is individual refit curves' deviation from their expected curve.Source: Figure 2 · p. 9

Appendix B.

  • Model variance arises from training-sample and training randomness; learner-PD/PFI average interpretations across refitted models.Source: Appendix B · p. 9
  • Model-PD/PFI variance estimators and confidence intervals account only for Monte Carlo approximation, not variation across model fits.Source: Appendix B · p. 9

6 Model-PD and Model-PFI

Model-PD and model-PFI quantify interpretations for a fixed model, with Monte Carlo uncertainty but not model-fit variance. Their Monte Carlo estimators are unbiased for theoretical model quantities, and are unbiased for DGP quantities under certain conditions.

6 Model-PD and Model-PFI

  • Model-PD and model-PFI condition on a fixed fitted model, so they account for Monte Carlo uncertainty but omit learning-process model variance.Source: §6 · p. 9
  • Model-PD/PFI Monte Carlo estimators are unbiased for theoretical model-PD/PFI and, under certain conditions, for DGP-PD/PFI.Source: §6 · pp. 9–10
  • Variance across test instances supports point-wise model-PD confidence bands and model-PFI confidence intervals based on a t-distribution.Source: §6 · p. 10
  • Because they ignore model variance, model-PD/PFI intervals are limited to fixed-model interpretation, such as a model audit, rather than DGP-level generalization.Source: §6 · p. 10

7 Learner-PD and Learner-PFI

Learner-PD and learner-PFI average interpretations across refitted models, while their bias relative to DGP targets depends on different conditions. The section also proposes variance correction and t-based confidence intervals.

7 Learner-PD and Learner-PFI

  • Learner-PD and learner-PFI average their values over refits from the same learning algorithm trained on different data samples.The learner-PD targets expected PD over the model distribution; learner-PFI similarly targets expected PFI.Source: §7 · pp. 10–11

7.1 Bias of Learner-PD

  • An unbiased learner-PD estimates the expected PD across models without bias, and its bias relative to DGP-PD is linked to model bias.If the ML model is unbiased, its PDs are unbiased as well.Source: §7.1 · p. 11

7.2 Bias of Learner-PFI

  • Learner-PFI is unbiased for expected PFI across models, but model unbiasedness alone does not generally make it unbiased for DGP-PFI.The paper studies this bias under squared-error loss.Source: §7.2 · p. 12
  • For L2 loss, unbiased models yield unbiased conditional model-PFI and learner-PFI; marginal PFI is unbiased only when the features are independent.With dependent features, marginal PFI has a variance-related bias.Source: §7.2 · p. 12
  • Marginal permutation can create extrapolation points outside DGP support, making model unbiasedness there a critical assumption and potentially introducing PFI bias.For dependent features, permutation may create combinations that conflict with the DGP.Source: §7.2 · p. 12

7.3 Variance Estimation

  • Their variance estimates include both variation across refitted models and Monte Carlo sampling, bringing inference closer to DGP-PD/PFI while leaving systematic bias unknown.The section proposes point-wise variance estimates for learner-PD and equivalently for learner-PFI.Source: §7.3 · p. 13
  • The correction term is zero for independently drawn simulation sets; with resampled fixed data, the proposed correction improves variance estimates but rests on a rough overlap-count assumption.It assumes refit correlation depends on the number of shared training observations, not which observations overlap.Source: §7.3 · p. 13

7.4 Confidence Bands and Intervals

  • Using the t-distribution with m − 1 degrees of freedom, the paper constructs confidence bands for learner-PD and intervals for learner-PFI; accounting for model variance can change interpretation.Resampling uses a larger share of data as test data than holdout does.Source: §7.4 · p. 14

8 Confidence Interval Coverage Simulation

The simulations compare confidence-interval coverage for learner-PD and learner-PFI across resampling strategies and variance corrections. Naive intervals are too narrow in the real-world setting; correction improves coverage, while resampling choice and refit count affect the coverage–width trade-off.

Confidence Interval Coverage Simulation

  • The simulations evaluated linear and non-linear DGPs and three learners, using 15 refits per setting and bootstrap or subsampling in the real-world setting.The simulation setting used fresh samples per refit; the real-world setting generated training sets from one sampled dataset.Source: §8 · p. 14
  • In the real-world setting, naive learner-PD and learner-PFI intervals were too narrow across scenarios and resampling strategies, with some linear-model coverage at 30%–40%.In the ideal simulation setting, standard variance estimation recovered the desired coverage probability.Source: §8 · p. 15
  • Variance correction drastically improved coverage, although intervals remained somewhat too narrow in simulated scenarios and adjusted tree intervals were sometimes too large.For linear models, coverage was around 80%–90% for PD and 60%–80% for PFI across DGPs and sample sizes.Source: §8 · pp. 15–16
  • The reported coverage targets the expected learner-PD and learner-PFI over models, not the DGP-PD or DGP-PFI.This experiment evaluates resampling and model-variance correction choices.Source: §8 · p. 15
  • Figure 4 compares confidence-interval width and coverage for bootstrap and subsampling methods, with correction applied to both; segments connect identical scenarios, without stating which method has better coverage or narrower intervals.Source: Figure 4 · p. 16
  • Increasing refits reduced interval width but worsened coverage; roughly 10–20 refits offered an acceptable trade-off, with little further width reduction above about 20.Below about 10 refits, intervals were large and mean PD/PFI estimates had high variance; additional refits can falsely reduce variance estimates.Source: §8 · pp. 16–17, Figure 5 · p. 17

9 Application

In the wine-quality application, learner-based interpretations reveal uncertainty across refitted random forests that fixed-model interpretations do not capture. The learner-PFI separates alcohol from sulphates more clearly, and learner-PDP bands are wider, especially at extreme alcohol values.

Application

  • The random forest had lower wine-quality prediction MSE than linear regression and the tree, and was significantly better by adjusted t-tests.MSEs were 0.342 for Random Forest, 0.425 for Linear regression, and 0.456 for Tree.Source: §9 · p. 17
  • Learner-PFI ranked alcohol above sulphates with non-overlapping confidence intervals, while model-PFI suggested their importance was almost equal.Both approaches gave a similar ordering for the top features overall.Source: §9 · p. 18, Figure 6 · p. 18
  • Learner-PDP confidence bands for alcohol were wider than model-PDP bands, especially at low and high alcohol values, indicating substantial model variance there.Ignoring model variance would make the partial-dependence curve appear overconfident.Source: §9 · pp. 18–19, Figure 6 · p. 18

10 Discussion

The discussion frames PD and PFI inference through DGP-grounded estimands, and notes that model-agnostic inference is used in practice but remains underexplored in theory.

10 Discussion

  • The authors relate PD and PFI to the DGP, propose variance estimates and confidence intervals, and discuss conditions for inference.Source: §10 · p. 19

A Bias and Variance of PD

B Bias and Variance of PFI

C Model-PD Unbiasedness Regarding Theoretical PD

D Model-PD Unbiasedness Regarding DGP-PD

E Model-PFI Regarding theoretical PFI

F PFI Biases for L2

Under squared loss, model-PFI bias has several components, while DGP-PFI is expressed through the squared difference between predictions on original and permuted features.

F PFI Biases for L2

  • Under squared loss, DGP-PFI is the expected squared difference between f(X) and f evaluated on permuted features.Source: F PFI Biases for L2 · p. 25
  • Under squared loss, model-PFI bias combines permutation-loss bias, squared model bias, and variance inflation; dependent features can bias marginal PFI even when the model is unbiased.Variance inflation vanishes for conditional PFI or independent feature groups.Source: F PFI Biases for L2 · p. 25

G DGP-PFI minus model-PFI for L2

H CI simulation results

The supplied figures report confidence-interval coverage and width for PD and PFI at sample sizes 100 and 1,000.

H CI simulation results

Loading 2109.01433v1…