Source-linked AI summary
Explanation of Machine Learning Models Using Shapley Additive Explanation and Application for Real Data in Hospital
Yasunobu Nohara, Koutarou Matsumoto, Hidehisa Soejima, Naoki Nakashima
TL;DR
Complex machine-learning models are difficult to interpret, and conventional gain-based feature importance can be inconsistent. This paper applies SHAP with new feature-importance and feature-packing techniques to hospital cerebral-infarction data, finding that 18 of the top 20 features matched gain rankings.
Problem
Complex machine-learning models are difficult to interpret, while gain-based feature importance can be inconsistent for comparing feature attributions.
Method
The paper uses SHAP to interpret a hospital-based GBDT model, introducing a SHAP feature-importance metric and feature packing for grouped-feature interpretation.
Results
18 of 20 top-ranked features matched between SHAP and gain, while SHAP identified total NIHSS as most important and the A/G ratio as sixth.
Takeaways & Limitations
The analysis identifies the A/G ratio as an unexpected important prognostic factor for cerebral infarction and finds SHAP summary plots useful for overviewing results.
Takeaways & Limitations
The reported relationships are correlations rather than established causal effects, and further analysis is necessary.
Abstract
from arXiv · showhide
When using machine learning techniques in decision-making processes, the interpretability of the models is important. In the present paper, we adopted the Shapley additive explanation (SHAP), which is based on fair profit allocation among many stakeholders depending on their contribution, for interpreting a gradient-boosting decision tree model using hospital data. For better interpretability, we propose two novel techniques as follows: (1) a new metric of feature importance using SHAP and (2) a technique termed feature packing, which packs multiple similar features into one grouped feature to allow an easier understanding of the model without reconstruction of the model. We then compared the explanation results between the SHAP framework and existing methods. In addition, we showed how the A/G ratio works as an important prognostic factor for cerebral infarction using our hospital data and proposed techniques.
1. Introduction
The paper addresses the interpretability challenge of accurate but complex machine-learning models in medical decision-making by applying SHAP to hospital-based GBDT data. It proposes a SHAP feature-importance metric and feature packing, then applies them to cerebral-infarction prognosis prediction.
- Motivation: Complex machine-learning models can achieve high accuracy but are harder to interpret than simple models, making interpretability important in medical decision-making.The introduction contrasts deep-learning and ensemble models with linear models.
- Proposed methods: The study applies SHAP to a GBDT model constructed from real hospital data and proposes a feature-importance metric whose interpretation matches that of a generalized linear model.The metric is presented as one of two techniques intended to improve interpretability.
- Proposed methods: Feature packing groups multiple similar features into one feature, enabling easier model understanding without reconstruction while maintaining accuracy.The paper states that the new metric helps select similar features, whereas feature packing improves understanding without rebuilding the model.
- Hospital application: The techniques are applied to a cerebral-infarction prognosis predictor, with explanation results compared between SHAP and the existing gain method.The application uses cerebral-infarction data from the authors’ hospital.
2. Background
The background contrasts interpretable generalized linear and single-tree models with more accurate ensemble trees that capture nonlinearities and interactions. It then motivates SHAP as a consistent, broadly applicable feature-importance method and relates SHAP values to GLM coefficients.
- Generalized linear models: GLMs interpret feature effects through coefficients, with positive or negative coefficients indicating the direction of a feature’s impact on the outcome.Increasing x_i by 1 changes the logistic model’s log-odds by coefficient a_i.
- Generalized linear models: Standardized beta coefficients are dimensionless, directly comparable, and assessed by absolute value, with larger values indicating greater feature importance.This comparison applies when features are standardized.
- Model interpretability and prediction: Although GLMs and single decision trees are easy to interpret, they are difficult to use for accurate modeling with nonlinear terms, interactions, or complex data.Single decision trees express outcomes through simple if-then rules but are difficult to make accurate.
- Ensemble tree models: Ensemble trees such as Random Forest and GBDT improve predictive performance and handle high-dimensional features, nonlinear relationships, and feature interactions, but make if-then rules harder to interpret.Their strength comes from combining multiple weak decision-tree learners into one strong learner.
- SHAP interpretation: SHAP addresses inconsistent gain-based importance by using Shapley values, representing each patient’s outcome as the sum of feature contributions and providing consistency across machine-learning algorithms.Polynomial-time SHAP algorithms are available for decision trees and ensemble trees through frameworks such as XGBoost and LightGBM.
- SHAP interpretation: In GLMs, each feature’s SHAP value is proportional to the feature value, with coefficient a_i as the proportionality factor, consistent with existing GLM interpretations.This establishes a direct correspondence between SHAP contributions and GLM coefficients.
3. Methods
The methods use SHAP plots to interpret feature effects and importance in the model. They introduce variance-based variable importance and feature packing to summarize correlated or similarly meaningful features without reconstructing the predictor.
- SHAP visualization: SHAP dependence plots relate each feature to its outcome effect, with SHAP values corresponding to logistic-regression log-odds in binary prediction.For GLM, the dependence plot shows the linear relationship given by Eq. (6).
- Variable importance: The proposed variable-importance measure uses the variance (L2-norm) of SHAP values instead of the sum of their absolute values (L1-norm).For standardized GLM features, its ranking exactly matches the ranking by absolute beta coefficients βi.
- Feature packing: Feature packing groups similar or correlated variables into one feature without reconstructing the predictor, avoiding reconstruction time and possible accuracy loss.Examples include weight, height, and BMI as correlated body-related features.
- Feature packing: Grouped feature importance increases with SHAP-value covariance and exceeds the sum of the individual feature importances.High covariance indicates similar effects on the outcome or strong interactions between features.
- SHAP visualization: The SHAP summary plot ranks features by Eq. (7) importance and combines dependence-plot information, with dots encoding patient SHAP values and feature values.Colors range from low (blue) to high (red), while black dots indicate missing values.
- Analysis workflow: The analysis workflow starts with a SHAP summary plot, proceeds to dependence plots for selected features, and optionally packs similar features into grouped features.Similarity is identified by feature meaning or SHAP-value covariance.
4. Experiment and Results
The experiment used admission-day hospital data to build and interpret a GBDT prognosis predictor for cerebral infarction. SHAP identified clinically interpretable feature effects, including NIHSS, D-dimer, A/G ratio, and grouped ADL variables.
- Data and predictor: Admission-day data from five hospital sources yielded 1714 one-hot-encoded features for a GBDT predictor of cerebral-infarction prognosis.Sources included standardized discharge summaries, neurology summaries, nutritional management, nursing care, and examination outcomes.
- Data and predictor: 0.788 was the predictor’s mean cross-validated AUC, with a standard deviation of 0.006.The paper describes this as good prediction accuracy.
- Importance comparison: 18 out of 20 features matched between gain-based and SHAP rankings, with total NIHSS ranked most important by both methods.The gain plot provided relative importance, whereas SHAP also represented feature effects and distributions.
- SHAP analysis: A/G ratio was an unexpected sixth important feature; a value of 1.5 marked the good-prognosis threshold, and high A/G ratio odds were 0.74[=exp(-0.3)].D-dimer showed an almost proportional relationship with bad-prognosis risk.
- SHAP analysis: Higher total NIHSS indicated higher bad-prognosis risk, with a worse-prognosis threshold of three in the SHAP dependence analysis.The SHAP summary plot showed feature direction, unlike the feature-importance plot.
- Feature packing: Packing ten correlated admission ADL endpoints into ‘adlAdmission.all’ concentrated their distributed importance into a grouped variable for easier model understanding.The grouped-variable result is presented in Figure 7.
5. Discussion
The discussion emphasizes that SHAP can explain how missing values affect outcomes, with interpretations depending on their generation mechanism. It also notes that A/G ratio associations may reflect albumin or globulin effects, while causal interpretation requires further analysis.
- Missing values: SHAP values can be calculated for missing values, and SHAP plots show how those missing values affect outcomes.This contrasts with general machine-learning methods that typically require missing-value imputation, whereas tree models can handle missing values directly.
- Missing values: Missing-value effects depend on the generation mechanism, with values missing at random producing impacts intermediate between high- and low-risk groups.Paralysis level and JCS level measurements were performed for all stroke patients, illustrating the MAR case described.
- Missing values: Nonrandom missingness can shift missing-value points toward the high- or low-risk side of a SHAP summary plot.Blood glucose was mainly measured for diabetic patients, so most missing values came from patients with low glucose levels.
- A/G ratio: A high A/G ratio may indicate high albumin or low globulin, and albumin alone is unlikely to explain the ratio’s importance because albumin has low variable importance.The paper notes albumin’s reported brain-protective effect but states that the mechanism behind the A/G ratio finding is unknown.
- A/G ratio: The estimated causal and correlation relationships shown in Figure 9 require further analysis because data analysis provides correlations only.Causal correlations were based on known facts rather than established solely by the observed data.
6. Conclusion
The paper used SHAP to interpret a gradient-boosting decision tree model built from real hospital data. It proposed a new SHAP-based feature-importance metric and feature packing to improve interpretability without reconstructing the model.
- SHAP was adopted to interpret a GBDT model constructed using real hospital data.
- The paper proposed a new metric of feature importance using SHAP.
- Feature packing grouped features to make the model easier to understand without reconstruction.
- The explanation results were compared between SHAP and an existing method such as gain feature importance.