Source-linked AI summary
A Perspective on Explainable Artificial Intelligence Methods: SHAP and LIME
Ahmed Salih, Zahra Raisi-Estabragh, Ilaria Boscolo Galazzo, Petia Radeva, Steffen E. Petersen, Gloria Menegaz, Karim Lekadir
TL;DR
XAI methods aim to make complex machine-learning decisions more understandable, but model dependence and feature collinearity may affect their explanations. This perspective examines SHAP and LIME using case studies and an interpretation framework, finding that their outputs are strongly affected by model choice and feature collinearity.
Problem
How model choice and feature collinearity affect XAI explanations remains important to examine, as end-users seek to understand complex machine-learning decisions.
Source: §1 · pp. 1–2Method
The perspective examines model-dependence and feature-collinearity effects on SHAP and LIME through case studies and proposes a framework for interpreting their outputs.
Source: §1 · p. 2Results
SHAP produced different feature-importance lists across models with relatively similar accuracy, while MIP changed the LGBM ranking to account for feature dependency.
Source: §3 · pp. 8–9Takeaways & Limitations
Users, especially those without technical backgrounds, should be aware of these issues to use XAI methods appropriately.
Source: §5 · p. 11Both methods have limitations with collinear features: SHAP assumes independence, while LIME's coefficient interpretation holds other features unchanged.
Source: §2.1 · p. 5, §2.2 · p. 8
Abstract
from arXiv · showhide
eXplainable artificial intelligence (XAI) methods have emerged to convert the black box of machine learning (ML) models into a more digestible form. These methods help to communicate how the model works with the aim of making ML models more transparent and increasing the trust of end-users into their output. SHapley Additive exPlanations (SHAP) and Local Interpretable Model Agnostic Explanation (LIME) are two widely used XAI methods, particularly with tabular data. In this perspective piece, we discuss the way the explainability metrics of these two methods are generated and propose a framework for interpretation of their outputs, highlighting their weaknesses and strengths. Specifically, we discuss their outcomes in terms of model-dependency and in the presence of collinearity among the features, relying on a case study from the biomedical domain (classification of individuals with or without myocardial infarction). The results indicate that SHAP and LIME are highly affected by the adopted ML model and feature collinearity, raising a note of caution on their usage and interpretation.
1 Introduction
XAI aims to make complex ML models more comprehensible, transparent, and trustworthy. This paper examines how model dependence and feature collinearity affect two common XAI methods, using biomedical case studies to explore their limitations and possible remedies.
- XAI seeks to demystify black-box models and increase transparency and end-user trust in their outcomes.The motivation is especially pressing in high-risk fields such as healthcare.Source: §1 · pp. 1–2
- The paper examines how model dependence and feature collinearity affect the quality of XAI outcomes.These issues are investigated through a biomedical case study involving two common XAI methods.Source: §1 · p. 2
- A second case study demonstrates how XAI methods can be implemented and explores possible solutions to their limitations involving model dependence and collinear features.Source: §1 · p. 2
2 eXplainable artificial intelligence
SHAP and LIME are popular XAI methods, but their explanations depend on the underlying model and can be affected by feature collinearity. The section outlines how their outputs are generated and why users should understand their assumptions and limitations.
2 eXplainable artificial intelligence
- SHAP uses feature coalitions to assign local and global explanations, whereas LIME fits a local interpretable model and provides local explanations; LIME’s linear surrogate cannot capture nonlinear relationships.SHAP’s coalition calculations become computationally complex as feature count increases, motivating the Kernel SHAP approximation.Source: §2 · p. 2, §2.1 · p. 4
- Because non-expert users may find XAI outputs difficult to understand, the authors aim to clarify SHAP and LIME’s underlying assumptions and promote appropriate use.SHAP [8] and LIME [9] are described as two of the most popular XAI methods in the literature.Source: §2 · pp. 2–3
2.1 SHAP
- Across four classifiers with comparable accuracy, SHAP agreed on the top three features but varied on the remaining seven, including BMI’s rank and near-zero scores for five features in DT.The models classified 1,500 UK Biobank subjects as MI or Non-MI using ten features.Source: §2.1 · p. 4, Figure 2 · p. 5
- SHAP scores should be interpreted through feature rankings, not as feature weights; its independence assumption can distort scores for collinear features, and biased classifiers can yield unrealistic explanations.A collinear feature may receive a low score despite association with the outcome because another correlated feature’s impact is already accounted for.Source: §2.1 · p. 5
- NMR assesses ranking stability as top features are removed iteratively, while MIP reorders any XAI feature list for multicollinearity without user intervention and can be applied to any model.The passage describes NMR as a stability measure, with smaller values indicating a more stable informative-feature list.Source: §2.1 · pp. 5–6
2.2 LIME
- For one subject, LIME showed model-dependent class probabilities and feature contributions, with the same feature sometimes supporting different classes across classifiers.Alcohol consumption supported MI in LR and SVC but Non-MI in DT and LGBM; the authors attribute similar effect sizes across models to LIME’s local linear approximation.Source: §2.2 · p. 6, Figure 3 · p. 7
- LIME’s linear coefficients imply one-unit feature changes while other features stay fixed, an unrealistic interpretation for collinear data; LIME can also produce explanations that fail to represent classifier bias.Because LIME reports surrogate-model coefficients, users may assign them more meaning than classical coefficients in linear models warrant.Source: §2.2 · pp. 6, 8
3 A case study
The case study shows that SHAP feature rankings vary across models and can be affected by collinearity. NMR identifies the most stable model output, while MIP modifies feature rankings to account for dependencies.
3 A case study
- The case study used 500 airline passengers, 22 features, and LGBM, LR, DT, and SVC; the feature correlation heatmap shows collinearity among some features.The models used default parameters, and the data were divided into training and testing sets (20%).Source: §3 · p. 8
- SHAP produced different informative-feature lists across the four models, including different top features, despite relatively similar accuracy apart from LGBM; Table 2 reports the lists.LGBM's higher accuracy does not establish that it is better, because default parameters were used and hyperparameter tuning could change model accuracy.Source: §3 · pp. 8–9, Table 2 · p. 9
- NMR values were LGBM 0.231, LR 0.275, DT 0.445, and SVC 0.273; LGBM's lowest value indicates its SHAP outcome was most robust.NMR assesses stability against collinearity but does not modify SHAP outcomes to account for collinearity.Source: §3 · p. 9
- For LGBM, MIP changed Ease of Online booking from fifteenth in the SHAP list to fifth, and can produce global or local informative-feature lists.For a local list, the authors propose applying XAI to a specific subject and then applying MIP; they also cite a method that modifies SHAP to account for collinearity.Source: §3 · p. 9, Table 3 · p. 10
4 Recommendations
The recommendations emphasize communicating XAI outputs and assumptions clearly, while accounting for model dependence, feature collinearity, and the limits of local explanations.
4 Recommendations
- Because SHAP and LIME are used in sensitive domains and practitioners may over-trust their explanations, clear interpretation is needed to avoid misuse.The passage warns that misinterpreting outcomes in sensitive domains may be expensive or critical.Source: §4 · p. 10
- Present SHAP results with corresponding plots, plain-language explanations, and assumptions such as feature independence and model dependence.Source: §4 · p. 10
- For collinear features, compare SHAP outcomes across ML models and use NMR to select a model with a more stable informative-feature list.Source: §4 · p. 10
- For collinearity, MIP [22] can enhance global explanations, while MIP [22] or shapr can support local explanations by accounting for feature collinearity.MIP can be applied to any XAI method, while shapr is a modified version of SHAP.Source: §4 · p. 10
- Convert SHAP feature scores, especially in classification, into a more digestible form to improve understanding.Source: §4 · p. 10
- Explain that LIME describes local linearity for one instance, may vary across instances, and is not a general interpretation; plain language can aid understanding.GraphLIME was proposed as an updated LIME for graph-based models, where non-linear association is more appropriately considered.Source: §4 · p. 10
5 Conclusions
The perspective emphasizes that users need to understand critical considerations when implementing widely used XAI methods, particularly for tabular data, to use them appropriately.
5 Conclusions
- End-users without technical backgrounds should understand critical considerations when implementing widely used tabular-data XAI methods to use them appropriately.The authors consider these issues significant across domains.Source: §5 · p. 11
6 Funding
The section acknowledges funding and institutional support for the authors and article, and notes that the views expressed are not necessarily those of the AI4VBH Consortium members, NHS, Innovate UK, or UKRI.
6 Funding
- AMS, IBG, GM, and ZRE acknowledge support from British Heart Foundation, Fondazione CariVerona, and NIHR programmes.AMS was supported by British Heart Foundation grant PG/21/10619; IBG and GM by Fondazione CariVerona’s EDIPO project; ZRE by the NIHR Integrated Academic Training programme and British Heart Foundation fellowship FS/17/81/33318.Source: §6 · p. 11
- SEP acknowledges NIHR Biomedical Research Centre at Barts support and EU Horizon 2020 funding for euCanSHare.The EU funding was under grant agreement No 825903.Source: §6 · p. 11
- The article is supported by AI4VBH, funded through the government’s Industrial Strategy Challenge Fund.The funding is managed and delivered by Innovate UK on behalf of UKRI; the authors state that their views are not necessarily those of AI4VBH Consortium members, the NHS, Innovate UK, or UKRI.Source: §6 · p. 11