Source-linked AI summary
Statistical stability indices for LIME: obtaining reliable explanations for Machine Learning models
Giorgio Visani, Enrico Bagli, Federico Chesani, Alessandro Poluzzi, Davide Capuzzo
TL;DR
Machine Learning models can improve prediction while making decision logic difficult to understand, and repeated LIME calls may yield inconsistent explanations. This paper proposes two complementary stability indices, applies them to Credit Risk data, and concludes that they assess consistency between LIME explanations while not guaranteeing fidelity to the underlying model.
Problem
Machine Learning models can be difficult to interpret, and repeated LIME calls under the same conditions may produce different explanations.
Method
The paper derives local-model coefficient distributions and proposes CSI for coefficient similarity and VSI for agreement among variables returned across repeated LIME calls.
Results
The two indices help practitioners assess whether LIME is likely to modify its output at the next call, and Machine Learning predictions outperformed classical models in the Credit Risk use case.
Takeaways & Limitations
The indices provide complementary evidence about LIME consistency, with higher values corresponding to greater stability.
Takeaways & Limitations
Stability assessment establishes concordance among LIME applications but does not ensure that explanations are close to the Machine Learning model.
Abstract
from arXiv · showhide
Nowadays we are witnessing a transformation of the business processes towards a more computation driven approach. The ever increasing usage of Machine Learning techniques is the clearest example of such trend. This sort of revolution is often providing advantages, such as an increase in prediction accuracy and a reduced time to obtain the results. However, these methods present a major drawback: it is very difficult to understand on what grounds the algorithm took the decision. To address this issue we consider the LIME method. We give a general background on LIME then, we focus on the stability issue: employing the method repeated times, under the same conditions, may yield to different explanations. Two complementary indices are proposed, to measure LIME stability. It is important for the practitioner to be aware of the issue, as well as to have a tool for spotting it. Stability guarantees LIME explanations to be reliable, therefore a stability assessment, made through the proposed indices, is crucial. As a case study, we apply both Machine Learning and classical statistical techniques to Credit Risk data. We test LIME on the Machine Learning algorithm and check its stability. Eventually, we examine the goodness of the explanations returned.
1. Introduction
Machine Learning supports fast, accurate, data-driven prediction but often obscures the grounds for its decisions. The paper focuses on LIME and proposes complementary indices to detect whether repeated explanations remain consistent.
- Machine Learning models can automate data-driven processes and deliver accurate results quickly, but their internal logic is difficult for humans to grasp.
- LIME is a local, model-agnostic interpretability framework developed to explain Machine Learning predictions.
- Repeated LIME applications under identical conditions may produce different explanations, yet this instability is often overlooked after a single method call.
- The paper introduces two complementary stability indices for measuring LIME stability and spotting potential issues.
- The indices compare repeated LIME results to help practitioners identify instability or assess whether the trained method is consistent.
2. Related Work
Related work organizes explainability methods by scope, model dependence, and explanatory strategy. It highlights feature-exclusion methods and surrogate models as major approaches, including LIME for local approximation.
- Explainability techniques are categorized as global or local, with model-agnostic methods designed to work across Machine Learning models.
- Feature-exclusion methods assess a feature’s contribution through the predictive loss incurred when that feature is removed.
- LOCO generalizes feature-exclusion analysis to a model-agnostic framework, while PDP, ICE, and ALE decompose predictive power under different assumptions.
- SHAP applies feature-contribution decomposition to local explanations through a game-based setting, but feature correlation can distort these methods’ results.
- Surrogate models mimic Machine Learning behavior; LIME uses this approach to provide local approximations and support what-if analysis.
3. LIME
LIME explains black-box predictions locally by fitting a sparse, weighted linear surrogate around an input. Its coefficients provide feature-level interpretation, but high dimensionality can undermine locality and feature discrimination.
- 3.1. General Idea: LIME fits an explainable local model g that approximates black-box model f around an input point x.
- 3.1. General Idea: The local model uses p of the black-box model’s P features, reducing complexity and selecting the function most similar to f near x.
- 3.1. General Idea: LIME generates points from independently modeled Normal feature distributions, weights them by Gaussian-kernel proximity to x, queries the black-box model, and standardizes the resulting dataset.
- 3.1. General Idea: Feature selection, usually with Lasso, retains a practitioner-chosen number p of variables to make the linear explanation compact and human readable.
- 3.2. LIME Algorithm in detail: Weighted Ridge Regression produces the local linear model, whose coefficient magnitude indicates response variation and whose sign indicates its direction.
- 3.3. LIME Drawbacks: In high-dimensional datasets, LIME’s pre-reduction weighting kernel cannot distinguish near from distant points, weakening locality and degrading performance.
- 3.3. LIME Drawbacks: This limitation restricts LIME’s use on high-dimensional black-box models and usually requires upstream feature selection.
4. LIME Stability issue
Repeated LIME applications under identical conditions can produce different explanations because each call samples different data points. The issue is especially concerning for high-dimensional models, where explanations may fail to distinguish important from irrelevant features.
- Repeated LIME calls for one individual can yield divergent explainable models and different explanations because sampling is random.Each call follows the same feature distribution but generates different points.
- LIME stability concerns whether explanations from repeated calls under the same conditions remain consistent.
- Figure 2 reports uninformative LIME explanations for a Gradient Boosting model using 100 features.
- Prior work has examined LIME robustness and proposed approaches aimed at addressing the stability issue.
Avoid the sampling step
One proposed way to obtain stable explanations is to bypass LIME’s sampling step and use the training units directly with clustering and nearest-neighbour techniques.
- Zafar and Khan bypass LIME’s sampling step by using training units with Hierarchical Clustering and K-Nearest Neighbour techniques.The method achieves stability but may approximate the ML function poorly in regions with few training points.
Evaluate the post-hoc stability
Post-hoc stability evaluates whether repeated LIME results are equivalent, focusing on coefficient variability or consistency in selected variables. The paper argues that existing stability metrics need stronger mathematical grounding.
- Post-hoc evaluations repeat LIME under identical conditions and test whether the resulting explanations are equivalent.
- Existing proposals compare Ridge-coefficient standard deviations or examine whether the same variables are selected.
- The authors argue that more work is needed to provide solid mathematical grounds and rigour for LIME-stability metrics.
5. Our Proposition
The paper proposes two complementary indices for assessing LIME stability across repeated calls: VSI evaluates consistency in selected variables, while CSI evaluates consistency in coefficients. CSI uses confidence-interval overlap because standard coefficient-equality testing is not directly valid for Ridge regression.
- LIME’s explainable model is a weighted Ridge Regression mapping variables to coefficients, with only p of P variables assigned nonzero coefficients.
- The framework repeats LIME m times on the same model and individual, producing m explainable models for comparison.
- Variables Stability Index (VSI): The Variables Stability Index (VSI) measures concordance in the variables included across pairs of repeated LIME explainable models.It averages pairwise intersection counts and ranges from 0 to 1, or 0 to 100 as a percentage.
- LIME generates weighted Ridge models from randomly sampled points, with kernel weights determined by each point’s distance from the explained individual.
- Coefficients Stability Index (CSI): The coefficient-equality t-test derivation breaks down for Ridge regression because regularisation makes coefficient estimators biased and their expected values depend on the design matrix.The authors therefore construct 95% confidence intervals from the Gaussian coefficient distribution and use their overlap to assess stability.
- Coefficients Stability Index (CSI): The Coefficients Stability Index (CSI) measures concordance among coefficients for the same variable across repeated LIME calls.It averages variable-level Partial Indices based on confidence-interval overlap and ranges from 0 to 1, or 0 to 100 as a percentage.
16 end
The two complementary indices assess distinct aspects of LIME stability: consistency of selected variables and reliability of feature coefficients. High values on both indicate stable explanations, while a low value on either identifies which stability condition is violated.
- High VSI values indicate that repeated LIME runs retrieve nearly the same explanatory variables.
- Low VSI values indicate that the same Machine Learning decision may receive explanations based on completely different variables.
- High CSI values indicate reliable LIME coefficients for individual features across repeated calls.
- Low CSI values warn that a feature’s coefficient may change between calls, producing substantially different explanations.Because coefficients represent feature impacts on the Machine Learning decision, changing coefficients alter the explanation.
- The indices remain separate so practitioners can identify which of the two complementary stability conditions has failed.High values for both indices indicate stability, but one index can be low while the other is not.
6. Practical Application to Credit Risk Data
The practical application evaluates Logistic Regression and Gradient Boosting on anonymised Italian credit-risk data, then tests LIME explanations for the Gradient Boosting model. Gradient Boosting improves the Gini index by more than 3 points, while LIME stability depends on its settings and can be assessed through repeated runs.
- 6. Practical Application to Credit Risk Data: Credit Risk Modelling estimates whether a debtor will repay, using demographic, economic, and financial predictors to classify applicants as good or bad payers.
- 6. Practical Application to Credit Risk Data: Interpretability is especially important in Credit Risk Modelling because the field is subject to GDPR and European trustworthy-AI and banking guidance.
- 6. Practical Application to Credit Risk Data: The study applies LIME to a well-performing black-box model to retain Machine Learning predictive power while providing explanations to applicants and regulators.
- 6. Practical Application to Credit Risk Data: Only the 20 most important features are retained because high-dimensional models impair classical modelling and can cause LIME to fail.
- 6.3. Model comparison: Gradient Boosting improves the Gini index by more than 3 points compared with Logistic Regression on the test set.Gradient Boosting hyperparameters are tuned by grid search and 10-fold cross-validation on the training set.
- 6.4. LIME applied to Credit Risk models: Repeated LIME explanations reveal stable results under suitable settings and instability when kernel width and Ridge penalty values are poorly chosen.The indices use 10 LIME applications, while the implementation permits a different repetition count and considers the seven most important features.
- 6.4. LIME applied to Credit Risk models: The stable explanation is economically plausible: zero unpaid instalment months and a good Credit Bureau Score are associated with the model’s good-payer classification.
- 6.4. LIME applied to Credit Risk models: The unstable LIME configuration produces different regression lines across calls for the same individual, making the explanations difficult to trust.
7. Discussion and conclusions
The paper applies LIME to Credit Risk data and examines whether repeated calls produce stable explanations. It proposes complementary stability indices that improve consistency assessment, while acknowledging that concordance across calls does not ensure fidelity to the underlying Machine Learning model.
- Machine Learning models often improve predictive accuracy but remain difficult to interpret, motivating the study of LIME explanations.
- The paper applies LIME to Credit Risk data and tests whether repeated calls on the same individual produce closely matching explanations.
- The proposed framework derives local-coefficient distributions and combines CSI and VSI, both ranging from 0 to 100, to assess coefficient similarity and feature-selection consistency.
- Using both indices helps practitioners assess whether LIME is likely to modify its output at the next call, improving trust in its reliability.
- The stability assessment establishes concordance across LIME applications but does not guarantee that explanations closely reflect the Machine Learning model.
Funding
The paper acknowledges financial support from CRIF S.p.A. and the University of Bologna.
- Financial support was provided by CRIF S.p.A. and Università degli Studi di Bologna.