Source-linked AI summary
DLIME: A Deterministic Local Interpretable Model-Agnostic Explanations Approach for Computer-Aided Diagnosis Systems
Muhammad Rehman Zafar, Naimul Mefraz Khan
TL;DR
LIME’s random perturbation can produce unstable explanations for the same prediction, a concern for medical applications requiring consistency. DLIME replaces perturbation with hierarchical clustering and KNN-based cluster selection, and experiments on three healthcare datasets show stable explanations compared with LIME.
Problem
Random perturbation in LIME can produce different explanations for the same prediction, limiting stability in computer-aided diagnosis settings.
Method
DLIME deterministically clusters training data with agglomerative Hierarchical Clustering, uses KNN to select the relevant cluster, and trains a linear model for explanations.
Results
Experiments on three healthcare datasets show stable DLIME explanations across iterations, while LIME explanations are unstable.
Takeaways & Limitations
DLIME provides consistent explanations for individual black-box predictions in the evaluated healthcare datasets.
Takeaways & Limitations
The dataset’s sample count may affect hierarchical-cluster quality and consequently the accuracy of local predictions.
Abstract
from arXiv · showhide
Local Interpretable Model-Agnostic Explanations (LIME) is a popular technique used to increase the interpretability and explainability of black box Machine Learning (ML) algorithms. LIME typically generates an explanation for a single prediction by any ML model by learning a simpler interpretable model (e.g. linear classifier) around the prediction through generating simulated data around the instance by random perturbation, and obtaining feature importance through applying some form of feature selection. While LIME and similar local algorithms have gained popularity due to their simplicity, the random perturbation and feature selection methods result in "instability" in the generated explanations, where for the same prediction, different explanations can be generated. This is a critical issue that can prevent deployment of LIME in a Computer-Aided Diagnosis (CAD) system, where stability is of utmost importance to earn the trust of medical professionals. In this paper, we propose a deterministic version of LIME. Instead of random perturbation, we utilize agglomerative Hierarchical Clustering (HC) to group the training data together and K-Nearest Neighbour (KNN) to select the relevant cluster of the new instance that is being explained. After finding the relevant cluster, a linear model is trained over the selected cluster to generate the explanations. Experimental results on three different medical datasets show the superiority for Deterministic Local Interpretable Model-Agnostic Explanations (DLIME), where we quantitatively determine the stability of DLIME compared to LIME utilizing the Jaccard similarity among multiple generated explanations.
1 INTRODUCTION
LIME explains individual black-box predictions locally but its random perturbation makes explanations unstable, which is problematic for CAD systems. DLIME replaces random perturbation with deterministic clustering and nearest-neighbor selection to produce consistent explanations.
- LIME explains individual predictions from any ML model by fitting a sparse linear model to responses from randomly perturbed samples.
- Random perturbation makes LIME non-deterministic, so the same instance can receive different explanations across runs.
- DLIME partitions training data with Hierarchical Clustering instead of random perturbation, then uses KNN to select similar points for a local linear model.
- The paper evaluates DLIME and LIME on three UCI healthcare datasets to assess explanation consistency.
2 RELATED WORK
Prior locally interpretable methods include perturbation-, feature-contribution-, gradient-, and game-theory-based approaches. This paper focuses on their stability problem, defined as variability in selected explanatory features for the same instance.
- Local interpretability explains individual classifier predictions by modeling learned relationships over small regions around an instance.
- LIME generates perturbed samples and fits a sparse linear model, while related methods use decision rules, LOCO, K-means, feature elimination, SHAP, or local gradients.
- Existing locally interpretable models can exhibit explanation-level uncertainty because simplifying a black-box model introduces instability.
- Here, stability specifically means intensional stability of feature selection, measured by variability in the selected feature set.
3 METHODOLOGY
LIME creates perturbed samples, queries the opaque model, and fits an interpretable local model, but repeated random sampling can change its explanations. DLIME instead clusters training data deterministically, assigns a test instance to a predominant cluster through KNN, and fits a linear model there.
- LIME: LIME generates synthetic samples around an instance, obtains opaque-model predictions, and fits an interpretable model such as linear regression.
- LIME: LIME’s random sampling makes repeated explanations differ, producing instability in the selected features and weights.
- DLIME: DLIME uses Hierarchical Clustering to partition training data into groups and replaces random perturbation with cluster-based local sampling.
- Hierarchical Clustering (HC): Agglomerative clustering starts with each point as a cluster and repeatedly merges similar groups into a binary hierarchy using distance-based similarity.
- Hierarchical Clustering (HC): The number of clusters is chosen by cutting the dendrogram at its largest gap; the binary datasets in the experiments used C = 2, while multiclass settings may differ.
- K-Nearest Neighbor (KNN): KNN finds neighbors of the test instance, assigns the majority cluster label, and supplies that cluster’s points to a linear regression model for explanation.
4.1 Dataset
The experiments use three healthcare datasets from the UCI repository: breast cancer, Indian liver patient, and hepatitis datasets, with differing numbers of observations and features.
- The experiments use three healthcare datasets from the UCI repository.
- The Breast Cancer Wisconsin (Original) dataset contains 699 observations and 11 features.
- The Indian liver patient dataset contains 583 observations and 11 features.
- The hepatitis dataset contains 155 observations and 20 features, with 80 observations used after missing-value removal.
4.2 Opaque Models
The experiments use Random Forest and Neural Network opaque models across three datasets, with 80% of data for training and 20% for evaluation. Both models exceeded 90% accuracy on each dataset, and LIME and DLIME were each run ten times on one test instance.
- Opaque models: Random Forest and Neural Network models were trained on all three datasets as opaque prediction models.The models were implemented using scikit-learn.
- Neural Network: The Neural Network used feed-forward architecture with two hidden layers containing five and two units, respectively.It was trained using backpropagation.
- Evaluation: Both opaque models achieved over 90% accuracy on each dataset.The authors did not further tune hyperparameters because the focus was deterministic explanations.
- Evaluation: For both models, 80% of the data was used for training and 20% for evaluation.Explanations were generated for a randomly selected test instance, with each algorithm executed for ten iterations.
4.3 Results
The results compare LIME and DLIME explanations using visual inspection and Jaccard distance across ten iterations. DLIME produced stable explanations with zero Jaccard distance in every reported scenario, whereas LIME produced varying explanations and significant distances.
- Visual comparison: DLIME generated deterministic and stable explanations for the same test instance, while LIME generated different explanations across iterations.The visual comparison used explanations from trained Neural Networks on the breast cancer dataset.
- Stability metric: Jaccard coefficient measures explanation-set similarity as the intersection size divided by the union size.A value of 1 indicates highly similar sets, while 0 indicates no intersection and highly dissimilar sets.
- Stability metric: Jaccard distance is defined as 1 − J(S1,S2), where S1 and S2 are two explanation sets.The distance was computed among explanations generated after ten iterations.
- Quantitative results: The Jdistance matrix for DLIME contained zeros, indicating identical explanations across iterations, whereas LIME showed significant Jdistance values.The matrix is 10 × 10, with zeros on the diagonal and pairwise distances in the off-diagonal entries.
- Quantitative results: Across every scenario in Table 1, DLIME had average Jdistance zero, while LIME had significant values.The comparison used two opaque models, three datasets, ten iterations, and one randomly selected test instance.
5 CONCLUSION
DLIME proposes deterministic explanations for black-box models by replacing random perturbation with a deterministic approach. Evaluations on three healthcare datasets found stable DLIME explanations, while hierarchical clustering remains sensitive to dataset sample size.
- DLIME proposes a deterministic approach to explaining black-box model decisions.
- DLIME uses hierarchical clustering to group similar data and KNN to select the cluster corresponding to a test instance.
- Experiments on three healthcare datasets found stable DLIME explanations across iterations, while LIME generated unstable explanations.
- Dataset sample size may affect hierarchical-cluster quality and consequently the accuracy of local predictions.
- The authors plan to investigate this clustering issue while preserving explanation stability and to test additional data types, including images and text.