BayLIME: Bayesian Local Interpretable Model-Agnostic Explanations

• Local-surrogate explanations for individual predictions face inconsistency across repeated runs, sensitivity to kernel settings, and a need for higher fidelity.Source: §1 · p. 1
• LIME’s repeated explanations vary because perturbed samples are generated randomly, with smaller samples increasing uncertainty.Source: §1 · p. 1
• BayLIME uses a Bayesian local surrogate to combine prior knowledge with estimates from new samples, with parameters fitted from data or elicited from application-specific knowledge.Source: §1 · p. 2
• The paper introduces quantitative metrics for repeated-explanation consistency and kernel-setting robustness that apply to feature-ranking XAI methods.Source: §1 · p. 2
• It illustrates prior elicitation using explanations on similar instances, other XAI methods, and verification and validation methods.Source: §1 · p. 2
• Experiments report that BayLIME outperforms LIME on consistency and kernel robustness and achieves higher fidelity than LIME, SHAP, and GradCAM.Source: §1 · p. 2

• LIME’s repeated explanations are unstable because random perturbed samples train the local surrogate, and enlarging the sample to reduce randomness may be impractical under efficiency constraints.Source: §2 · p. 2
• Limiting black-box model queries to improve LIME’s efficiency can make inconsistency more severe.Source: §2 · p. 2
• Kendall’s W measures agreement among repeated explanations, while an importance-weighted index-of-dispersion metric distinguishes explanations with identical feature rankings but different importance vectors.Source: §2 · p. 2

3 ROBUSTNESS TO KERNEL SETTINGS

• The comparison studies kernel width in LIME’s default exponential kernel; other kernel-setting parameters can be studied similarly.The kernel width controls the size of the neighborhood used to train the local surrogate, and the paper notes that changing it can turn explanations around.Source: §3 · p. 3
• A global Lipschitz value L bounds changes in feature-importance vectors against kernel-width changes, but is difficult to solve analytically or estimate numerically.The difficulty stems from the high complexity and non-linearity of the importance-vector function.Source: §3 · p. 3
• The empirical robustness measure uses the median ratio of perturbed distances across kernel widths in a bounded range, offering an easier-to-estimate alternative to optimizing L.The bounds [llo, lup] can be determined empirically after a few trials, such as cross-validation.Source: §3 · p. 3

4 EXPLANATION FIDELITY

• Fidelity is assessed through actual-causality indicators, using deletion and insertion metrics alongside neural backdoors that provide ground truth trigger areas.The paper uses these complementary evaluation strategies because the underlying model’s ground-truth behavior is generally unavailable.Source: §4 · p. 3
• Deletion removes top-ranked features and measures the predicted-label probability drop; a sharper drop and lower AUC indicate a better explanation.Insertion measures the complementary probability increase as important features are added; higher AUC indicates a better explanation.Source: §4 · p. 3
• For backdoor evaluation, IoU measures overlap between the true trigger and highlighted area, with higher IoU indicating better trigger localization.The paper adds a secondary metric in Appendix A.2 because IoU ignores geometric closeness when the areas do not overlap.Source: §4 · pp. 3–4

5 A BAYESIAN RETROFIT OF LIME

• BayLIME models the local surrogate with Bayesian linear regression, using a Gaussian response and conjugate Gaussian prior; its simplifying assumption uses one precision parameter λ.The local model is trained on perturbed samples weighted by their proximity to the instance being explained.Source: §5 · p. 4, §5.1 · p. 4

5.1 EMBEDDING PRIOR KNOWLEDGE IN LIME

• The posterior combines the prior estimate and the weighted-sample MLE as a weighted sum, balancing prior knowledge against new observations.The paper explains that prior information can average out randomness in perturbed samples and kernel settings, while diverse prior information can benefit fidelity.Source: §5.1 · pp. 4–5

5.2 THE BAYLIME FRAMEWORK

• BayLIME offers non-informative, partial-informative, and full-informative prior options, fitting unknown parameters by Bayesian model selection or using known parameters directly.The non-informative option fits λ and α; the partial-prior option fixes λ and fits α; the full-prior option assumes both are known.Source: §5.2 · p. 5

5.3 WAYS OF OBTAINING PRIOR KNOWLEDGE

• Prior knowledge may come from V&V methods, diverse XAI explainers, or similar instances, though eliciting accurate priors is application-specific and challenging.The examples include NeuralCleanse results, GradCAM explanations, and explanations of similar instances.Source: §5.3 · p. 5

Evaluation design

• The experiments cover Boston house prices, breast-cancer data, and pretrained ImageNet and GTSRB CNNs, with priors drawn from similar instances, GradCAM, or NeuralCleanse.RQ1 varies perturbed sample size and repeats explanations; RQ2 samples 5,000 kernel-width pairs to assess robustness.Source: §6 · pp. 5–6

RQ1

• At n < 200, LIME shows low repeated-explanation consistency; informative-prior BayLIME can average sampling noise, with stronger prior influence generally improving consistency.Consistency may first worsen and then improve as n grows, and all plots eventually converge as n approaches infinity.Source: §6 · pp. 6–7, Figure 2 · p. 6

RQ2

• Informative priors improve robustness to kernel settings, while LIME and BayLIME with non-informative priors show similar robustness.The authors attribute the improvement to priors being independent of kernel settings, unlike new data; the box plots show robustness for eight explainers.Source: §6 · p. 7, Figure 3 · p. 7

RQ3.a

• For n≈100–400, BayLIME has better fidelity than SHAP, LIME, and GradCAM on deletion and insertion metrics, with results converging as n increases.GradCAM performs better only when n is extremely small, even smaller than the number of features.Source: Figure 4 · p. 7, §6 · p. 7

RQ3.b

• BayLIME using NeuralCleanse priors achieves the best IoU among the compared methods, although NeuralCleanse’s reversed trigger can miss the ground-truth trigger entirely.In the BadNet example, the reversed trigger has IoU = 0; statistics averaged over 500 random images confirm the comparison.Source: §6 · p. 8, Figure 5 · p. 8, Table 2 · p. 8

LIME extensions

• Prior LIME-related work modifies perturbed sampling to address feature correlation, uses reinforcement learning to select perturbed samples for local-surrogate training, and combines local linear regressors with random forests to weight instances and optimize the local linear model.Source: §7 · p. 8

Consistency and Bayesian approaches

• BayLIME seeks inherently different solutions in a Bayesian way, uniquely embedding prior knowledge; direct experimental comparisons with DLIME and ALIME are not sensible because they cannot represent that role of prior knowledge.Source: §7 · p. 8

Bayesian XAI

• The cited Bayesian-flavoured model-agnostic methods differ from BayLIME: one uses a global approximation, while another uses posterior credible intervals to choose sample size.BayLIME is described as the first to exploit prior knowledge for consistency, kernel-setting robustness, and explanation fidelity.Source: §7 · p. 8

8 DISCUSSION

• Priors can come from other XAI tools, V&V methods, or explanations of similar instances, with additional application-specific sources possible.The authors describe these as three typical implemented sources and plan to investigate more practical ways to obtain priors.Source: §8 · p. 8
• Because black-box ground-truth behavior is unavailable, prior quality cannot be known with certainty, although reliable V&V sources may provide practical confidence.The authors distinguish priors that reflect the model’s unknown behavior from priors chosen to produce explanations that look good to users.Source: §8 · p. 8
• A biased prior can make BayLIME’s explanation “consistently and robustly bad.”The authors caution that this possibility does not mean BayLIME is useful only when prior quality is certain; they regard priors as evidence whose trustworthiness must be considered.Source: §8 · pp. 8–9
• The authors advise against discarding priors or observations without sound reasons, while treating trustworthy evidence acquisition as a separate engineering problem.They note that Bayesian methods generally depend on good priors, not only BayLIME.Source: §8 · p. 9

9 CONCLUSIONS AND FUTURE WORK

• Theoretical analysis and extensive experiments indicate that BayLIME improves consistency, robustness to kernel settings, and explanation fidelity.The conclusion describes BayLIME as a principled Bayesian method for integrating useful prior knowledge into LIME.Source: §9 · p. 9
• Future work will investigate more practical prior sources, use posterior confidence for better explanations, and develop new BayLIME use cases.The authors connect this agenda to emerging techniques for probing black-box behavior and obtaining knowledge about it.Source: §9 · p. 9

A APPENDIX: SUPPLEMENTARY MATERIALS FOR RQ3

• The appendix presents supplementary RQ3 experimental design details and results, with experiments run on a 64-bit MacBook Pro with an 8-core 2.3 GHz Intel i9 CPU and 16 GB RAM.Source: A APPENDIX: SUPPLEMENTARY MATERIALS… · p. 11

A.1 RQ3.A: PRIOR KNOWLEDGE FROM OTHER XAI METHODS

• GradCAM’s pixel heatmap is segmented into superpixels, whose pixel importances are averaged and scaled to the coefficient-vector scale for BayLIME’s prior mean.The scaling uses maximum absolute value; the example uses GradCAM-derived prior parameters while fitting α from data.Source: §A.1 · p. 11
• GradCAM feature maps act like equivalent perturbed samples, and their distance-related weights inform the prior estimates and precision λ.The appendix describes λ as a weighted pseudo-count of prior samples and calculates it from input-image and final convolutional-map dimensions.Source: §A.1 · p. 11
• For feature selection, BayLIME uses all m features when n > m and the top five when n ≤ m.Source: §A.1 · p. 11
• GradCAM generally performs worse than the other explainers at larger sample sizes (n > 100), especially on ResNet50; the authors conjecture that upsampling artifacts and signal loss contribute.They point to ResNet50’s 7 × 7 → 224 × 224 upsampling as requiring a relatively more precise process than for InceptionV3 and Xception.Source: §A.1 · p. 11
• BayLIME improves fidelity over LIME while adding negligible time; Figure 6 evaluates deletion and insertion for four explainers on three ImageNet CNNs across sample sizes.The appendix says BayLIME’s runtime is effectively the sum of running GradCAM for priors and LIME for surrogate fitting, with only slightly more time than LIME.Source: §A.1 · pp. 11–12, Figure 6 · p. 12
• With default hyperparameters, kernel SHAP performs noticeably worse than LIME and BayLIME in both fidelity metrics and computational time across these experiments.The authors omit kernel SHAP examples from Figure 7 for this reason.Source: §A.1 · p. 12

A.2 RQ3.B: PRIOR KNOWLEDGE FROM V&V METHODS

• The trigger-prior pipeline turns NeuralCleanse reversed triggers into RBF-interpolated heatmaps, segments the image, and converts the result into superpixel priors.The experiments use 1 × 1 and 2 × 2 square superpixels for the two infected CNNs, respectively.Source: §A.2 · pp. 12–13
• The prior precision λ is set to the number of class-wise L1 trigger norms used in NeuralCleanse’s anomaly-detection distribution.Source: §A.2 · p. 13
• AMD complements IoU by measuring geometric closeness to ground truth, which IoU ignores when highlighted regions do not overlap.AMD uses the centers of the explainer’s top-k features and normalizes distance to [0, 1], regardless of image dimensions.Source: §A.2 · pp. 13–14
• On 500 backdoor-attacked images, BayLIME has the best IoU scores for both infected models; for AMD, it outperforms LIME but not the NeuralCleanse prior.The Table 4 caption identifies IoU as higher-is-better and AMD as smaller-is-better; the authors note NeuralCleanse’s minimal trigger can be close to ground truth while missing overlap.Source: §A.2 · pp. 13–14, Table 4 · p. 14

B AN INCONSISTENCY METRIC WITH MORE EXPERIMENTS TO RQ1

• For each feature, normalized importance and rank across repeated explanations form probability distributions, while the index of dispersion measures rank variability.The repeated-explanation count should be large enough for these distributions to converge for more accurate measurements.Source: B AN INCONSISTENCY METRIC WITH MORE… · p. 14
• The inconsistency measure is a weighted sum of each feature’s rank dispersion, with more important features receiving greater weight.The authors choose this intuitive metric to complement Kendall’s W and double-check experimental conclusions, not to justify one particular metric.Source: B AN INCONSISTENCY METRIC WITH MORE… · p. 14
• The new metric complements Kendall’s W by distinguishing explanations with identical feature ranks but different importance and reducing the effect of rank fluctuations among less-important features.Figures 9 and 10 show further experiments with both measures, which the authors say confirm their RQ1 conclusions.Source: Figure 9 · p. 15, Figure 10 · p. 15, B AN INCONSISTENCY METRIC WITH MORE… · p. 15

C PRELIMINARIES ON LIME

• LIME perturbs an instance, probes the black-box model, weights samples by proximity, and fits an interpretable local surrogate whose coefficients indicate feature importance.The absolute coefficient values are used to rank features.Source: C PRELIMINARIES ON LIME · pp. 15–16