Explainable AI in Credit Risk Management

1 Introduction

• Credit scores reduce customer data to a quality measure, but complex ML predictions can be difficult to trace, raising concerns about trusting individual predictions and deployed models.The paper distinguishes trust in a specific prediction from trust that a model will behave reasonably when deployed.Source: §1 · p. 2
• German supervisory expectations and GDPR Articles 13–15 make explanations relevant to financial institutions’ decisions about individuals.The cited report says supervised entities are responsible for ensuring AI-based decisions can be explained to third parties; GDPR covers meaningful information about logic in certain solely automated decisions.Source: §1 · p. 2
• Research on explainability in finance remains limited, and prior credit-risk studies discussed here each implement only one of the two main frameworks.Bussman et al. (2020) use Shapley values with 15,000 SME companies; Miller et al. (2020) apply the Shapley method to P2P lending credit scoring.Source: §1 · p. 2
• This paper examines both SHAP and LIME for credit-risk management and outlines practical hurdles and proposed solutions in financial applications.Source: §1 · p. 2

2 Interpretability vs Explainability

• Interpretability concerns observing cause and effect within a system, whereas explainability concerns describing its internal mechanics in human terms.The paper applies the distinction by using LIME for instance-level explanations and SHAP for global explanations.Source: §2 · pp. 2–3

2.1 LIME

• LIME approximates a black-box model with a local interpretable model to explain an individual prediction, independently of the original classifier.It fits a local model using sample points similar to the instance; candidate interpretable models include linear models and decision trees, while its locality-aware loss measures approximation unfaithfulness.Source: §2.1 · p. 3

2.2 SHAP

• SHAP is a unified framework for assigning feature-importance values to a particular prediction, with a class of additive measures whose solution is unique under stated desirable properties.Its attributions describe changes in expected model prediction when conditioning on features and connect the base value E[f(z)] to the current output f(x).Source: §2.2 · pp. 3–4, Figure 2 · p. 4

3 Data pre-processing and ML models

• The experiments use over 2.2 million three-year Lending Club loans with known outcomes to classify default versus no default.The dataset contains 145 original features, including demographic and payment-behavior information, and is highly imbalanced with many empty values.Source: §3 · p. 4, §3.1 · p. 4

3.2 Data Pre-Processing

• Preprocessing removed columns missing values in over 90% of records, eliminated correlated features, recoded categorical variables, and reduced loan status to Fully Paid or Default.The resulting feature space had 86 features and approximately 700,000 records; grade levels E, F, and G were combined with D.Source: §3.2 · p. 4
• Feature engineering was minimal, with most input features retained in their original form, and the authors identify data-driven selection and engineering as future work.The paper says this limited emphasis reflects its focus on implementing post-hoc explainability techniques and documenting practical challenges.Source: §3.2 · pp. 4–5

3.3 Machine Learning Models

• The study trains Logistic Regression, XGBoost, Random Forest, SVM, and a neural network, using grid search for SVM hyperparameters.The SVM results were described as similar to less-computationally-intensive tree models; the two-layer neural network was trained for 20 epochs in about ten minutes.Source: §3.3 · p. 5

4 Using LIME to explain Local Instances

• For tabular LIME explanations, the authors created feature and target-category lists and a function returning class probabilities, passed these with training data to the explainer, then selected an instance to obtain its top 10 influential features.Source: §4 · pp. 5–6

4.1 LIME on Tree Based Models

• The Python LIME framework could not accommodate a GridSearchCV object, so the authors retrained XGBOOST using the best parameters to create an xgboost object.Source: §4.1 · p. 6

4.1.1 Interpretation

• For the XGBOOST fully-paid example, $18,127.90 in total_payment and no recoveries pushed toward Fully Paid, while loan_amnt above $10,000 and high total_rec_int pushed toward Default.The model predicted no default with absolute certainty for this instance.Source: §4.1.1 · p. 7
• In the XGBOOST default example, $3,644.24 in total interest and $450 in recoveries pushed toward Default, although the top ten features collectively favored Fully Paid.Considering all features together favored Default.Source: §4.1.1 · p. 8
• For the Random Forest fully-paid example, no recoveries, an $11,362.82 last payment, verified status, and high total payments pushed toward Fully Paid.The last payment exceeded $3,600, and total payments were $18,127.90.Source: §4.1.1 · p. 8
• In another interpreted instance, homeownership pushed the prediction toward Default, but not enough to reverse the prediction..Source: §4.1.1 · p. 8
• The authors judge tree-model explanations broadly consistent with financial logic and say they can help users trust predictions and developers identify problematic features.They cite late fees, recoveries, lower repayments, and unverified loans as high-risk signals, while no recoveries, lower interest, and higher payments indicate lower risk.Source: §4.1.1 · p. 8

4.2 LIME on SVM

• For SVM, the authors tried enabling probability output or converting decision distances through a sigmoid/softmax; only the first approach let LIME run, and its explanations were irrelevant given poor predictive performance.The paper suggests overfitting as the likely reason for SVM's lower predictive utility than the other classifiers.Source: §4.2 · p. 8

• SHAP summary plots show both the importance of features and their impact on the model output.The leading features include total_pymnt, loan_amount, and least_pymnt_amount.Source: §5 · p. 9
• Higher total_pymnt values are associated with a lower probability of default.Source: §5 · p. 9
• The feature ranking places total_pymnt first, followed by loan_amount and least_pymnt_amount.Source: §5 · p. 9

5.1.1 Interpretations

• The top five important features are total_pymnt, loan_amnt, last_pymnt_amnt, total_rec_int, and recoveries.Source: §5.1.1 · p. 10
• Higher total_pymnt and lower recoveries are associated with lower default probability, while higher recoveries are associated with higher default probability.Source: §5.1.1 · p. 10
• The SHAP feature-effect plots indicate that home ownership has no significant impact on the model output.Source: §5.1.1 · p. 10
• SHAP explanations from tree and kernel explainers show consistent top-20 features and expected impacts across samples of 100 and 2,000 test points.The authors also note that some features are non-monotonic.Source: Figure 8 · p. 9, §5.1.1 · p. 10
• In the loan_amnt dependence plot, lower amounts are associated with fully repaid loans and higher amounts with increased default probability.The plot also represents feature interactions; total_amnt is identified as the feature most interacting with loan_amnt.Source: §5.1.1 · pp. 10–11, Figure 9 · p. 10
• Force plots can explain individual predictions and stack instance explanations to show a dataset-wide view.For total_pymnt, higher payment amounts correspond to lower likelihood of a loan being tagged as defaulting.Source: §5.1.1 · p. 11, Figure 10 · p. 11

5.2.1 Model Development

• The neural-network experiments varied hidden layers, neuron counts, learning rates, and tanh or relu activations before selecting the model with the best test-set ROC_AUC.Binary cross entropy was used for the binary classification task.Source: §5.2.1 · p. 11

5.2.2 Explanations

• Deep explainers had lower computation time than kernel explainers for the neural-network classifier.The authors report that deep explanations combined values for both classes, while tree explanations provided values separately by class that were exact opposites.Source: §5.2.2 · p. 11

5.2.3 Interpretations

• On 2,000 test points, deep and kernel explainers returned considerably overlapping top-20 features, with some differences in feature positions.Both interpretations associate lower loan amounts with lower default probability and positive recoveries with higher risk; kernel explanations show smoother transitions for loan_amnt and clearer interpretations for total_rec_int.Source: §5.2.3 · p. 12, Figure 11 · p. 12
• The dependence plot indicates little interaction between recoveries and debt_settlement_flag.The authors attribute the nearly straight line to the absence of that interaction.Source: §5.2.3 · pp. 12–13, Figure 12 · p. 12

• Information gain highlights recoveries as the only significant feature, while SHAP mean absolute values show contributions spread across the top five features.Source: §5.3 · p. 13
• The comparison may not be fair because information-gain scores use the training set and SHAP explanations use 2,000 test points.Source: §5.3 · p. 13
• The comparison was undertaken to assess whether the two methods convey the same feature information and ranking.Source: §5.3 · p. 13

5.4 Accumulated Local Effects

• ALE plots were computed only for Logistic Regression, SVM, and Neural Network classifiers because tree-based models have piece-wise constant, non-differentiable predictions.Intervals were chosen by visualization to contain similar numbers of data points, and alternative interval counts were tested for plot stability.Source: §5.4 · p. 13

5.4.1 Interpretations

• The prediction decreased by approximately 0.65 points as total_pymnt rose from 0 to 40,000, pushing it toward Fully Paid.The paper interprets this as higher total payments increasing the chances of being tagged as a risk-free consumer.Source: §5.4.1 · p. 13
• As total received interest increased, the chances of being tagged as a high-risk consumer increased.Source: §5.4.1 · p. 14
• The plot’s y-axis represents each feature’s impact magnitude, and total payments had a higher impact on predictions than total interest.Source: §5.4.1 · p. 14

6 Practical Challenges

• LIME was run on an independently trained model after the best GridSearchCV hyperparameters were selected.Source: §6 · p. 14
• For SVM, the authors tried two methods to generate probabilities required by LIME.Source: §6 · p. 14
• The authors distinguished SHAP explainers that combine class values from those that do not, then generated graphs accordingly.Source: §6 · p. 14
• Kernel SHAP’s exponential computational requirements led the authors to use a 2,000-row test-set subsample.Source: §6 · p. 14
• To reduce background-data processing time, the authors used shap.kmeans to select 30 representative rows from a 20,000-point training-data subsample.Source: §6 · p. 14

7 Conclusion and Future Work

• The authors suggest wider AI adoption in credit risk management may support financial inclusion and financial system diversity.They describe lower-priced credit for some borrowers otherwise classified as subprime under traditional criteria as one possible route to inclusion.Source: §7 · p. 14
• LIME and SHAP explanations were consistent with financial logic, and the 20 most important features remained stable across changing test sizes.Source: §7 · p. 15
• SHAP values communicate feature importance but can take extremely long to generate because of exponential runtime, while LIME is limited to probabilistic models and certain model objects.Source: §7 · p. 15