Source-linked AI summary
Communicating Credit Risk with Large Language Models: Evaluation of Explanations from Standard and Alternative Data-Based Models
Sahab Zandi, Noah Kostesku, Christophe Mues, María Óskarsdóttir, Cristián Bravo
TL;DR
Credit decisioning is a high-stakes risk assessment involving professionals, auditors, regulators, and affected customers. The paper studies LLM-based explanation layers using Freddie Mac loan-level data and finds that explanation-quality components and stakeholder standards differ substantially.
Problem
Credit decisioning is a high-stakes risk assessment involving risk professionals, auditors, regulators, and affected customers.
Method
The paper uses a framework and Freddie Mac single-family loan-level data to study LLM-based explanation layers for credit risk models.
Results
The first component of decomposed explanation quality dominates the second by roughly an order of magnitude, while stakeholder acceptance criteria differ.
Takeaways & Limitations
Directional error is identified as the governance-relevant failure mode in regulated credit communication, and professional raters apply stricter standards than non-professionals.
Takeaways & Limitations
The human-evaluation finding is within the bimodal pipeline and may not generalise uniformly to single-modality pipelines.
Abstract
from arXiv · showhide
Credit decisioning is a high-stakes task in which model outputs must be accurate and explainable to support compliant decisions. Although modern credit risk models such as eXtreme Gradient Boosting (XGBoost) and Graph Neural Networks (GNNs) improve predictive performance, their explanations are often too technical for stakeholders creating communication gaps that can shape approvals, denials, and fairness judgments. We examine whether Large Language Models (LLMs) can serve as explanation layers that translate post-hoc explanation artefacts into stakeholder-appropriate risk narratives. Using Freddie Mac single-family loan-level data, we develop three pipelines: standard tabular (XGBoost + SHAP), and two with alternative data, a pure network-based (GNN + GNNExplainer), and a bimodal one (combining tabular and network data). We generate narratives with three LLM configurations: a small fine-tuned LLM (Gemma 3 4B), a large fine-tuned LLM (DeepSeek R1 70B), and a zero-shot commercial LLM (Gemini 2.5). Explanation quality is evaluated through automated checks across all pipelines and a human study of bimodal explanations comparing credit risk professionals and non-professionals on eight decision-relevant dimensions. We have three main findings. First, the pipeline accounts for higher variance in evidence-grounding scores than the language model, meaning that the binding constraint on explanation quality is the evidence representation, not the model used. Second, the explanation narratives reliably name the influential factors but are less reliable when stating the direction of influence, which may be consequential for adverse-action communication. Finally, professionals apply stricter evidentiary standards than non-professionals. We discuss implications for the governance of risk models, including deployment considerations and the value of domain-aligned LLMs in regulated credit settings.
1. Introduction
Credit decisioning requires explanations that are both accurate and usable, but complex models and technical post-hoc artefacts create a communication gap. The paper evaluates LLM explanation layers across evidence modalities and finds that evidence representation and stakeholder standards are central to explanation quality.
- Credit decisions affect household welfare, institutional losses, approvals, denials, disputes, and compliance exposure, making explanation quality consequential.
- Complex machine-learning models can improve predictive performance while reducing transparency, motivating explanation methods and communication layers.
- The paper studies whether LLMs can translate SHAP and GNNExplainer artefacts into stakeholder-appropriate narratives across tabular, network, and bimodal pipelines.
- The framework uses Freddie Mac loan-level data, XGBoost plus SHAP, GNN plus GNNExplainer, a bimodal pipeline, and three LLM configurations.
- Automated checks and human ratings compare explanation grounding, trustworthiness, communicability, usability, and professional versus non-professional evaluation.
- The pipeline explains roughly an order of magnitude more variance than the language model, while directional errors are more governance-relevant than omissions and professionals apply stricter criteria.
2. Background and Related Work
Prior work establishes tensions among predictive performance, transparency, relational modelling, and stakeholder-appropriate communication. It motivates evaluating LLM-generated narratives for evidence fidelity, auditability, and audience-specific usefulness.
- Higher-capacity credit models can improve predictive performance while decreasing transparency, increasing reliance on post-hoc explanation methods.
- Technical transparency does not guarantee useful communication because explanation quality depends on stakeholder needs and context.
- Relational credit-risk models represent shared contextual factors and may capture dependencies that tabular models miss, but they make interpretation more complex.
- GNNExplainer is used as a widely adopted instance-level baseline, while converting graph explanations into narratives risks exceeding structural evidence.
- Governance requires evaluation beyond fluency, including fidelity, trustworthiness, auditability, and controls such as prompting rules, output constraints, and versioning.
- LLMs can verbalise technical artefacts into audience-appropriate narratives, but fluent outputs may overstate, omit, or mischaracterise underlying evidence.
- Professionals prioritise evidentiary adequacy and traceability, whereas non-professionals weight clarity and completeness, motivating cohort-based evaluation with automated grounding checks.
3. Methodology
The methodology separates prediction, post-hoc explanation, and LLM-based narration across tabular, network, and bimodal evidence modalities. It evaluates these pipelines using Freddie Mac loan data, predictive models, explainers, structured evidence blocks, and multiple LLM configurations.
- 3.1. Overview of explanation architecture: The framework separates prediction, explanation generation, and narrative generation to compare how evidence types affect interpretability.The LLM acts as a verbalisation layer and does not alter predictive outputs or raw attribution values.
- 3.3. Network construction: The network construction links loans sharing a zip-code prefix, lender, or both, creating bidirectional cliques with one-hot-encoded edge types.Graphs use origination-stage attributes and fixed time windows to maintain temporal separation and avoid outcome leakage.
- 3.4. Predictive models and post-hoc explainers: The tabular pipeline combines XGBoost predictions with per-instance SHAP attributions, while the network pipeline uses a GAT with GNNExplainer evidence.GNNExplainer provides influential node features and a compact explanatory subgraph for local decisions.
- 3.6. Large language models: Narratives are generated with Gemma 3 4B, DeepSeek R1 70B, and Gemini 2.5, spanning small fine-tuned, large fine-tuned, and zero-shot configurations.Structured evidence is enriched through feature ranking, percentile mapping, and short descriptive phrases before prompting.
- 3.7. Proposed explanation pipelines: The bimodal pipeline runs XGBoost and GAT in parallel and asks the LLM to synthesize SHAP and GNNExplainer evidence into one narrative.The design preserves evidence provenance and avoids unsupported causal claims.
4. Results
The results compare how tabular, network-based, and bimodal evidence shape explanation fidelity across LLM configurations and human evaluations. Bimodal explanations balance both evidence modalities, while narratives more reliably identify influential factors than their directions and professionals apply stricter standards.
- Automated evidence alignment: Tabular pipelines score highly on tabular fidelity but near the lower bound on network metrics.This pattern is consistent with the absence of relational evidence in tabular pipelines.
- Automated evidence alignment: Network pipelines show the complementary pattern, with very low tabular metrics but stronger coverage and consistency for network features.Across all three LLMs, tabular metrics are approximately 1, while network evidence is more effectively referenced and characterised.
- Automated evidence alignment: In the bimodal pipeline, TFC reaches 4.73 while TDC stalls at 3.67, indicating that narratives name relevant risk factors more reliably than their direction of influence.This directional weakness may matter when explanations communicate whether factors increase or decrease risk.
- Variance decomposition: Network directional consistency distributes variance more evenly across pipeline, LLM, and their interaction.Pipeline×LLM interactions are significant for all metrics, so LLM differences depend on the available evidence type.
- Human evaluation: CRPs distinguish LLM explanation styles more clearly and apply stricter evidentiary standards than NCRPs.CRP–NCRP differences are more apparent in operational dimensions such as trustworthiness and usability, while overall cohort differences are not consistently directional.
- Human evaluation: Human ratings are positive overall, with systematic but modest cohort differences and generally overlapping uncertainty across LLMs.Gemma 3 receives the highest mean ratings across all eight CRP metrics and seven of eight NCRP metrics, except NCRP satisfaction, where DeepSeek R1 scores highest.
- Linguistic analysis: Higher-rated explanations use more neutral tone and greater numeric density across several perceived-quality dimensions.Trustworthiness is associated with higher neutral sentiment (d = 0.24), lower positive emotional language (d = −0.26), and numeric density effects of approximately d ≈0.05–0.16.
5. Discussion
The discussion identifies evidence modality as the dominant determinant of automated explanation fidelity, while LLM choice has a secondary, pipeline-dependent role. It also links domain alignment with perceived operational adequacy and highlights risks from unverifiable network-based narratives.
- 5.1. Impact on credit risk assessment and communication: Evidence modality explains substantially more variance in feature coverage and directional consistency than LLM choice.LLM choice remains statistically significant but secondary, with its strongest role in network directional consistency.
- 5.2. Model and domain alignment: Fine-tuning advantages depend on the pipeline: fine-tuned LLMs perform most clearly in tabular settings, while the zero-shot LLM remains competitive for bimodal and network evidence.The comparison is therefore conditional on the evidence modality supplied to the explanation layer.
- 5.2. Model and domain alignment: Domain alignment is associated with stronger perceived operational adequacy, particularly among credit risk professional evaluators.Non-professionals also tend to rate Gemma 3 higher in communicability and usability in some settings.
- 5.3. Limitations and governance: Network-based narratives can drift toward plausible but unverifiable contextual statements, limiting confidence in their evidentiary grounding.This boundary is especially relevant when extending findings from the bimodal study to single-modality pipelines.
Appendix E), which motivates tighter grounding constraints in regulated settings.
Explanation quality depends on evidence representation, LLM configuration, and evaluator standards, with directional accuracy emerging as the key governance concern. Domain-aligned systems and auditable grounding are particularly important for regulated credit communication.
- Governance-relevant failure modes: Directional errors are more consequential than omissions because naming a correct variable with an inverted sign can breach adverse-action communication requirements.Fluent evaluation may miss this failure when readers cannot inspect the underlying attribution.
- Evidence and model effects: A 4-billion-parameter model matched or exceeded a 70-billion-parameter model and a commercial API on perceived explanation quality.This suggests that frontier-scale models are not required for adequate explanation quality in this task.
- Human evaluation: Higher-rated explanations tend to use neutral language and concrete numeric detail, while affective framing can increase communicability but weaken trustworthiness and operational adequacy.The reported linguistic associations are modest, and domain alignment is linked to factual, evidence-grounded statements.
- Limitations and practice: The study is limited to consumer credit in a single jurisdiction, while outputs are sensitive to prompt phrasing, stochastic decoding, provider updates, shared training distributions, and evaluator heterogeneity.These limitations motivate continued governance oversight and broader evaluation across jurisdictions, products, languages, and stakeholder groups.
- Human evaluation: Credit risk professionals apply stricter evidentiary and operational standards, whereas non-professionals place greater weight on narrative clarity and communicability.Non-professionals favored narrative context even when evidentiary linkage was weaker, while professionals valued precise numerical justification.
- Evidence and model effects: Evidence modality is the dominant driver of automated fidelity, while both pipeline and LLM choice contribute to explanation quality.The evidence supplied to the explainer fundamentally shapes fidelity, so LLM-level comparisons require caution.
Appendix B. Mathematical definitions of explainability methods
Appendix B formalizes SHAP and GNNExplainer as the post-hoc explanation methods underlying the paper’s tabular, network-based, and bimodal pipelines. It also documents prompt components that translate their evidence into constrained narratives.
- SHAP: SHAP decomposes a predictive model’s output into a baseline expected output and feature-level Shapley contributions.TreeSHAP computes these contributions without exponential cost.
- GNNExplainer: GNNExplainer identifies a subgraph and feature mask that preserve a target node’s prediction.Its differentiable objective combines cross-entropy with sparsity regularisation for the feature and adjacency masks.
- Prompt architecture: The pipelines pass predicted outcomes and SHAP and/or GNNExplainer evidence blocks to the LLM alongside structured reasoning instructions.Prompts also include guided subgoals, counterfactual framing, and explicit output constraints.
- Prompt architecture: Shared prompt constraints require neutral, evidence-based language, prohibit unsupported factors and causal guarantees, and enforce output format limits.These rules are instantiated across tabular-based, network-based, and bimodal examples.
- Network evidence: The network example foregrounds top node features and edge connections, including geographic-area, area-provider, and same-provider relationships.The instructions require plain-language explanation of the predicted outcome and at least two connection types without introducing unseen attributes.
Appendix D. Supplementary Methods and Automated Proxy Results
Appendix D supplements the main evaluation with statistical specifications for automated proxies and human ratings. The analyses emphasize effect sizes, uncertainty, repeated measures, rater heterogeneity, and multiplicity control.
- Automated evaluation: The automated fidelity measures cover tabular feature coverage, tabular directional consistency, network feature coverage, and network directional consistency.These metrics are paired with simulated stakeholder ratings under CRP- and NCRP-oriented rubrics.
- Human evaluation: Human evaluation is restricted to bimodal explanations and summarizes participant-level 1–5 Likert means for each metric and LLM.Mixed-effects models include participant random intercepts and fixed effects for cohort, LLM, and metric.
- Multiplicity control: Holm-corrected paired comparisons control family-wise error for targeted contrasts, while Benjamini–Hochberg controls false-discovery rates for broader comparisons.The latter is used because many simultaneous comparisons make family-wise-error methods substantially more conservative.
- Reliability analyses: Variance analyses use random-intercept models for participant and explanation-case components, while linguistic differences between highly and poorly rated explanations are summarized with Cohen’s d.Aggregated participant-level means are treated as approximately continuous in mixed designs.
D.2. Fluency via Perplexity Scoring
The fluency analysis uses perplexity as a comparative proxy under a fixed GPT-Neo-125M evaluator. Results show that LLM identity explains more perplexity variation than pipeline configuration, with interactions between them.
- Results: Averaged across LLMs, tabular explanations have the lowest perplexity, network-based explanations are slightly higher, and bimodal explanations are highest.The ordering depends on the LLM configuration.
- Results: DeepSeek R1 achieves the lowest perplexity within each pipeline, while Gemini 2.5 has substantially higher perplexity, especially for bimodal explanations.The results suggest that combining evidence sources can increase linguistic variability for some LLMs.
- Variance analysis: The LLM main effect explains the largest share of perplexity variance, while pipeline configuration has a smaller statistically significant effect.A significant interaction means relative fluency differences between LLMs depend on the pipeline.
- Interpretation: Perplexity is interpreted as relative linguistic regularity under a fixed evaluator rather than a definitive measure of writing quality.Domain terminology, numeric content, explanation length, and fine-tuning can affect absolute perplexity.
D.3. Simulated Ratings via Claude Sonnet 4.6
Simulated Claude Sonnet 4.6 ratings compare explanations through CRP-oriented evidence and operational relevance versus NCRP-oriented clarity and helpfulness. Pipeline effects are large, with network-only explanations weakest under CRP criteria.
- Overall patterns: NCRP-oriented ratings are consistently higher than CRP-oriented ratings across metrics, particularly for clarity and surface fluency.The rubrics respectively emphasize communicative quality versus evidence and compliance.
- CRP ratings: Under CRP-oriented ratings, tabular-based and bimodal pipelines generally outperform network-based pipelines.The difference between tabular-based and bimodal pipelines is modest and metric-dependent.
- CRP ratings: Network-based pipelines consistently receive the lowest CRP ratings across LLMs.Relational evidence is comparatively difficult to operationalize in ways aligned with evidence-driven criteria.
- NCRP ratings: NCRP ratings show stronger LLM and interaction effects on communicative dimensions such as UND, COM, and USB.These patterns were evaluated with separate two-way ANOVAs for CRP and NCRP ratings.
- Overall patterns: Pipeline effects are large for both CRP- and NCRP-oriented ratings, indicating that evidence modality is the main driver of perceived quality.Pipeline-by-LLM interactions also occur, while LLM main effects are smaller overall.
- Caveats: Simulated ratings are treated as complementary proxies because Claude Sonnet 4.6 may contribute stylistic preferences, with the human survey prioritized as the primary reference.The simulated study uses mean ratings with 95% confidence intervals across pipeline–LLM configurations.
E.1. Tabular-Based Pipeline Example
The tabular example emphasizes loan-level attributes, while network-based explanations add relational context from neighboring loans and clusters. These narratives are clear but can oversimplify borderline trade-offs or become more abstract than feature-based explanations.
- Tabular explanations emphasize loan-level attributes such as credit score and debt-to-income ratio.
- 97% approval example attributes the outcome primarily to a credit score of 744 and lower-than-usual monthly debts.
- The example also presents applying alone and choosing a shorter loan term as positive factors for approval.
- Network narratives reference neighboring loans, provider or geographic clusters, and group-membership effects as supporting evidence.
- Network-based explanations can be more abstract than feature-based explanations, while clear outputs may oversimplify trade-offs in borderline cases.
E.3. Bimodal Pipeline Example
The bimodal example combines tabular and relational evidence in one narrative. When the signals align, the explanation is more holistic; when they conflict, careful structuring is required to avoid contradictions.
- Bimodal explanations integrate tabular and relational evidence into a single narrative.
- When tabular and network signals align, the resulting rationale is more holistic; conflicting signals require careful structuring to avoid contradictions.
- The Gemini example reports a 75% approval chance and highlights an 809 credit score alongside a debt-to-income ratio above 40% of other applicants.
- The example combines strong credit and down-payment factors with positive local connections, totaling five connections across geographic and lender-type relationships.
- The narrative recommends paying down debts over 6–12 months to improve the debt-to-income ratio.
- The combined assessment describes a strong financial foundation and positive community ties as supporting confidence in the assessment.
Appendix F. Participant demographics and additional survey information
The appendix reports supplementary statistics about the survey population used in the main text's Section 4.3. Differences in participant counts across figures reflect optional demographic questions.
- The appendix reports supplementary statistics about the survey population used in Section 4.3 of the main text.
- Differences in the numbers of credit risk professionals and non-professionals across figures result from optional demographic questions.
- Figure F.1 presents demographic and background characteristics of survey participants.