PaperlayerView exact revision on arXiv

Source-linked AI summary

Scalable Clinical Data Infrastructure and Comparative ML Evaluation for Hospitalisation Risk Prediction in Elderly Patients with Multiple Long-Term Conditions using CPRD

Asra Aslam, Volodymyr Chapman, Maurice M. O'Connell, Aseel S. Abuzour, Michael Abaho, Danushka Bollegala, Gary Leeming, Eduard Shantsila, Andrew Clegg, Lauren E. Walker, Iain Edward Buchan, Samuel D. Relton

arXiv:2608.29419v1cs.LGcs.AI

TL;DR

The paper addresses limited evidence that complex deep learning improves risk prediction over interpretable models in sparse, real-world primary-care EHRs. It builds a CPRD patient-timeline pipeline and benchmarks TG-CNN, LASSO, and Random Forests for 12-month emergency hospitalisation prediction. LASSO provides the strongest held-out discrimination and the most deployable calibration profile, supporting evaluation beyond AUC-ROC alone.

  • Problem

    Evidence that deep learning outperforms simpler interpretable models remains limited for large-scale, sparse, long-horizon primary-care EHR data.

  • Method

    The study builds a CPRD patient-timeline pipeline and compares TG-CNN with LASSO Logistic Regression and Random Forests for 12-month all-cause emergency hospitalisation prediction.

  • Results

    LASSO achieved the highest held-out discrimination, while Random Forest and TG-CNN showed calibration limitations requiring post-hoc correction or remaining substantially miscalibrated.

  • Takeaways & Limitations

    AUC-ROC alone is insufficient for clinical model selection; calibration, interpretability, and rigorous baseline comparison should inform deployment decisions.

  • Takeaways & Limitations

    The study evaluates only all-cause hospitalisation and does not conduct subgroup or fairness analyses across characteristics such as ethnicity, deprivation, or age band.

Abstract

from arXiv · show

Deep learning architectures are increasingly proposed for patient trajectory modeling in electronic health records (EHRs), yet their advantage over simpler, more interpretable models is rarely subjected to rigorous empirical scrutiny in real-world clinical settings. We present a comprehensive patient timeline pipeline applied to elderly patients in CPRD Aurum, incorporating 260 clinical conditions classified via a three-tier automated framework including specialised detection logic for 17 complex conditions. Using this infrastructure, we benchmark Temporal Graph Convolutional Neural Networks (TG-CNN) against Logistic Regression with LASSO regularisation and Random Forests for predicting 12-month all-cause emergency hospitalisation risk, motivated by (but not filtered to) the elevated risk of adverse drug reactions. Under cross-validation, TG-CNN achieves a marginally higher mean AUC-ROC than LASSO (0.712 vs. 0.705), whereas on the held-out test set LASSO achieves the highest discrimination of three models (AUC-ROC 0.733, versus 0.710 for Random Forest and 0.702 for TG-CNN). We show, that discrimination alone is an incomplete criterion for clinical deployment: after Platt calibration, LASSO is the only model with an acceptable calibration slope (0.817), while Random Forest (0.759) and, TG-CNN (0.391) remain substantially miscalibrated. We argue that LASSO, not the highest-discriminating model, is the model best suited to direct clinical deployment. We present lessons for the machine learning and healthcare community regarding data infrastructure, model selection, and value of calibration and interpretability in high-stakes decision support.

1 Introduction

The paper tests whether temporally complex deep learning improves hospitalisation-risk prediction over interpretable baselines in elderly CPRD patients. It argues that deployment decisions must consider calibration and interpretability alongside discrimination.

  • Motivation: MLTC patients face treatment burden and heightened harm risk from complex drug-drug and drug-disease interactions, motivating improved risk stratification.Structured Medication Reviews exist, but patient selection remains subjective and inconsistent.
  • Study aim: The study applies a patient timeline pipeline to CPRD elderly patients and compares TG-CNN with LASSO Logistic Regression and Random Forests for 12-month all-cause emergency hospitalisation.The outcome excludes planned or elective admissions and is motivated by, but not filtered to, adverse drug reactions.
  • Contributions: The infrastructure classifies clinical conditions through a three-tier framework, including specialised detection logic for 17 complex conditions.The benchmark uses a large-scale CPRD setting and cross-validation for models.
  • Findings: Interpretable models can match or exceed temporally complex deep learning on sparse, irregular, high-dimensional clinical data.The paper cautions that theoretical architectural advantages do not automatically translate into empirical gains on routine EHR data.
  • Clinical deployment: Calibration matters for deployment because systematically inflated risk scores cannot be communicated directly to clinicians without post-hoc correction.The paper identifies calibration as a necessary complement to discrimination when selecting clinical prediction models.
  • Implications: Interpretability and rigorous comparison against simpler baselines provide clinical value beyond marginal performance differences.The paper specifically connects explainable predictions and baseline comparison to deployability in healthcare decision support.

2 Related Work

Related work shows strong interest in deep learning for longitudinal EHR modelling, but evidence of superiority in sparse, long-horizon primary-care data remains limited. Prior work also motivates comparing complex models with interpretable alternatives and evaluating calibration alongside discrimination.

  • Existing risk tools: Existing risk tools largely target single endpoints rather than the dynamic longitudinal interplay of conditions, medications, and outcomes.This leaves temporal information in EHRs underused for risk stratification and treatment optimisation.
  • Deep learning for EHRs: Deep learning methods such as RETAIN, Med2Vec, BEHRT, and MedBERT model longitudinal clinical codes, visits, temporal positions, or attention patterns.These approaches have reported strong results across several EHR prediction settings.
  • Evidence gap: Evidence of deep learning superiority remains limited for large-scale, sparse, primary-care EHRs with irregular events and long observation windows.Many prior results come from dense inpatient or secondary-care datasets such as MIMIC-III.
  • Transferability: TG-CNN’s clustering performance on an intermediate-complexity sequential dataset does not necessarily predict discrimination on sparse primary-care binary prediction tasks.The paper therefore treats its CPRD evaluation as an empirical test rather than assuming transfer from clustering results.
  • Interpretability: High-stakes healthcare decisions motivate preferring interpretable models unless complex models demonstrate a decisive performance advantage.This literature frames the paper’s comparison between TG-CNN and interpretable baselines.
  • Evaluation criteria: Calibration should complement discrimination because clinicians and patients act on absolute risk estimates rather than rankings alone.The paper positions its results as an empirical case study of this tension in primary-care EHR prediction.

3 Methodology

The methodology builds a reusable CPRD patient-timeline infrastructure and transforms longitudinal records into temporally structured inputs for model training. Conditions are detected through tiered clinical logic, while TG-CNN receives event sequences with learnable temporal decay and prospectively defined labels.

  • Data preparation: GCAF supports comprehensive CPRD codelist development by combining automation with trusted sources such as CALIBER and the Quality Outcomes Framework.The framework addresses the time burden of traditional manual codelist review.
  • Data preparation: The Patient Timeline pipeline transforms CPRD tables into structured timelines containing medical events, conditions, drug prescriptions, and patient characteristics.Source tables include clinical observations, prescriptions, demographics, and patient identifiers.
  • Condition detection: Conditions are classified into simple, intermediate, and complex tiers according to diagnostic complexity and required temporal or clinical logic.Intermediate examples combine diagnostic codes with laboratory results, while complex conditions use specialised logic functions.
  • Condition detection: Complex-condition detection first checks explicit diagnostic codes, then applies threshold-based criteria, temporal aggregation, or audit-score evaluation.The approach provides reusable detection logic for 17 complex conditions.
  • Data preprocessing: The preprocessing workflow retrieves raw data, standardises SNOMED to medcodes, prepares codelists, and processes simple, intermediate, and complex conditions.Its output is a unified dataset containing patient IDs, dates, events, medications, and condition labels.
  • Training preparation: Raw events and labels are merged, dates are converted to timestamps, and identifiers are integer-encoded for matrix-based computation.The preprocessing operates in training and test modes and uses distributed data handling for large datasets.
  • Temporal representation: Temporal edge weights use G(i, j, k) = exp(−γt), where elapsed time t is weighted by a learnable parameter γ optimized with the network.Event dates are converted to integer days from 1 January 1955 to establish a common temporal scale.
  • Outcome labelling: Each patient receives a fixed 1 April 2019 index date, with predictors using records through 31 March 2019 and labels covering emergency admissions through 31 March 2020.No event on or after the index date contributes to predictor construction.

4 Cohort

The cohort uses elderly CPRD Aurum patients and combines demographic, clinical, medication, and longitudinal event data. The infrastructure addresses sparse, irregular primary-care timelines across many possible codes.

  • Patients aged 65 years or older were drawn from CPRD Aurum, with predictors derived from clinical records available by the observation cutoff.
  • Raw data covered demographics, clinical observations, prescriptions, and linked hospitalisation outcomes, with prescriptions mapped to BNF medication categories.
  • Features included demographics, condition indicators, medication categories, and interactions between high-risk conditions and medication classes.
  • Primary-care timelines are sparse because patients activate only a small fraction of hundreds of possible clinical codes over long observation windows.
  • The cohort and pipeline summary reports the dataset characteristics used to contextualise the modelling results.

5 Results

The study evaluates model performance using large-scale cross-validation and held-out testing, alongside calibration and practice-level diagnostics. Results show that simpler models can match or exceed TG-CNN, while calibration differentiates deployment readiness.

  • Quantitative Results: 0.734 was the highest cross-validation AUC-ROC, achieved by Random Forest on combined demographic and event features, versus 0.712 for TG-CNN.
  • Quantitative Results: N = 570,125 patients formed the training cohort for 5-fold cross-validation of 12-month all-cause hospitalisation prediction.
  • Quantitative Results: TG-CNN requires temporal event sequences, so the demographics-only comparison used a structurally matched static neural-network baseline.
  • Qualitative Analysis: Platt calibration was applied to held-out test predictions, with inverse-frequency class weighting used for all three final models.
  • Qualitative Analysis: LASSO probabilities tracked observed event rates near the ideal diagonal and showed reasonably consistent performance across primary-care practices.
  • Qualitative Analysis: Random Forest overestimated hospitalisation risk across risk deciles, despite achieving the best discrimination with AUC-ROC 0.734.
  • Qualitative Analysis: Figure 2 combines predicted-versus-observed calibration plots with GP-stratified C-statistic plots, marking practices outside the confidence envelope.
  • Qualitative Analysis: TG-CNN showed a similar or more pronounced pattern of risk overestimation, while its practice-level C-statistics had a wide spread.

6 Discussion

The benchmark shows that TG-CNN’s theoretical temporal advantages did not yield clear empirical superiority over simpler baselines in sparse primary-care EHR data. LASSO offers strong deployment advantages, while generalisability and subgroup fairness remain important boundaries.

  • Model interpretation: TG-CNN’s negative result is specific to this architecture, not temporal deep learning architectures generally.The authors selected TG-CNN for computational feasibility on health-system-scale hardware and note competitive performance in comparable EHR settings.
  • Model interpretation: Sparse, irregular, heterogeneous primary-care timelines may neutralise temporal weighting benefits designed for denser sequential data.TG-CNN also has more parameters than logistic regression and may require more data to learn effectively.
  • Clinical implications: LASSO combines competitive predictive performance with rapid retraining, transparent coefficients, and outputs traceable to conditions and medication categories.These properties align with clinicians’ reported preference for explainable, well-calibrated models that support clinical reasoning.
  • Limitations and future directions: The study’s scope is limited to all-cause hospitalisation in patients aged 65+ from a specific CPRD observation window, with incomplete capture of some secondary-care or community events.Generalisability to younger MLTC populations and different observation periods requires further study.
  • Limitations and future directions: No subgroup or fairness analysis was conducted, despite known disparities in EHR data quality and coding practice across patient subgroups.The authors identify this as an important direction for future work.
  • Limitations and future directions: Future work will investigate sparse long-horizon architectures, hybrid interpretable-temporal models, and trajectory-clustering approaches for patient stratification.The proposed direction includes combining LASSO-selected features with lightweight temporal encoders.

TG-CNN Architecture, Input Definitions, and Training Diagnostics

The paper defines three input configurations for comparing static and temporally encoded patient information, then reports TG-CNN weights selected at the minimum cross-validation loss. LASSO is identified as the only model both well-calibrated and directly deployable without post-hoc correction.

  • Input definitions: Events are binary prescription or condition-code features for LASSO and Random Forest, while TG-CNN additionally encodes event dates.The event inventory includes 802 BNF prescription codes and 190 unique condition codes.
  • Input definitions: Demographics comprise 19 features, including scaled age, deprivation categories, ethnicity categories, and sex.Age is scaled as (age-65)/25, while missing IMD values receive a one-hot category.
  • Input definitions: Events & Demographics uses both event and demographic data for model development.
  • Training diagnostics: TG-CNN training results use model weights from epoch 1, where cross-validation loss reached its minimum.The training curves mark this early-stopping point with dashed annotations.
  • Training diagnostics: LASSO is the only evaluated model described as both well-calibrated and directly deployable without post-hoc correction.

Feature Inventory

The feature inventory consolidates clinical conditions, prescriptions, and demographics for the models, with condition coding reduced to 190 final labels after merging related or redundant categories. The supplied figure passages identify TG-CNN architecture and training configurations but provide no additional inventory encoding details.

  • Feature inventory: The final feature space combines clinical condition codes, BNF prescription codes, and demographic features.The original condition list comprised 260 conditions before codelist construction and consolidation.
  • Architecture and training figures: Figure 3 presents the TG-CNN architecture with dual-layer configuration flexibility.
  • Architecture and training figures: Figure 4 covers TG-CNN training for demographics, events, and combined events with demographics, marking epoch 1 as the early-stopping point.
  • Feature consolidation: Nine codelists were condensed into combined labels to reduce redundancy, including cholesterol and stroke-related categories.The consolidation also merged ischaemic heart disease with its history label.
  • Feature consolidation: The final inventory contains 190 condition codes after the described consolidations.

Test-Set Discrimination and Calibration

On the unseen test partition, calibration procedures improved agreement between predicted and observed risks, while LASSO achieved the strongest combined discrimination, calibration, and prediction-error profile. Practice-level results indicate that Random Forest discrimination estimates were unstable at many low-volume, low-event practices.

  • Calibration: Post-hoc Platt scaling and isotonic regression improved agreement with observed outcome proportions across all three model architectures.Calibration transformations were fitted on a dedicated held-out 20% subset of training data to prevent leakage.
  • Calibration: Platt scaling provided TG-CNN greater overall stability than isotonic regression by avoiding a high-probability tail artifact in sparse bins.
  • Model comparison: LASSO achieved AUC-ROC = 0.733, AUC-PR = 0.380, calibration slope 0.817, and Brier score = 0.132.It was reported as the top-performing and most reliable model, with low ECE and stable calibration after Platt scaling.
  • Practice-level analysis: 42.1% of GP practices had a Random Forest C-statistic below 0.5, indicating no better-than-chance discrimination at those sites.These practices had smaller patient lists and lower hospitalisation prevalence, consistent with reduced statistical power.

Order-Shuffled TG-CNN Ablation

The order-shuffled ablation tests whether TG-CNN benefits from preserving event sequence. Event ordering improves TG-CNN relative to its shuffled baseline but does not make it outperform simpler models.

  • Ablation design: 50 independent shuffles randomised within-patient event order while preserving the same event codes on the held-out test set.TG-CNN was re-evaluated against the original-order baseline using cross-validation-selected weights.
  • Calibration context: Figure 5 compares raw predictions with Platt scaling and Isotonic regression for LASSO, Random Forest, and TG-CNN calibration curves.The figure provides the calibration context for interpreting model suitability beyond discrimination.
  • Performance boundary: TG-CNN’s unshuffled test-set AUC-ROC was 0.702, below LASSO at 0.733 and Random Forest at 0.710.Temporal sequencing improved TG-CNN over its own shuffled baseline without closing the gap to either simpler baseline.
  • Ablation findings: Shuffling increased recall, indicating that intact event order produced lower recall and more false negatives.The authors suggest temporal weighting may under-weight clinically informative events occurring earlier in a patient’s history.

Interpretability: Permutation Importance and SHAP Analysis

Interpretability analyses identify medication burden and functional status as important predictive signals, with age dominant in SHAP analysis. Random Forest additionally surfaces diagnosis-specific predictors that LASSO does not rank as prominently.

  • Interpretability methods: Permutation importance measures the mean increase in held-out-test loss after randomly shuffling feature values, with LASSO as the primary deployable model.Patterns were compared against Random Forest using permutation importance and SHAP values.
  • Permutation importance: LASSO’s top predictors include medication-related BNF codes, missing ethnicity data, and housebound status; no individual diagnosis ranks among its top 10.Medication features include oral anticoagulants, vitamin D, insulin, cholesterol, and antiplatelet drugs.
  • Permutation importance: Random Forest ranks housebound status and oral anticoagulants first and second, while also placing dementia, atrial fibrillation, and iron-deficiency anaemia among its top ten predictors.This contrasts with LASSO’s more prominent medication and functional-status pattern.
  • SHAP analysis: Age is the dominant SHAP predictor for both models, followed by medication-related and functional features.Oral anticoagulants, vitamin D, and housebound status recur in the SHAP and permutation-importance results.
  • Model configuration: The TG-CNN search used matched 5-fold cross-validation and optimized graph filters, fully connected layers, and temporal modelling strategy.The selected architecture used eight graph convolution filters and a 32-unit fully connected layer with residual connections.
  • Model configuration: TG-CNN’s final architecture concatenated outputs from all fully connected blocks before classification, allowing features from multiple abstraction levels to inform predictions.Demographic information was processed through a dedicated single-layer network.
Loading 2608.29419v1…