PaperlayerView exact revision on arXiv

Source-linked AI summary

Predicting Patient Outcomes with Graph Representation Learning

Emma Rocheteau, Catherine Tong, Petar Veličković, Nicholas Lane, Pietro Liò

arXiv:2101.03940v1cs.LG

TL;DR

ICU outcome prediction has largely underused sparse diagnoses, especially because rare disease patterns are difficult to learn from conventional late-stage feature concatenation. The paper connects similar patients in a diagnosis graph and combines LSTMs with GNNs to model temporal and neighbourhood information. LSTM-GNNs outperform LSTM-only baselines on eICU length-of-stay prediction, while gains are less consistent for in-hospital mortality.

  • Problem

    ICU outcome prediction has focused heavily on physiological time series, while sparse diagnoses are difficult to model because of combinatorial patterns and limited data for rare diseases.

    Source: §1 · p. 2

  • Method

    The paper constructs a diagnosis-based patient graph and combines LSTMs for temporal features with GNNs for patient-neighbourhood information in an end-to-end model.

    Source: §1 · pp. 2–3

  • Results

    LSTM-GNN models significantly improve over both LSTM baselines on LOS, with LSTM-MPNN surpassing LSTM by 3–15% on all LOS metrics except MAPE; IHM gains are small and statistically insignificant in AUPRC while AUROC decreases.

    Source: §5 · p. 8

  • Takeaways & Limitations

    Graph representations provide an independent and substantial LOS benefit by supplying neighbourhood context for rarer disease patterns alongside conventional diagnosis encoding.

    Source: §6 · pp. 9–10

    The paper leaves sensitivity to α, c, and k for future work and proposes extending the graph to shared medications and procedures.

    Source: §6 · p. 10

Abstract

from arXiv · show

Recent work on predicting patient outcomes in the Intensive Care Unit (ICU) has focused heavily on the physiological time series data, largely ignoring sparse data such as diagnoses and medications. When they are included, they are usually concatenated in the late stages of a model, which may struggle to learn from rarer disease patterns. Instead, we propose a strategy to exploit diagnoses as relational information by connecting similar patients in a graph. To this end, we propose LSTM-GNN for patient outcome prediction tasks: a hybrid model combining Long Short-Term Memory networks (LSTMs) for extracting temporal features and Graph Neural Networks (GNNs) for extracting the patient neighbourhood information. We demonstrate that LSTM-GNNs outperform the LSTM-only baseline on length of stay prediction tasks on the eICU database. More generally, our results indicate that exploiting information from neighbouring patient cases using graph neural networks is a promising research direction, yielding tangible returns in supervised learning performance on Electronic Health Records.

1 Introduction

Patient outcome models have often emphasized physiological time series while overlooking sparse diagnoses, whose rarity and combinatorial patterns make them difficult to model. The paper proposes representing related patients in a graph so diagnosis information can contribute relationally to outcome prediction.

  • Physiological time-series features dominate prior ICU outcome prediction, potentially missing clinically important information from sparse EHR data.Source: §1 · p. 2
  • Diagnoses are difficult to use because their many features create combinatorial comorbidity patterns and provide too little data for rare diseases.Source: §1 · p. 2
  • The common preprocessing strategy discards the diagnosis long tail and concatenates remaining features late, trading modelling difficulty against valuable information lost.Source: §1 · p. 2, Figure 1 · p. 2
  • The proposed strategy constructs a patient graph whose nodes are related patients, using similarity to exploit clinicians’ reliance on experience with similar cases.Source: §1 · p. 2
  • The hybrid LSTM-GNN uses diagnoses with data from the first 24 ICU hours to predict in-hospital mortality and length of ICU stay, and can extend to other sparse medical data.Source: §1 · p. 3

2 Related Work

Prior work provides strong temporal modelling and several graph-learning choices, but inter-patient graph sharing remains largely unexplored in clinical prediction. This work situates its approach at that intersection using a static patient graph.

  • GNNs transform node representations and aggregate information from neighbouring nodes; the paper evaluates GCN, GAT, GraphSAGE, and MPNN for patient relationships.Source: §2 · p. 3
  • LSTMs are selected for the time-series component because recurrent models have achieved state-of-the-art patient outcome results on MIMIC and eICU.Source: §2 · p. 3
  • Sequential models combined with GNNs have been explored outside healthcare, but this paper uses a simpler approach because patient diagnoses are assumed to remain relatively static during an ICU stay.Source: §2 · p. 3
  • Clinical graph research has mainly used knowledge graphs or structured EHR graphs without inter-patient sharing, while Malone et al. used a separate embedding-propagation step for missing-data imputation.Source: §2 · p. 3
  • The approach overview combines a patient graph with end-to-end LSTM-GNN encoding, then concatenates temporal, graph, and static representations for prediction.Source: Figure 2 · p. 4

3 Methods

The method constructs a diagnosis-based patient graph and processes each patient with temporal, neighbourhood, and static encoders. End-to-end training combines full-model and LSTM losses while neighbourhood sampling supports scalable learning.

LSTM-GNN architecture

  • The model concatenates LSTM, GNN, and static embeddings from each patient graph node before a fully connected layer produces per-node predictions.Source: §3 · pp. 4–5, Figure 3 · p. 5

Diagnosis Graph Construction

  • Diagnosis graph construction represents patients with multi-hot diagnosis vectors, computes pairwise similarity scores, and connects each node to k nearest neighbours.Source: §3 · pp. 4–5
  • Shared rare diagnoses receive greater similarity weight, while a penalty for total diagnoses limits highly diagnosed patients from becoming hubs.Source: §3 · p. 4
  • Alternative thresholded and BERT-based graph constructions were explored, but the presented similarity method performed best in manual inspection and preliminary experimentation.Source: §3 · p. 5

LSTM-GNN architecture

  • A bidirectional LSTM converts each patient’s time series into a temporal embedding, while the GNN locally reweights and propagates that embedding across neighbours.Source: §3 · p. 5
  • Static inputs pass through a fully connected layer, and the resulting static representation is concatenated with temporal and neighbourhood representations before prediction.Source: §3 · p. 5

Training

  • Fixed-size neighbourhood sampling enables mini-batch, scalable, inductive training, with training neighbours restricted to training nodes and testing evaluated only on test nodes.Source: §3 · pp. 5–6
  • The loss combines full LSTM-GNN and standalone LSTM losses; binary crossentropy is used for IHM and squared logarithmic error for LOS.Source: §3 · p. 6

4 Data

The study uses patient-level splits from a large multicentre eICU cohort, combining static features, hierarchical diagnoses, and hourly time-series measurements. Diagnoses are represented sparsely across hierarchy levels, while missing or irregular measurements receive explicit preprocessing.

Cohort and splits

  • 89,123 adult patients from 208 U.S. hospitals were split at the patient level into 70% training, 15% validation, and 15% testing sets.Included admissions had at least 24 ICU hours and one recorded observation; one admission was randomly selected for patients with multiple admissions.Source: §4 · p. 6

Static Features

  • 20 non-time-varying features were scaled to [-1, 1], with percentile-based boundaries, absolute cutoffs, binary coding, and one-hot categorical encodings.The preprocessing used the 5th and 95th percentiles as scaling boundaries and capped values at [-4, 4].Source: §4 · p. 6

Diagnoses

  • Diagnoses were encoded across hierarchical class levels as a 4,436-dimensional multi-hot vector averaging 99.5% sparsity.Diagnoses above 0.5% prevalence were included, with rarer diseases retained through qualifying parent classes; only diagnoses recorded before ICU hour 24 were used.Source: §4 · p. 6

Time Series

  • Eighteen time-varying features were extracted hourly during each stay and up to 24 hours beforehand, then processed like static features.Irregular observations were resampled to one-hour intervals and forward-filled.Source: §4 · p. 6

Static Features

  • Table 1 includes indicators for age values capped above 89 and for height or weight values imputed with the mean.These indicators preserve whether capping or mean imputation occurred.Source: Table 1 · p. 7

5 Results

LSTM-GNN models significantly improve length-of-stay prediction over LSTM baselines, while effects on in-hospital mortality are mixed. Ablations indicate that graph-based diagnosis representations and LSTM-based time-series processing contribute differently across tasks.

LSTM-GNN Performance

  • LSTM significantly outperforms LSTM*, confirming that including diagnoses through encoder concatenation adds predictive value to both tasks.Table 3 compares the diagnosis-free and diagnosis-processing LSTM baselines.Source: §5 · pp. 7–8, Table 3 · p. 8
  • All LSTM-GNN models significantly outperform both LSTM baselines on LOS, with LSTM-MPNN improving LOS metrics by 3–15% except MAPE.The gains mainly reflect reduced errors for long stays, explaining larger improvements in MSE than MAPE.Source: §5 · p. 8, Table 3 · p. 8
  • On IHM, LSTM-GNN models show a small statistically insignificant AUPRC increase but reduced AUROC.The result contrasts with the consistent LOS gains.Source: §5 · p. 8

Ablation Studies

  • Graph-only LSTM-GNN* models outperform LSTM* and show that patient graphs alone can represent diagnosis information informatively.The graph-only advantage is stronger for LOS than IHM, while the combined LSTM-GNN approach remains best overall.Source: §5 · pp. 8–9
  • Removing the LSTM and supplying raw time series to GraphSAGE or GAT significantly worsens performance relative to the corresponding LSTM-GNN models.This supports using an LSTM to process temporal features before graph-based prediction.Source: §5 · pp. 8–9, Table 4 · p. 8

Interpretability

  • LSTM-GAT* assigns edge attention weights that can be inspected qualitatively, emphasizing a neighbour sharing the patient’s congestive heart failure diagnosis.The example links the learned attention pattern to clinically relevant patient similarity.Source: §5 · p. 9, Figure 4 · p. 9

6 Discussion

The LSTM-GNN represents diagnoses as a patient graph and provides a substantial additional benefit when combined with an encoder on length-of-stay prediction. The graph is especially strong for LOS, while combining graph and encoder representations offers complementary information.

6 Discussion

  • Graph representation of diagnoses provides an independent and substantial performance benefit when combined with an encoder on the LOS task.The authors attribute this to complementary representations rather than replacing the encoder approach.Source: §6 · pp. 9–10
  • The encoder captures common comorbidities, whereas the graph supplies neighbourhood context for rarer disease patterns and can augment poor-quality patient data.The paper gives sepsis as an example of a common comorbidity represented by the encoder.Source: §6 · p. 10

Performance differences between LOS and IHM

  • The graph method is particularly strong on LOS, possibly because diagnoses influence operational factors such as differing discharging practices.This explanation is presented as a possible attribution, not a demonstrated causal mechanism.Source: §6 · p. 10
  • The LOS explanation does not hold for IHM, where vital signs and common diagnoses remain the most reliable mortality-risk predictors.Common diagnoses can be readily extracted by the diagnosis encoder.Source: §6 · p. 10

Future Work

  • Future work includes incorporating shared medications and procedures into similarity scores and characterising sensitivity to α, c and k.The authors frame graph representations of sparse EHR data as a potentially rewarding research direction.Source: §6 · p. 10

Dynamic LSTM-GNNs

Dynamic LSTM-GNNs construct patient graphs from temporal-feature similarity rather than diagnoses. They generally match LSTM* on IHM and offer an advantage on LOS, although the largest gains still come from diagnosis-based graph construction.

Dynamic LSTM-GNNs

  • The dynamic variant trains an LSTM on temporal features, computes pairwise Euclidean distances between batch hidden vectors, and applies k-NN to form the graph.This graph is learned from time series alone, without diagnoses.Source: Dynamic LSTM-GNNs · p. 12
  • Dynamic LSTM-GNNs generally perform similarly to LSTM* on IHM but have an advantage on the LOS task.The comparison is reported in Table 5.Source: Dynamic LSTM-GNNs · p. 12, Table 5 · p. 12
  • Dynamic LSTM-GCN* performs similarly to LSTM despite lacking diagnoses, suggesting that relating patients through a graph can help model outcomes independently of diagnoses.The authors suggest neighbouring patients may help when an individual patient's data are poor or missing.Source: Dynamic LSTM-GNNs · p. 12
  • The most visible gains still arise when diagnoses are used to construct the graph.This comparison is tied to Table 3a in the cited passage.Source: Dynamic LSTM-GNNs · p. 12

Implementation and Hyperparameter Search Methodology

The experiments used repeated random hyperparameter trials and standard deep-learning tooling. Models were trained with Adam, with a 25-epoch maximum and early stopping often ending training sooner.

Implementation and Hyperparameter Search Methodology

  • Each model underwent 10 random hyperparameter trials, with search ranges and selected values reported in the project repository.The repository is identified as the source for the detailed search configuration.Source: Implementation and Hyperparameter… · p. 12
  • Deep-learning methods were implemented in PyTorch and optimized with Adam, using PyTorch Lightning and Tune for experiment organization and hyperparameter comparison.These tools structured the experiments rather than defining the model architecture.Source: Implementation and Hyperparameter… · p. 12
  • Training allowed at most 25 epochs, although early stopping caused many models to finish earlier.The stated maximum was therefore not necessarily reached by every model.Source: Implementation and Hyperparameter… · p. 12

Related papers

Loading 2101.03940v1…