Early detection of sepsis utilizing deep learning on electronic health record event sequences

Simon Meyer Lauritsen; Mads Ellersgaard Kalør; Emil Lund Kongsgaard; Katrine Meyer Lauritsen; Marianne Johansson Jørgensen; Jeppe Lange; Bo Thiesson

Source-linked AI summary

Early detection of sepsis utilizing deep learning on electronic health record event sequences

Simon Meyer Lauritsen, Mads Ellersgaard Kalør, Emil Lund Kongsgaard, Katrine Meyer Lauritsen, Marianne Johansson Jørgensen, Jeppe Lange, Bo Thiesson

arXiv:1906.02956v1Jun 7, 2019cs.LGstat.APstat.ML

TL;DR

Early sepsis detection remains difficult, and existing models have been limited by restricted clinical data and ICU-focused evaluation. This study develops a CNN-LSTM system using heterogeneous event-sequence data outside ICUs and proposes retrospective evaluation of intervention potential. The system learns representations from raw events and outperforms strong baseline models, while its evaluation accounts for antibiotics and blood cultures.

Problem
Early sepsis detection remains difficult, while existing models use restricted clinical parameters and have largely been studied in intensive care settings.
Method
The study uses a CNN-LSTM deep learning system on heterogeneous EHR event sequences and evaluates clinical utility through retrospective assessment of interventions.
Results
The system learns key-factor representations and interactions from raw event-sequence data and outperforms strong baseline models such as GB-Vital.
Takeaways & Limitations
Sequential deep learning models can detect sepsis at a very early stage using heterogeneous data outside intensive care units.
Takeaways & Limitations
The retrospective clinical-utility evaluation is conservative because it focuses on patients for whom an intervention occurs and excludes harms and benefits for noninterventions.

Abstract

from arXiv · show

The timeliness of detection of a sepsis event in progress is a crucial factor in the outcome for the patient. Machine learning models built from data in electronic health records can be used as an effective tool for improving this timeliness, but so far the potential for clinical implementations has been largely limited to studies in intensive care units. This study will employ a richer data set that will expand the applicability of these models beyond intensive care units. Furthermore, we will circumvent several important limitations that have been found in the literature: 1) Models are evaluated shortly before sepsis onset without considering interventions already initiated. 2) Machine learning models are built on a restricted set of clinical parameters, which are not necessarily measured in all departments. 3) Model performance is limited by current knowledge of sepsis, as feature interactions and time dependencies are hardcoded into the model. In this study, we present a model to overcome these shortcomings using a deep learning approach on a diverse multicenter data set. We used retrospective data from multiple Danish hospitals over a seven-year period. Our sepsis detection system is constructed as a combination of a convolutional neural network and a long short-term memory network. We suggest a retrospective assessment of interventions by looking at intravenous antibiotics and blood cultures preceding the prediction time. Results show performance ranging from AUROC 0.856 (3 hours before sepsis onset) to AUROC 0.756 (24 hours before sepsis onset). We present a deep learning system for early detection of sepsis that is able to learn characteristics of the key factors and interactions from the raw event sequence data itself, without relying on a labor-intensive feature extraction work.

1. Introduction

Sepsis is a major global health problem, and early detection remains difficult because its initial symptoms overlap with many other conditions. Prior work supports machine-learning detection from EHR data, while this study extends deep-learning approaches beyond ICUs and evaluates intervention-related clinical utility.

More than six million people die of sepsis annually, and many deaths are considered preventable.
Early detection can improve patient outcomes, but sepsis remains difficult to identify accurately and promptly.Even experienced physicians may struggle because early symptoms are nonspecific.
Machine-learning models trained on individual patient EHR data have outperformed established clinical early-warning scores such as NEWS.
The study presents a scalable deep-learning approach for early sepsis detection using heterogeneous data collected outside intensive care units.
The study proposes realistic sequence evaluation and assesses clinical utility using preceding blood cultures and intravenous antibiotics.

2.1. Data population and data sources

The study combines longitudinal secondary-sector EHR data with nationwide registers for adults in four Danish municipalities. The resulting data include raw clinical events and contextual covariates spanning multiple health-data domains.

Data covered residents aged 18 years or older in four Danish municipalities from 2010 to 2017.
Secondary-health-sector data included biochemistry, medicine, microbiology, medical imaging, and patient-administration information.
Nationwide registers supplied contextual information on diagnoses, procedures, admissions, marital status, and housing situation.
The machine-learning inputs were raw sequential health events, supplemented by covariates such as comorbidities, age, and marital status.

2.2. Inclusion criteria and dataset preparation

The dataset was formed by filtering hospital contacts and excluding short admissions and departments with very low sepsis prevalence. Eligible contacts were split into training, validation, and test sets, with positive cases oversampled during training.

From 1,002,450 hospital contacts, outpatient contacts and admissions shorter than three hours were removed before further selection.
Departments with sepsis prevalence below 2% were excluded, leaving 52,229 contacts in the dataset.
Sepsis-positive contacts were divided into training, validation, and test sets using an 80%/10%/10% split.
Sepsis-positive contacts in the training data were oversampled by a factor of ten.
Each sequence contained at most five days before its label time, with the observation and prediction windows together spanning five days.

2.3. Target definition for the early detection of sepsis

Admissions were labeled using an EHR registration based on the 2001 consensus sepsis definition. Positive labels corresponded to qualifying sepsis registrations during admission, while negative sequences used randomly selected admission times as pseudo-label times.

The definition required at least two SIRS criteria together with suspected infection.
The SIRS criteria included abnormal heart rate, temperature, respiratory rate or PaCO2, and white-cell count.
For positive sequences, the label time was sepsis onset; for negative sequences, it was randomly selected within the admission.
A hospital admission was labeled sepsis-positive when its EHR contained a registration fulfilling the 2001 consensus sepsis definition.
The sepsis registration could occur before vital-sign measurements were entered because it was recorded continually during admission rather than only at discharge.

2.4. Data representation

The study represents each patient as an ordered sequence of EHR events, vectorizes event values, aggregates them over time, and adds patient context separately.

Each patient sample is a time-ordered sequence of EHR events containing a timestamp, event category, and value.
Raw events are transformed into sparse vectors covering observed event-value types, including multiple entries for measurements such as blood pressure.
Events within time intervals are aggregated using counts for binary outcomes and minimum, maximum, and mean values for numerical measurements.
A separate context vector encodes patient metadata and comorbidities independently of event-sequence ordering.

2.5. Data preprocessing and model design

The study compares clinically selected and data-driven baselines with a sequential CNN-LSTM model that preprocesses raw EHR events into regularized temporal inputs and incorporates patient context.

Model approaches: Three model approaches are evaluated: a clinically selected baseline, a non-sequential data-driven neural network, and a sequential neural network.
Baseline models: GB-Vital uses 30 features from six vital-sign event types in a gradient boosted decision-tree classifier.
Baseline models: The MLP aggregates full event vectors across retrospective time windows, largely ignoring event order, and combines approximately 30,000 event features with contextual features.
Data preprocessing: Preprocessing groups events into five-minute blocks, fills temporal gaps with empty vectors, and produces a regular sequence matrix.
Data preprocessing: The final sequence concatenates each aggregated event vector with the same context vector containing metadata such as age, gender, and comorbidities.
CNN-LSTM model: The CNN-LSTM projects sparse inputs into dense vectors, uses convolutional blocks for short-term temporal patterns, and feeds their outputs to LSTM cells for longer-term temporal development.
Implementation: The model was trained on an NVIDIA Tesla V100 GPU and reached convergence after approximately 90 minutes.

2.6. Model evaluation

The evaluation combines discrimination metrics with decision-analytic and intervention-aware assessments to better reflect clinical usefulness. It accounts for persistent prior predictions and examines whether interventions preceded prediction times.

Discrimination: The model outputs a risk from zero to one, and discrimination is evaluated across thresholds using ROC, PR, AUROC, or mAP.These measures support comparison with existing and future studies.
Decision curve analysis: Decision curve analysis evaluates net benefit across intervention thresholds or equivalent harm–benefit exchange ratios, including comparisons with never intervening or always intervening.Clinicians can select the model with the highest net benefit at their preferred harm–benefit ratio.
Limitations: The DCA formulation focuses only on patients for whom an intervention occurs, making it more conservative than formulations that also include noninterventions.Hospital contacts without intervention continue as usual and are not affected by the prediction model.
Clinical usefulness: Generic metrics such as ROC, PR, and DCA do not by themselves capture how a sepsis model should be used clinically.Clinical usefulness depends on the consequences of decisions and whether predictions can lead to action.
Sequence evaluation: Sequence predictions retain the maximum probability observed up to each prediction time, maintaining a positive classification after an intervention-triggering prediction.This reflects that an intervention cannot be withdrawn once initiated.
SERAIP: The intervention-aware evaluation retrospectively examines intravenous antibiotics and blood cultures recorded before prediction, including registrations up to 72 hours earlier.True-positive predictions are categorized by intervention status, including cases with no intervention to indicate potential for early intervention.

3. Results

Results show that vital-sign completeness declines substantially farther before sepsis onset, while model performance and clinical utility vary across models, prediction times, and thresholds. The CNN-LSTM achieved strong discrimination and net benefit across broad probability thresholds, and SERAIP quantified opportunities for interventions not yet initiated.

Vital-sign completeness: 100% of septic patients had at least two vital signs recorded from t−6 to t−0, declining to 32% from t−24 to t−30.Completeness was 65% from t−9 to t−3 and 43% from t−18 to t−12.
Gradient Boosting: The GB-Vital model achieved AUROC 0.786 and mAP 0.797 three hours before sepsis in the vital-sign test set.Its net benefit exceeded treating none or all patients above a 32% probability threshold.
Multilayer perceptron: The MLP model achieved AUROC 0.764 and mAP 0.689 three hours before sepsis in the vital-sign data set.Its net benefit exceeded both treating no patients and treating all patients above 45%.
Multilayer perceptron: On the full data set, MLP AUROC declined from 0.872 at t−15 min to 0.619 at t−24 hours, while mAP peaked at 0.578 at three hours.The MLP model's net benefit exceeded both reference strategies between 20% and 45% probability thresholds.
CNN-LSTM: The CNN-LSTM achieved AUROC 0.856 and mAP 0.79 three hours before sepsis in the vital-sign test data.On the full data set, its AUROC was 0.879 at t−15 min and 0.752 at t−24 hours, while net benefit exceeded both reference strategies from 5% to 60%.
SERAIP: SERAIP found that 27 of 39 true positives in an example row had no intervention initiated at prediction time.The table assesses interventions using intravenous antibiotics and blood cultures; the reported false-positive ratio was 9.28 false alarms per true positive.

4. Discussion

The study presents deep-learning approaches for early sepsis detection outside ICUs and evaluates their predictive performance, clinical utility, calibration, and practical limitations. Sequential modeling appeared more stable across prediction times and had higher decision-curve net benefit than the MLP, while broader deployment remains constrained by data availability, interpretability, generalizability, and validation scope.

Models and scope: The study compared GB-Vital, non-sequential MLP, and sequential CNN-LSTM models for early sepsis detection on heterogeneous data outside ICUs.The GB-Vital model used vital signs, whereas the MLP and CNN-LSTM used thousands of features.
Baseline performance: 0.786 AUROC was achieved by GB-Vital three hours before sepsis onset among patients with registered vital signs.This performance was comparable to a similar model evaluated with a similar missing-value rate, but lower than some ICU-based studies.
Limitations: Only 65% of sepsis patients had at least two vital signs measured three hours before onset, limiting the broader usefulness of vital-sign-only models.The authors note that vital parameters are not recorded as frequently across hospital departments as in ICUs.
Prediction over time: The CNN-LSTM and MLP had similar AUROC at three hours, but the MLP had better average precision while the CNN-LSTM appeared more stable across prediction times.The authors attribute the stability difference to sequential modeling.
Clinical utility and calibration: The CNN-LSTM had higher net-benefit values than the MLP across the full threshold range in decision-curve analysis.The MLP also showed poor calibration, especially at predicted-risk ranges of 0.05–0.2 and 0.6–0.8.
Clinical utility and calibration: Retrospective assessment incorporated intravenous antibiotics and blood-culture requisitions to estimate whether predictions could precede interventions.The evaluation was intended to approximate clinical use across departments rather than assess discrimination alone.
Limitations: Clinical acceptance would benefit from explanation methods, while generalizability concerns arise because high-dimensional models may learn organizational and process-related signals.The study also did not test its models on MIMIC-III and used simple rules to sample negative cases.

5. Conclusion

The study presents a deep-learning system for early sepsis detection using heterogeneous multicenter data from outside ICUs. It learns from raw event sequences and evaluates potential clinical utility by accounting for prior antibiotics and blood cultures.

Conclusion: The system detects sepsis early from heterogeneous, multicenter data outside ICUs using representations learned directly from raw event sequences.The approach avoids labor-intensive feature extraction and models key factors and interactions from the event data.
Conclusion: The retrospective evaluation accounted for intravenous antibiotics and blood-culture requisitions when assessing potential clinical utility.The evaluation showed that many sepsis patients had not initiated either intervention at early detection.
Conclusion: Future work should add explanation methods to improve clinical acceptance and test the models on the MIMIC-III database.These directions address interpretability and external evaluation.

6. Conflict of interest statement

The authors Simon Meyer Lauritsen, Mads Ellersgaard Kalør, Emil Lund Kongsgaard, and Bo Thiesson are employed at Enversion A/S.

Conflict of interest statement: The authors Simon Meyer Lauritsen, Mads Ellersgaard Kalør, Emil Lund Kongsgaard, and Bo Thiesson are employed at Enversion A/S.