Source-linked AI summary
DeepSOFA: A Continuous Acuity Score for Critically Ill Patients using Clinically Interpretable Deep Learning
Benjamin Shickel, Tyler J. Loftus, Lasith Adhikari, Tezcan Ozrazgat-Baslanti, Azra Bihorac, Parisa Rashidi
TL;DR
Critically ill patients need timely acuity assessment, but traditional SOFA calculations use static thresholds and do not fully exploit streaming temporal data. DeepSOFA applies interpretable deep learning to hourly ICU measurements, achieving higher mortality-prediction accuracy than SOFA baselines and supporting real-time clinical interpretation.
Problem
Traditional ICU severity methods use time-consuming static thresholds and do not capture time-sensitive physiological patterns from streaming EHR data.
Method
DeepSOFA uses recurrent deep learning with gated units and real-time self-attention to model temporal ICU measurements and identify influential time steps.
Results
0.90 mean AUC for the entire ICU stay exceeded Bedside SOFA at 0.79 and Traditional SOFA at 0.85 in external validation.
Takeaways & Limitations
DeepSOFA may provide automated early warning and support therapeutic, goals-of-care, and resource-utilization decisions through real-time mortality assessment.
Abstract
from arXiv · showhide
Traditional methods for assessing illness severity and predicting in-hospital mortality among critically ill patients require time-consuming, error-prone calculations using static variable thresholds. These methods do not capitalize on the emerging availability of streaming electronic health record data or capture time-sensitive individual physiological patterns, a critical task in the intensive care unit. We propose a novel acuity score framework (DeepSOFA) that leverages temporal measurements and interpretable deep learning models to assess illness severity at any point during an ICU stay. We compare DeepSOFA with SOFA (Sequential Organ Failure Assessment) baseline models using the same model inputs and find that at any point during an ICU admission, DeepSOFA yields significantly more accurate predictions of in-hospital mortality. A DeepSOFA model developed in a public database and validated in a single institutional cohort had a mean AUC for the entire ICU stay of 0.90 (95% CI 0.90-0.91) compared with baseline SOFA models with mean AUC 0.79 (95% CI 0.79-0.80) and 0.85 (95% CI 0.85-0.86). Deep models are well-suited to identify ICU patients in need of life-saving interventions prior to the occurrence of an unexpected adverse event and inform shared decision-making processes among patients, providers, and families regarding goals of care and optimal resource utilization.
Introduction
ICU acuity assessment must recognize evolving illness severity and support timely intervention and shared decisions. DeepSOFA addresses limitations of static-threshold SOFA by using temporal measurements and deep learning to generate dynamic mortality predictions.
- SOFA summarizes six organ systems using worst measurements over an interval and static thresholds to assess illness severity and predict mortality.
- Dynamic acuity assessment may identify patients needing life-saving interventions and inform goals-of-care and resource-utilization decisions.
- DeepSOFA uses temporal measurements and deep learning to capture increasingly complex representations of raw ICU data without manual feature extraction.
Results
DeepSOFA was evaluated across ICU cohorts and showed stronger mortality discrimination than SOFA baselines throughout ICU stays. Its real-time predictions and attention mechanism also support patient-level interpretation of changing risk.
- External validation: 0.94 AUC at ICU discharge exceeded Bedside SOFA at 0.81 and Traditional SOFA at 0.85 in the MIMIC cohort.
- External validation: 0.90 mean AUC for the entire ICU stay exceeded Bedside SOFA at 0.79 and Traditional SOFA at 0.85 in external validation.
- External validation: DeepSOFA significantly outperformed traditional SOFA models across external validation cohorts regardless of which cohort supplied model development.
- Usability: In a single patient encounter, DeepSOFA estimated 60-90% mortality probability among non-survivors versus 20-40% among survivors.
- Interpretability: Self-attention highlights influential time steps and updates their weights at each hour using currently available ICU information.
Discussion
DeepSOFA combines dynamic deep learning with an established acuity-score framework to improve mortality prediction and interpretability. The authors position it as an automated aid for streaming EHR-based early warning and shared decision-making, while identifying broader inputs and live-stream evaluation as future work.
- DeepSOFA was externally validated in heterogeneous ICU populations and predicted in-hospital mortality more accurately than traditional methods.
- Attention visualizations identify ICU time periods whose inputs significantly affected predictions, improving interpretability and clinical application.
- Automated DeepSOFA may use streaming EHR data to provide early warning and inform therapeutic, goals-of-care, and resource-utilization decisions.
- Future studies should evaluate live streaming EHR data and expand inputs beyond SOFA components to the full spectrum of EHR variables.
Methods
The study assembled longitudinal ICU data, transformed SOFA-related measurements into hourly time series, and modeled mortality with a recurrent neural network using gated recurrent units. External validation trained on one cohort and tested on the other, with baseline models applied to the same test cohorts.
- UFHealth and MIMIC supplied longitudinal ICU cohorts for model development and external cross-validation.
- The analysis extracted SOFA-related physiologic, laboratory, medication, and ventilation variables from electronic health records.
- Raw time series were resampled hourly, with prior values propagated for vital signs and laboratory tests and zeros substituted for vasopressors and mechanical ventilation.
- DeepSOFA used a recurrent neural network with gated recurrent units to incorporate current inputs and temporal patterns across ICU admissions.
- External validation trained DeepSOFA on one cohort and tested it on the other, while baseline models were applied to the same testing cohorts.
Additional information
The figures present DeepSOFA’s external validation, outcome-stratified predictions, patient-level attention, and model abstraction. Table 1 summarizes cohort variables and ICU utilization characteristics.
- Cohort information: Table 1 summarizes demographics and included variables for the UFHealth and MIMIC cohorts, with statistics calculated after FiO2 imputation and outlier removal.The cohort summary includes ICU admissions and ICU stays requiring vasopressors or mechanical ventilation.
- External validation: DeepSOFA performance is evaluated against Bedside SOFA and Traditional SOFA for mortality prediction across two external validation cohorts.Figure 1 covers the first 100 hours after ICU admission, while Figure 2 covers the 100 hours preceding death or discharge.
- Interpretability: Patient-level and model-level figures visualize self-attention over time, linking mortality predictions to clinical measurements, events, interventions, and salient sequence patterns.Figure 3 presents one UFHealth encounter, Figure 5 contrasts a survivor with a non-survivor, and Figure 6 summarizes model inputs, outputs, and abstraction levels.
- Outcome-stratified predictions: The visualizations examine predicted mortality probabilities stratified by outcome during both the first and final 100 hours of ICU care.Figure 4 also reports the number of ICU encounters at each time point and displays 95% confidence intervals.
Supplementary Material
DeepSOFA uses recurrent and gated recurrent networks with attention to process ICU time-series data and generate continuously updated mortality predictions. Its self-attention mechanism weights prior time steps while preserving only information available at each prediction hour.
- Recurrent neural networks: An RNN updates its hidden state by combining each time step with a representation of previously observed data, enabling sequential modeling.The current hidden state represents the sequence through the current time and can be passed to later classification layers.
- Real-time predictions: DeepSOFA’s recurrent-network outputs differ from conventional sequence classification by producing predictions at every ICU hour rather than only a final sequence prediction.Supplementary Figure S1 contrasts compact and expanded RNN views and identifies the all-time-step predictions used for real-time acuity assessment.
- Gated recurrent units (GRU): GRUs add reset and update gates that expand recurrent memory for distant information while requiring fewer parameters than LSTMs.The gating mechanisms make the network more selective about information it learns and retains.
- Self-attention: Self-attention recalculates weights at each hour, emphasizing prior time steps that most influence the current mortality prediction.DeepSOFA uses learned projections of hidden states and updates attention distributions using only currently available EHR information.
- DeepSOFA architecture: DeepSOFA combines a GRU, self-attention, dropout, and a final classification layer for hourly ICU mortality prediction.The model replicates the mortality target across input time steps and trains with dynamically masked sequences to prevent future information from influencing current predictions.
- Supplementary evaluations: Supplementary evaluations compare DeepSOFA, Bedside SOFA, and Traditional SOFA using AUC over the first 100 hours and before death or discharge.Additional panels evaluate DeepSOFA models based on variable sets corresponding to SOFA organ-system classifications.