Source-linked AI summary
Learning a Continuous Sepsis Severity Score Without Hour-by-Hour Supervision: A Two-Site Retrospective Study
Kevin Zhu, Ryan Zhang, Baraa Abed, Tilendra Choudhary, Malvern Madondo, Mehak Arora, Yixuan Yang, Alasdair Gent, Aditya Nagori, Omer T. Inan, Krista L. Haines, Patrick Georgoff, Suresh M. Agarwal, Vijay Krishnamoorthy, Tetsu Ohnuma, Mihai V. Podgoreanu, Michael R. Pinsky, Gilles Clermont, Craig M. Coopersmith, Craig S. Jabaley, Rishikesan Kamaleswaran
TL;DR
Existing sepsis scores use fixed, single-timepoint designs that may not represent patient trajectories, while prior learned models supervise individual timepoints. This study learned a continuous hourly index from outcome-ranked treatment windows using T-REX and 43 routinely charted variables across two ICU cohorts. The index separated survivors from non-survivors within presenting-severity strata, tracked physiologic changes, and showed at least comparable mortality discrimination to established indices and direct outcome supervision, although cross-institution agreement fell below the within-institution ceiling.
Problem
Existing severity indices retain single-timepoint, association-based scoring designs, while prior learned models attach outcome supervision independently to each hour.
Method
T-REX learned a continuous severity score from outcome-ranked 72-hour treatment windows using 43 variables across MIMIC-IV and Emory Healthcare cohorts.
Results
The index separated survivors from non-survivors within every presenting-severity stratum and discriminated in-hospital mortality at least as well as established indices and direct outcome supervision.
Takeaways & Limitations
Outcome-ranked trajectory learning may complement expert-derived sepsis scoring, provided local validation addresses institutional differences.
Takeaways & Limitations
Because the score learns from care as delivered, its associations can reflect treatment patterns alongside illness, and cross-institution agreement fell short of the within-institution ceiling.
Abstract
from arXiv · showhide
Currently used sepsis severity indices rely on fixed variables and weights established decades ago, which are coarsely discretized and calibrated to a cohort that no longer reflects contemporary critical care. No alternative learned directly from patient trajectories is in routine use. We conducted a retrospective two-cohort study on a total of 29,116 and 7,691 adult patients meeting Sepsis-3 criteria from two hospital systems in Massachusetts and Georgie, respectively. We developed a sepsis index using 43 routinely charted variables over a 72-hour treatment window. Unlike previous studies, we use mortality as a treatment-level ranking signal rather than a per-state target, allowing credit to be redistributed non-uniformly across timesteps. Evaluation was done on a permanent 20% test holdout, using clinical vignettes and Spearman correlation. Uncertainty intervals were obtained by bootstrap resampling of whole patients. Under this ranking scheme, non-survivors scored 1.19-1.64 points higher than survivors on a 0-10 scale within all strata of baseline SOFA-2, with similar results stratifying within lactate, mean arterial pressure (MAP), and creatinine. Within-patient change in the index correlated with change in lactate (Spearman rho = 0.39; n = 1,854). Similar, weaker correlations were found for MAP and creatinine. On a cohort level, cross-institutional agreement measured by Spearman correlation between models trained on different sites, were 70-77% of same-site correlation. External within-patient correlations were 0.54 and 0.59 against ceilings of 0.92 and 0.90. Our index also correlated with established indices, while null controls stayed near zero. Our index demonstrated hourly prognostic information that meaningfully separates patient outcomes and is consistent with clinical expectation, indicating potential as a decision support tool complementing clinical judgement.
Site Retrospective Study
The listed authors include contributors affiliated with Duke University and the Georgia Institute of Technology. The affiliations span biomedical engineering, electrical and computer engineering, surgery, anesthesiology, population health sciences, and critical care medicine.
- The author list includes Kevin Zhu, Ryan Zhang, and Baraa Abed among the contributors.
- The affiliations identify Duke University departments in biomedical engineering, electrical and computer engineering, and surgery.
- The affiliations also include Georgia Institute of Technology programs in electrical and computer engineering and biomedical engineering.
Key Points
Sepsis severity scoring remains important, but established indices use single-timepoint associations and pointwise learned models can label every hour of a non-survivor’s stay as high risk. This study instead learns from ranked treatment trajectories using IRL and reports associations with outcomes, established scores, and physiologic change.
- Motivation: 166 million incident cases and 21.4 million sepsis-related deaths were estimated worldwide in 2021.
- Problem: Established indices preserve single-timepoint scoring despite recalibration efforts aimed at contemporary critical-care cohorts.
- Related Work: Hourly learned models can fit every hour of an eventual non-survivor’s stay toward high risk, including hours before deterioration.
- Novelty: Trajectory-ranked IRL constrains accumulated scores across treatment windows rather than attaching the endpoint label independently to each timepoint.
- Contribution: T-REX-trained indices correlated with established scores, discriminated mortality as well as direct outcome supervision, and separated outcomes within baseline-severity strata.
Methods
This retrospective pilot developed and evaluated a trajectory-based sepsis severity score using two independent adult ICU cohorts. The study used a 72-hour window, 43 routinely collected variables, mortality-related outcomes, and permanent site-specific test holdouts.
- Outcome: The outcome was discharge disposition, with hospice discharge counted as death; mortality occurred in 23.7% of Emory patients and 15.1% of MIMIC-IV patients.
- Study Design: The treatment window covered up to 72 hours, from 24 hours before through 48 hours after sepsis onset.
- Study Design: Forty-three physiologic, laboratory, and treatment variables were collected for model development.
- Evaluation: Within each site, 20% of patients were reserved as a permanent mortality-stratified test holdout and excluded from training and model selection.
Data Processing
The analysis aggregated and imputed hourly patient observations, trained T-REX models under three trajectory-ranking schemes, and used label-randomized controls. Evaluation assessed prognostic information, physiologic concordance, cross-institution interpretability, and bootstrap uncertainty.
- Data Processing: Hourly observations were aggregated, truncated to physiologically plausible ranges, forward-filled within admissions, and then median-imputed using the training partition.
- Score Development: T-REX learned the severity score by comparing patient-stay pairs under mortality, mortality-plus-treatment, and severity-matched rankings.
- Score Development: The mortality-plus-treatment scheme used vasopressor dose and then cumulative fluids as tie-breakers conditional on survival.
- Controls: Shuffled and random preference-label ablations tested how much reported effects depended on a learnable preference signal.
- Evaluation: Evaluation measured baseline-stratified outcome separation and within-patient score changes against concurrent physiologic changes.
- Uncertainty: Spearman correlations were averaged in Fisher z space, with 95% percentile intervals from whole-patient bootstrap resampling.
Results
The mortality-ranked learned score separated non-survivors from survivors across every baseline SOFA-2 severity stratum. The mortality scheme was selected because it was simplest and had the highest agreement across training replicates.
- Model Selection: The mortality scheme was selected because it used the simplest ranking and achieved the highest agreement between replicates across seeds and data folds.All evaluated schemes correlated similarly with established severity indices.
- Patient Stratification: 1.19–1.64 points separated non-survivors from survivors across all four baseline SOFA-2 strata on the frozen 0–10 scale.The gaps were 1.19, 1.42, 1.36, and 1.64 points from lowest to highest severity strata.
- Patient Stratification: The score’s positive survivor–non-survivor gap also appeared across all eight Emory strata and under lactate, MAP, and creatinine stratification.Lactate-stratified gaps were 1.48, 1.73, 1.66, and 1.33 points across the reported bins.
Comparison Against Physiology
Within-patient score changes tracked physiologic changes, most clearly for lactate, while cross-institutional scores showed moderate agreement against higher within-site ceilings.
- Within-Patient Physiology: Change in score tracked change in lactate with Spearman ρ = 0.39 across 1,854 patients.The association rose from 0.25 to 0.55 among patients with increasingly high baseline lactate.
- Within-Patient Physiology: Increases in MAP decreased the score, most strongly among patients presenting below 65 mmHg and weakest among those above 85 mmHg.Associations for MAP and creatinine were much weaker than for lactate.
- Cross-Institutional Transfer: Cross-institutional within-patient correlations were 0.54 for MIMIC-IV patients and 0.59 for Emory patients, versus ceilings of 0.92 and 0.90.Negative correlations occurred in 14.2% and 12.7% of patients, respectively.
Comparison of Ranking Schemes
Agreement was strongest for the mortality ranking scheme and declined as training conditions changed from seed to fold to institution. Learned scores correlated with established severity indices, while shuffled and randomized controls did not.
- Established Severity Indices: Mean Spearman correlations with established indices ranged from 0.25 to 0.46 in MIMIC-IV and 0.18 to 0.59 at Emory.The ordering was highest for APACHE II with age and lowest for SIRS across all six scheme-by-site combinations.
- Established Severity Indices: Shuffled-preference and randomized-label controls stayed near zero, with no mean correlation exceeding 0.08 in absolute value.This covered 24 control conditions across both sites.
- Ranking-Scheme Stability: 0.78–0.79 mean Spearman correlation marked same-site mortality-scheme agreement across random seeds, falling to 0.56–0.63 across institutions.Cross-institutional agreement retained 70%–77% of corresponding within-institution agreement.
- Ranking-Scheme Stability: Treatment-intensity ranking reduced agreement to 0.51–0.67 across seeds and 0.38–0.44 across institutions.Severity matching also degraded performance, but less than adding treatment intensity.
Mortality Discrimination Against Controls
The mortality-ranked score discriminated in-hospital mortality substantially better than null ablations and matched pointwise mortality-trained neural-net ensembles at both sites.
- Mortality Discrimination: 0.764 in MIMIC-IV and 0.742 at Emory were the mortality scheme’s median per-model AUROCs over the first 24 hours.The assessment covered admission-time and first-24-hour discrimination against null ablations and a methodological baseline.
- Mortality Discrimination: 0.486–0.533 was the median per-model AUROC range for shuffled and randomized controls across schemes and sites.Both ablations lost essentially all mortality signal.
- Baseline Comparison: Consensus mortality scores matched consensus pointwise mortality-trained neural nets: 0.791 versus 0.789 in MIMIC-IV and 0.765 versus 0.768 at Emory.The corresponding Spearman correlations were 0.74 and 0.43.
Discussion
The pilot supports learning a continuous hourly sepsis severity index from ranked patient trajectories and routinely charted physiology, while identifying important limits in treatment confounding and cross-institution transferability.
- Learning approach: Preference-based reward learning produced a continuous, per-hour sepsis index from outcome-ranked trajectories rather than cross-sectional mortality associations.The approach allows credit to be redistributed across hours within a treatment window.
- Clinical behavior: Survivors and non-survivors separated within every presenting-severity stratum at essentially every hour from onset onward.This suggests the hourly estimate contains information not fully captured by severity at presentation.
- Clinical behavior: The score’s physiologic changes matched clinical expectations, rising with lactate, falling as MAP returned to nominal ranges, and increasing in rank with creatinine.These directions were assessed by comparing the beginning and end of each patient’s treatment window.
- Limitations: The score reflects treatment patterns in its training cohorts because the state representation excludes vasopressors, fluids, renal replacement, and mechanical ventilation.Future iterations should add organ-support variables and test whether responses recover in the sickest strata.
- Limitations: Cross-institutional generalization remained below within-institution agreement, and the discrepancy depended on how trajectory preferences were ranked.Treatment intensity reduced reproducibility, while practice-dependent treatment exposure may encode local responses rather than patient state.
Conclusions
This pilot applies preference-based inverse reinforcement learning to learn a continuous hourly sepsis index from routinely charted data. The index separated outcomes across presenting-severity strata and tracked patients’ physiology, while cross-institution agreement remained below within-institution ceilings, supporting local validation before bedside use.
- Conclusions: Preference-based IRL yielded a continuous hourly sepsis index learned from routinely charted data.
- Conclusions: The index separated survivors from non-survivors within every stratum of presenting severity and moved with patients’ own physiology over their stays.
- Conclusions: Cross-institution agreement fell short of within-institution ceilings, so local validation should precede bedside use.
Tables and Figures
The figures and table present the study population, the learned 0–10 severity trajectory, patient-level physiologic associations, cross-institution agreement, and comparisons with established indices.
- Table 1 summarizes the demographic characteristics of the study population.
- The pipeline uses hourly EHR physiologic and laboratory data with T-REX to learn severity from outcome-ranked treatment periods rather than hourly clinician labels.
- The score is plotted over 48 hours after sepsis onset within four baseline SOFA-2 strata, separating survivors and non-survivors on a frozen 0–10 scale.
- Figure 3 compares within-patient score changes with changes in physiologic variables across the treatment window, using one point per held-out patient.
- Figure 4 compares hourly scores from MIMIC-IV-trained and Emory-trained models, showing cross-institution series alongside within-institution ceilings and shuffled controls.
- Figure 5 displays Spearman agreement with four established severity indices using box plots, model points, Fisher-z means, and negative controls.