Source-linked AI summary
Clinical Intervention Prediction and Understanding using Deep Networks
Harini Suresh, Nathan Hunt, Alistair Johnson, Leo Anthony Celi, Peter Szolovits, Marzyeh Ghassemi
TL;DR
The paper addresses real-time ICU intervention prediction from noisy, heterogeneous, imbalanced data. It integrates multimodal ICU records with LSTM and CNN models for forward-facing prediction and interprets their learned signals. Deep models outperform baselines across intervention tasks while yielding clinically relevant interpretability insights.
Problem
Real-time prediction of multiple ICU interventions is difficult because clinical data are noisy, sparse, heterogeneous, and outcomes are imbalanced.
Method
The study integrates vitals, labs, notes, and demographics in LSTM and CNN models to predict intervention onset and weaning with a 6-hour gap.
Results
Deep learning methods beat state-of-the-art prior AUCs for multiple forward-facing ICU intervention prediction tasks.
Takeaways & Limitations
Feature occlusion and CNN trajectory analyses identify clinically relevant modality and trajectory patterns that aid model interpretation.
Takeaways & Limitations
The interpretability results are an initial step, with further work needed before these models can be adopted in real clinical settings.
Abstract
from arXiv · showhide
Real-time prediction of clinical interventions remains a challenge within intensive care units (ICUs). This task is complicated by data sources that are noisy, sparse, heterogeneous and outcomes that are imbalanced. In this paper, we integrate data from all available ICU sources (vitals, labs, notes, demographics) and focus on learning rich representations of this data to predict onset and weaning of multiple invasive interventions. In particular, we compare both long short-term memory networks (LSTM) and convolutional neural networks (CNN) for prediction of five intervention tasks: invasive ventilation, non-invasive ventilation, vasopressors, colloid boluses, and crystalloid boluses. Our predictions are done in a forward-facing manner to enable "real-time" performance, and predictions are made with a six hour gap time to support clinically actionable planning. We achieve state-of-the-art results on our predictive tasks using deep architectures. We explore the use of feature occlusion to interpret LSTM models, and compare this to the interpretability gained from examining inputs that maximally activate CNN outputs. We show that our models are able to significantly outperform baselines in intervention prediction, and provide insight into model learning, which is crucial for the adoption of such models in practice.
1. Introduction
The paper targets forward-facing ICU intervention prediction across heterogeneous clinical data, using deep networks to support timely and interpretable predictions.
- The study predicts onset, weaning, and continuity for five ICU interventions spanning ventilation, vasopressors, and fluid boluses.
- Deep learning is used to capture complex relationships across heterogeneous clinical data with limited hand-engineering.
- The models make forward-facing hourly predictions intended for use at the time of care.
- Feature occlusion and maximally activating CNN inputs provide complementary approaches for interpreting model predictions.
2. Background and Related Work
Prior work motivates broader ICU intervention prediction by highlighting uncertainty, specialized cohorts, and limited coverage of multiple outcomes.
- Clinical decision-making often occurs under limited knowledge and high uncertainty, motivating analysis of routinely collected healthcare data.
- Related studies applied recurrent networks to sequential EHR data and compared CNNs with LSTMs for longitudinal outcome prediction.
- Earlier ICU intervention models commonly focused on single outcomes or specialized cohorts rather than general ICU populations.
3. Data and Preprocessing
The study builds hourly ICU examples from multimodal MIMIC-III data, using sliding windows to predict intervention status after a clinically separated gap.
- The dataset contains 34,148 first ICU stays from MIMIC-III patients aged 15 or older with stays lasting 12–240 hours.
- Each patient record combines static demographics, 29 time-varying vitals and labs, clinical-note topic vectors, intervention state, and time of day.
- Physiological words discretize z-scored vital values into integer bins from -4 to 4 while explicitly representing missingness.
- A 6-hour sliding input window predicts a 4-hour intervention window after a 6-hour gap, yielding 1,154,101 examples.
- Ventilation and vasopressor tasks use onset, wean, stay-on, and stay-off classes, whereas bolus tasks use onset versus no onset.
4. Methods
The methods compare LSTM and temporal CNN models against logistic regression and use occlusion, trajectory inspection, and activation maximization for interpretation.
- LSTM: The LSTM maps an input sequence through hidden states to a softmax probability distribution over intervention outcomes.
- CNN: The CNN applies temporal convolutions at three granularities, max pooling, concatenation, fully connected layers, dropout, and softmax output.
- Evaluation: Performance is evaluated with per-class AUC and macro AUC, which weights classes equally despite class imbalance.
- Baselines: The models are compared with L2-regularized logistic regression using the same imputed 6-hour numerical inputs.
- Interpretability: LSTM feature occlusion measures prediction shifts after masking inputs, while CNN interpretation examines extreme real trajectories and activation-maximizing inputs.
5. Results
Deep architectures achieved state-of-the-art intervention prediction, while feature analyses showed that different data modalities and physiological patterns were informative across intervention types and tasks.
- Overall results: Deep architectures achieved state-of-the-art prediction results across five intervention types and four prediction tasks.The models used six-hour chunks of normalized, mean-imputed raw data or discretized physiological words.
- Overall results: Physiological words increased AUC for ventilation onset from 0.61 to 0.75 and colloid bolus onset from 0.52 to 0.72.These interventions had the lowest proportion of onset examples, and the authors suggest physiological words may provide smoothing.
- Feature-level interpretation: Feature occlusion showed that vitals, labs, topics, and static data were important for different intervention predictions.Mechanical ventilation relied on consistent physiological laboratory features, while ventilation onset additionally emphasized Glasgow Coma Score and a consciousness-related topic.
- Feature-level interpretation: Vasopressor predictions emphasized physiological variables, whereas non-invasive ventilation and fluid-bolus onset predictions emphasized clinical-note topics.Vasopressor onset AUC differences reached 0.16, while non-invasive ventilation onset and weaning AUC variation was 0.14 versus 0.01.
- CNN interpretation: Maximally activating CNN trajectories highlighted physiological patterns associated with future ventilation, non-invasive ventilation, and bolus interventions.The reported patterns included higher respiratory and heart rates with lower oxygen saturation for ventilation, decreased creatinine, phosphate, oxygen saturation, and blood urea nitrogen for non-invasive ventilation, and general physiological decline for boluses.
- CNN interpretation: Model-generated hallucinations identified blood-pressure drops for vasopressor onset and decreasing respiratory rate for ventilation onset, but these trajectories were not necessarily physiologically plausible.The authors use them to inspect network behavior rather than as clinical trajectories.
6. Conclusion
The study combines deep models with interpretability analyses for forward-facing ICU intervention prediction. It reports clinically relevant trajectory patterns and identifies representation and modeling considerations for future clinical adoption.
- Deep learning methods beat prior state-of-the-art AUCs for forward-facing prediction of ICU intervention onset and weaning across multiple physiological systems.
- Physiological words significantly improved performance on the two lowest-incidence intervention tasks, possibly by encoding normality and robustness to missingness.
- LSTM occlusion linked physiological data to more invasive tasks and clinical-note topics to less invasive interventions, while weaning tasks generally showed lower AUC variance.
- CNN filters learned clinically relevant patient-trajectory trends that were further mimicked by input hallucinations, although more complex models might capture additional dependencies.
- The interpretability analyses are presented as an initial step toward broader neural-network interpretation and eventual adoption in clinical settings.
A. Dataset Statistics
The dataset-statistics section is represented by tables describing the study dataset.
- Table 3 presents the variables used in the study dataset.
- Table 4 presents statistics describing the dataset.
- The section identifies dataset statistics as a tabular component of the paper.
B. Physiological Word Generation
The paper converts continuous physiological time series into discrete physiological words. This representation uses standardized, rounded categories and preserves missingness through zero-valued rows.
- Continuous physiological time series are converted into discrete physiological words.
- Numeric values are z-scored, rounded, and converted so that each z-score becomes its own category.
- A glucose value two standard deviations below the mean is represented as the category glucose -2.
- A row containing all zeros for a variable marks that variable as missing at the timestep.
C. LSTM Model Details
The LSTM processes each new input with update equations whose learned parameters and element-wise operations define its recurrent state. Training uses categorical cross-entropy comparing predicted class probabilities with true labels.
- C. LSTM Model Details: The LSTM update equations use learned weights and biases, with sigmoid and element-wise product operations applied to recurrent states and inputs.The formulation extends to multiple layers by feeding the previous layer's hidden state as the next layer's input.
- C. LSTM Model Details: Categorical cross-entropy measures prediction loss across N examples and M classes using predicted class probabilities and true labels.
D. Generated Topics
Table 5 presents the most probable words in topics identified as important for intervention predictions.
- D. Generated Topics: Table 5 lists the most probable words associated with topics considered important for intervention predictions.