Source-linked AI summary
Improving Palliative Care with Deep Learning
Anand Avati, Kenneth Jung, Stephanie Harman, Lance Downing, Andrew Ng, Nigam H. Shah
TL;DR
Physicians may miss patients who could benefit from palliative care, while manual chart review is costly and time-consuming. This paper uses deep learning on EHR data to predict 3–12-month mortality as a referral proxy, achieving strong discrimination and enabling proactive outreach with patient-specific explanations.
Problem
Physician non-referral and the expense of manually reviewing every admission can prevent timely identification of patients likely to benefit from palliative care.
Method
A Deep Neural Network analyzes patients’ EHR histories to predict 3–12-month all-cause mortality, using the prediction to recommend palliative care referrals and code-ablation analyses to explain them.
Results
AUROC was 0.93 overall and 0.87 for admitted patients; average precision was 0.69 overall and 0.65 for admitted patients.
Takeaways & Limitations
The model is being piloted for daily proactive outreach, with explanations intended to help practitioners act on its recommendations.
Abstract
from arXiv · showhide
Improving the quality of end-of-life care for hospitalized patients is a priority for healthcare organizations. Studies have shown that physicians tend to over-estimate prognoses, which in combination with treatment inertia results in a mismatch between patients wishes and actual care at the end of life. We describe a method to address this problem using Deep Learning and Electronic Health Record (EHR) data, which is currently being piloted, with Institutional Review Board approval, at an academic medical center. The EHR data of admitted patients are automatically evaluated by an algorithm, which brings patients who are likely to benefit from palliative care services to the attention of the Palliative Care team. The algorithm is a Deep Neural Network trained on the EHR data from previous years, to predict all-cause 3-12 month mortality of patients as a proxy for patients that could benefit from palliative care. Our predictions enable the Palliative Care team to take a proactive approach in reaching out to such patients, rather than relying on referrals from treating physicians, or conduct time consuming chart reviews of all patients. We also present a novel interpretation technique which we use to provide explanations of the model's predictions.
I. INTRODUCTION
Palliative care access is increasing, but many patients still receive aggressive hospital-based end-of-life care. The paper motivates automated, objective identification of patients who may benefit from palliative care.
- Approximately 80% of Americans prefer to spend their final days at home, but only 20% do.
- Although palliative care access has increased, less than half of the relevant hospital population is covered by available teams.
- A shortage of palliative care professionals makes manual chart review for candidate identification expensive and time-consuming.
- Physician overoptimism, time pressures, and treatment inertia can prevent referrals and contribute to patients’ wishes not being carried out.
- Existing palliative prognostic scores require clinical assessment or clinician survival prediction and are difficult to implement at scale.
Prognostic tools in the Intensive Care Unit
Prior prognostic tools target settings such as the ICU, terminal illness, or selected diseases, with varying inputs and prediction horizons. Their scope limits their use for earlier, broad identification of patients who may benefit from palliative care.
- Prognostic tools in the Intensive Care Unit: ICU scores such as APACHE-II, APACHE-III, and SAPS II support treatment decisions for patients already in intensive care.
- Prognostic tools in the Intensive Care Unit: ICU scoring systems have limited use for identifying longer-term mortality risk while patients can still discuss goals and values.
- Prognostic tools for Early Identification: CriSTAL uses eighteen predictors to identify elderly patients nearing the end of life and estimate risk of in-hospital or shortly post-discharge death.
- Prognostic tools for Early Identification: CARING uses six criteria to identify patients at risk of death within one year, while PREDICT was refined from CARING using six prognostic indicators.
- Prognostic tools for Early Identification: Other models include six-month mortality prediction from Medicare claims, laboratory-based 30-day-to-five-year risk scores, and admission-based logistic regression tools.
Prognosis in the age of Big-Data
The paper uses high-dimensional EHR data and deep learning to address limitations of earlier prognostic tools. It predicts 12-month mortality as a proxy for palliative care need, including among admitted patients.
- Prognosis in the age of Big-Data: Earlier tools were limited by small datasets, few variables, simple models, or restriction to particular subpopulations.
- Prognosis in the age of Big-Data: The approach is largely agnostic to disease type, disease stage, admission severity, and age.
- Prognosis in the age of Big-Data: Given a patient and date, the model predicts mortality within 12 months using that patient’s preceding year of EHR data.
- Prognosis in the age of Big-Data: The task is formulated as supervised binary classification, with separate interest in mortality prediction for currently admitted patients.
Data Source
The study uses STRIDE EHR data and constructs prediction examples around encounter dates that provide sufficient history and a 3–12-month lead time before death. Inpatient dates are prioritized when eligible.
- Data Source: STRIDE contains EHR data for approximately 2 million patients treated at Stanford Hospital or Lucile Packard Children’s Hospital between 1995 and 2014.
- Constructing a Dataset for Supervised Learning: Patients with a recorded death date are labeled positive, while other patients are labeled negative.
- Constructing a Dataset for Supervised Learning: The prediction date divides each record into past and future events, using virtual-past data to predict survival 3–12 months ahead.
- Constructing a Dataset for Supervised Learning: Eligible prediction dates are encounters at least 3 months and at most 12 months before death.
- Constructing a Dataset for Supervised Learning: Prediction dates must follow the first encounter by at least 12 months to provide sufficient history.
- Constructing a Dataset for Supervised Learning: Eligible inpatient admissions are preferred, and the earliest qualifying candidate date is selected.
Negative Cases:
Negative cases are patients without a recorded death date who were alive at least 12 months after prediction; prediction dates satisfy multiple EHR-history constraints.
- Negative Cases:: Negative cases require patients without a death date to be alive for at least 12 months from the prediction date.
- Negative Cases:: Prediction dates must be recorded encounter dates at least 12 months after first encounter and before the final 12-month EHR period.
- Negative Cases:: Inpatient encounters are preferred when constraints are met, and the latest eligible candidate date is selected.
- Negative Cases:: Patients with inpatient prediction dates are classified as admitted, and their dates are shifted to the second admission day.
Admitted patients:
The dataset contains 221,284 included patients, with approximately 5% admitted and higher mortality prevalence among admitted patients; deceased patients are skewed toward older ages.
- Admitted patients:: 221,284 patients met the inclusion criteria.
- Admitted patients:: Patients in the deceased positive class were skewed toward older age at prediction time.
- Admitted patients:: The included patients were randomly split into training, validation, and test sets in an approximate 8:1:1 ratio.
- Admitted patients:: Approximately 5% of patients were admitted, defined by a prediction date on the second day of an admission.
- Admitted patients:: 11% of admitted patients died, compared with approximately 7% mortality prevalence among all included patients.
Feature Extraction
Features summarize one year of prior EHR activity using temporally weighted slices, code-category statistics, demographics, and a sparse pruned feature matrix.
- Feature Extraction: The 12-month observation window includes ICD9, CPT, RxNorm, and encounter data to create patient features.
- Feature Extraction: The observation window is divided into four slices, with the most recent slice first and intentionally greater emphasis on recent data.
- Feature Extraction: Each slice records per-patient counts of individual codes within prescription, billing, and other code categories.
- Feature Extraction: Demographic features include age, gender, race, and ethnicity, alongside code-category summary statistics such as unique, total, maximum, minimum, and range counts.
- Feature Extraction: 13,654 features remained after pruning features occurring in 100 or fewer patients; the matrix was approximately 99.5% sparse.
Algorithm and Training
The model is an extensively tuned deep neural network with 13,654 input dimensions, 18 hidden layers, and a scalar output, trained with logistic loss and Adam.
- Algorithm and Training: The DNN contains a 13,654-dimensional input layer, 18 hidden layers of 512 dimensions each, and a scalar output layer.
- Algorithm and Training: The network uses SeLU activations, logistic loss, Adam optimization, and mini-batches of 128 examples.
- Algorithm and Training: The final model is the snapshot with the best validation performance, selected after searching network depths from 2 to 32 and multiple activation functions.
- Algorithm and Training: Implementation used Python 2.7, PyTorch, scikit-learn 0.17.1, and an NVIDIA TitanX GPU with CUDA 8.0.
Evaluation
The model was evaluated on imbalanced test data using calibration, precision-recall, and ROC analyses. It showed reasonable calibration and strong early-recall behavior, with reported performance across the full and admitted-patient cohorts.
- Evaluation metrics: Average Precision (AP) was selected for model evaluation because accuracy and ROC analysis can be misleading with 7% mortality prevalence.The selected model was based on the best AP score on the validation set and evaluated on the test set.
- Calibration: Brier score was 0.042, indicating reasonably calibrated predictions.At high thresholds, the model was slightly conservative or under-confident in its probability estimates.
- Precision-recall performance: AP was 0.69 overall and 0.65 among admitted patients.These values summarize the interpolated precision-recall performance.
- Precision-recall performance: Recall at precision 0.9 was 0.34 overall and 0.32 among admitted patients.The figure marks precision 0.9 and the recall achieved at that precision.
- ROC performance: AUROC was 0.93 overall and 0.87 among admitted patients.The ROC curve reports model performance on the test set.
Qualitative Analysis
The qualitative analysis examined whether high-probability predictions identified patients appropriate for palliative-care referral, including false positives. The paper also introduces patient-specific explanations based on ablating EHR codes and measuring probability drops.
- Qualitative analysis: High-probability false positives often had terminal illness or high healthcare utilization despite surviving beyond 12 months.False positives were inspected to assess the mortality proxy against the underlying palliative-care identification problem.
- Qualitative analysis: All 50 randomly reviewed patients in the test set’s top 0.9-precision bracket were judged appropriate for referral on their prediction date.This included patients who survived more than one year.
- Motivation: Explanations are intended to help practitioners trust and act on complex model decisions.The paper distinguishes interpreting the model from interpreting an individual decision.
- Prediction explanations: Direct feature perturbation is unsuitable here because changing an absent diagnosis can reveal general importance without explaining an individual patient.The example concerns changing a zero brain-cancer count to a nonzero value, which may sharply increase predicted mortality.
- Prediction explanations: The explanation method removes each patient’s ICD-9, CPT, RXNORM, or Encounter code and measures the resulting drop in predicted probability.The largest influences are reported by ranking codes within each category, with the top five selected per category.
- Prediction explanations: Demographic influence is estimated by zeroing age and swapping gender, then measuring the corresponding probability drops.The same patient-specific influence framework is applied to these demographic features.
VI. CONCLUSION
The study demonstrates that routinely collected EHR data can prioritize hospitalized patients for palliative-care follow-up, while deep-learning explanations support practitioner confidence in acting on recommendations.
- Routinely collected EHR data can create a system that prioritizes patients for palliative-care follow-up.
- All-cause mortality prediction can serve as a proxy for identifying patients who may need palliative-care consultation.
- The model is being piloted for daily proactive outreach, with objective outcomes such as consultation and goals-of-care documentation to be collected.