Source-linked AI summary
Does the "Artificial Intelligence Clinician" learn optimal treatment strategies for sepsis in intensive care?
Russell Jeter, Christopher Josef, Supreeth Shashikumar, Shamim Nemati
TL;DR
Healthcare AI is expanding rapidly, but complex models and large datasets can receive limited oversight. This paper examines an AI Clinician trained with reinforcement learning on MIMIC III data to recommend vasopressors and IV fluids for sepsis. Although the learned policy receives higher estimated value, the critique finds that it can miss prolonged hypotension and that evaluation and representation choices may distort returns.
Problem
Healthcare machine-learning applications are attracting attention despite complexity and relatively little oversight, motivating scrutiny of an AI Clinician proposed for sepsis treatment.
Method
The AI Clinician uses reinforcement learning with discretized MIMIC III patient states, 25 vasopressor–fluid actions, and policy iteration.
Results
The best AI policy has much higher estimated expected value than clinician, zero-drug, and random policies, while zero-drug policy usually outperforms clinician policy.
Takeaways & Limitations
High expected returns do not ensure clinically responsive individual dosing, because the reproduced AI Clinician missed prolonged hypotensive episodes.
Takeaways & Limitations
The critique identifies long-term-only rewards, importance-sampling bias, coarse four-hour discretization, and questionable cohort definitions as limitations affecting interpretation.
Abstract
from arXiv · showhide
From 2017 to 2018 the number of scientific publications found via PubMed search using the keyword "Machine Learning" increased by 46% (4,317 to 6,307). The results of studies involving machine learning, artificial intelligence (AI), and big data have captured the attention of healthcare practitioners, healthcare managers, and the public at a time when Western medicine grapples with unmitigated cost increases and public demands for accountability. The complexity involved in healthcare applications of machine learning and the size of the associated data sets has afforded many researchers an uncontested opportunity to satisfy these demands with relatively little oversight. In a recent Nature Medicine article, "The Artificial Intelligence Clinician learns optimal treatment strategies for sepsis in intensive care," Komorowski and his coauthors propose methods to train an artificial intelligence clinician to treat sepsis patients with vasopressors and IV fluids. In this post, we will closely examine the claims laid out in this paper. In particular, we will study the individual treatment profiles suggested by their AI Clinician to gain insight into how their AI Clinician intends to treat patients on an individual level.
1 Introduction
Machine-learning research is expanding rapidly in healthcare, but its complexity and large datasets have enabled relatively little oversight. This critique examines Komorowski et al.’s AI Clinician for sepsis, focusing on the individual vasopressor and IV-fluid treatment profiles it recommends.
- PubMed publications using “Machine Learning” increased 46%, from 4,317 to 6,307, between 2017 and 2018.
- Healthcare AI studies have attracted broad attention amid rising costs and demands for accountability, despite complex applications and large datasets receiving relatively little oversight.
- Komorowski et al. propose training an artificial intelligence clinician to treat sepsis with vasopressors and IV fluids.
- The analysis examines the claims and individual treatment profiles produced by the proposed AI Clinician.
2 Terminology
The paper introduces reinforcement-learning terminology for sepsis treatment, representing patient measurements as states and vasopressor–fluid combinations as actions. Policy value concerns survival under a proposed policy relative to observed clinical treatment.
- A state summarizes a patient’s clinical measurements at one time, with each 4-hour period assigned to one of 750 unique states.
- An action combines IV-fluid and vasopressor treatment levels, producing 25 normalized intervention combinations.
- A policy is the recommended vasopressor or IV-fluid action for a given patient state.
- Policy value is defined by survival likelihood under the proposed policy, requiring retrospective counterfactual reasoning against the policy that generated observed data.
- Septic shock involves persistent fluid-unresponsive hypotension requiring vasopressors and suspected elevated lactate, while some hypotensive septic patients with normal lactate are excluded from that label.
3 Summary
The authors frame sepsis treatment as reinforcement learning: patient data define a discrete environment, and policy iteration seeks treatment actions that maximize expected survival. The reported evaluation favors the learned AI policy, but also shows the zero-drug policy outperforming clinician policy in most realizations.
- The framework uses reinforcement learning and policy iteration to search a large state space for a policy maximizing expected value, corresponding here to minimizing mortality.
- MIMIC III patient data are clustered into 750 states and 25 actions, then used to construct a 750 x 750 x 25 transition matrix for the learned environment.
- The transition matrix is used with policy iteration to construct an AI Clinician that doses patients with vasopressors and IV fluids.
- The best AI policy has much higher estimated expected value than clinician, zero-drug, and random policies, although zero-drug policy usually outperforms clinician policy.Figure 2 compares estimated policy values across 500 environment realizations.
- The zero-drug result raises a counterintuitive evaluation question because not intervening appears more valuable than treating septic patients under clinician policy.
- The critique reports that requested code and data were not made available for months, prompting an attempted reproduction of the study.
4 Results from Implementing Their Methods
A reproduction using 5,366 MIMIC III patients found that the AI Clinician could achieve high expected returns while failing to respond to prolonged hypotension. The critique attributes this mismatch to the state/action representation and counterfactual policy evaluation.
- Reproducibility is treated as essential for verifying controversial computational biomedical findings, with code sharing identified as an important enabling step.
- The reproduction used 5,366 MIMIC III septic patients who received both vasopressors and fluids, with one-hour rather than four-hour bins.
- The AI Clinician’s 95th-percentile policy had much higher expected value than actual clinicians, while individual dosing suggestions were plotted against patient MAP.
- Despite high expected returns, the AI Clinician failed to learn the relationship between MAP and treatment, leaving a patient untreated during hours of MAP below 65 mm Hg.
- The critique concludes that both the patient-state/action representation and counterfactual evaluation method skewed the AI Clinician’s expected returns.
5 Policy Evaluation
The evaluation uses importance sampling and an in silico ICU, but these choices can bias estimated AI policy value toward cases resembling clinician actions and away from difficult patient trajectories.
- 5.1 Importance Sampling Introduces Bias in Evaluating the AI policy in Patients with Poor Outcomes: The best AI policy reaches an expected policy value of 40, roughly corresponding to 30% mortality, whereas the cohort’s 40% mortality corresponds to approximately 15–20.The figure compares clinician and AI policy values across 500 environment realizations using rewards of +100 for survival and -100 for death.
- 5.1 Importance Sampling Introduces Bias in Evaluating the AI policy in Patients with Poor Outcomes: The analysis grades the AI Clinician in an oversimplified simulated ICU rather than through direct interaction with real patients.Retrospective data provide the environment, because prospective trial-and-error learning on patients would be unethical.
- 5.1 Importance Sampling Introduces Bias in Evaluating the AI policy in Patients with Poor Outcomes: Importance sampling heavily weights AI decisions that resemble clinician actions, while assigning little value to substantially different actions regardless of whether outcomes improve or worsen.This can let the AI avoid dosing very sick patients whose deaths are attributed to clinician-managed trajectories.
- 5.1 Importance Sampling Introduces Bias in Evaluating the AI policy in Patients with Poor Outcomes: The AI Clinician’s learned associations between interventions and poor outcomes can lead it to avoid clinically appropriate actions in difficult cases, biasing expected policy value upward.The evaluation does not grade the AI on these tough cases.
- 5.2 Limitations of Long-Term Rewards: The approach evaluates only long-term survival outcomes and omits intermediate rewards such as maintaining physiologic blood pressure, limiting what dosing relationships the AI can learn.The omission may permit non-standard behaviors, including vasopressor use in normotensive patients.
6 System Representation
The system compresses patient trajectories into four-hour summaries and 750 states, limiting sensitivity to rapid deterioration and potentially obscuring the timing between hypotension, treatment, and recovery.
- 6.1 Trajectory Discretization: Four-hour bins can miss rapid changes in heart rate, blood pressure, and urine output, limiting the AI Clinician’s ability to detect and respond to deterioration.The patients deteriorating fastest are also those most in need of treatment.
- 6.1 Trajectory Discretization: Because vasopressors act nearly instantaneously, four-hour sampling fails to represent the timing between a blood-pressure change and the intervention responding to it.This weakens the learned relationship between states and actions.
- 6.1 Trajectory Discretization: A single four-hour summary may show normal MAP even when MAP fell, vasopressors were administered, and MAP recovered within that interval.The resulting state-action pair can make vasopressors for normal-MAP patients appear optimal.
- 6.2 States, Actions, and Transitions: The 750-state representation captures only current and next states, so low-frequency events such as rapid decompensation are effectively ignored.Each four-hour combination of measurements and pre-existing conditions is reduced to one predefined state.
- 6.2 States, Actions, and Transitions: The in silico model is biased toward healthier patients, who comprise nearly two thirds of both cohorts, helping explain why the zero-drug policy appears successful.This population composition interacts with importance-sampling-based evaluation metrics.
7 Interpretability
The critique argues that the paper’s interpretability analysis explains policy discrepancies globally, not the AI Clinician’s individual dosing decisions.
- 7 Interpretability: The analysis trains random forests to predict medication decisions and estimates the relative importance of input variables.The paper presents this as evidence that the reinforcement-learning decisions rely on clinically sensible parameters.
- 7 Interpretability: The critique identifies three problems with treating these random-forest feature-importance plots as evidence of individual-decision interpretability.The separate clinician and AI plots should be essentially identical for the binary classification task, while observed differences can arise from random-forest artifacts.
- 7 Interpretability: The feature-importance method reveals factors contributing to disagreement between clinician and AI policies, but not why the AI recommends each dose.At best, it provides a global interpretability measure across patients and time points.
8 Externally Validated?
The paper’s external-validation claim is difficult to assess because the eRI cohort’s sepsis incidence and treatment-record structure are insufficiently clear, while dose-response patterns differ substantially between cohorts.
- 8. Externally Validated?: The external-validation cohort contains 3.3 million ICU patients supplied by Philips Healthcare, but the critique questions whether it is a valid validation dataset.The authors use this cohort to address external validation, which is needed to assess robustness beyond the training data.
- 8. Externally Validated?: The eRI cohort’s validity is difficult to evaluate because its sepsis incidence cannot be calculated from the presented data and the authors used a modified sepsis definition.The reported MIMIC-III incidence was 31.4%, whereas applying Figure 6 numbers to Philips data yields an implausible 83.5%, attributed to biased exclusions.
- 8. Externally Validated?: The eRI treatment records may cluster within individual four-hour bins, leaving long periods without recorded fluids or vasopressors.MIMIC appears to have denser recording, with 20.4 intravenous-fluid and 31.1 vasopressor records reported in the cited passage.
- 8. Externally Validated?: Under-dosing IV fluids by up to 1.2 L appears mortality-neutral in Philips data but nearly doubles mortality in MIMIC, while vasopressor dose-response also differs substantially.These cohort differences raise questions about whether the AI Clinician performs similarly across datasets.
- 8. Externally Validated?: Dose-mortality curves alone cannot establish AI performance because individual treatment-dose pairs are confounded by baseline comorbidities and timely antibiotic treatment.A patient receiving extra fluid may have been sicker initially, while the AI Clinician may have chosen to withdraw treatment.
9 The Wrong Cohort Entirely
The critique argues that the evaluation cohort is dominated by patients who received no vasoactive treatment, allowing the AI Clinician’s apparent success to be driven by recommending inaction.
- 9. The Wrong Cohort Entirely: The authors trained and evaluated the AI Clinician on patients meeting Sepsis-3 criteria, but the critique questions whether this is the appropriate evaluation group.The concern is that the selected cohort does not adequately represent patients requiring clinical intervention.
- 9. The Wrong Cohort Entirely: Nearly two-thirds of patients received no vasopressors during their ICU course, so only about one-third warranted meaningful vasoactive-treatment consideration.Figure 8 is presented as evidence that the evaluation disproportionately rewards non-intervention.
- 9. The Wrong Cohort Entirely: The evaluation’s apparent preference for doing nothing is linked to importance-sampling metrics and a model biased toward healthier patients.The critique states that healthier patients comprise nearly two-thirds of both cohorts, helping explain why zero drug appears successful.
- 9. The Wrong Cohort Entirely: Because intermediate rewards are absent and treatments are binned over four hours, the AI Clinician could learn to administer vasopressors to normotensive patients.The critique identifies this as potentially iatrogenic because evidence-based sepsis practice reserves vasopressors for hypotension, commonly MAP < 65.
10 Final Comments
The final comments call for greater reproducibility and caution that promoting impressive but insufficiently vetted big-data claims can erode public trust.
- 10. Final Comments: The corresponding authors had not provided requested source code by October 2018, limiting independent reproduction of the work.A later GitHub link provided Matlab code for reproducing parts of the analysis, but the exact cohort may remain unrecoverable without more guidance.
- 10. Final Comments: The critique warns that hasty promotion of impressive but unvetted claims contributes to erosion of public trust and a possible “big-data winter.”The passage connects this concern to the broader problem of insufficient scrutiny in medical machine-learning research.
- 10. Final Comments: Reproducing the study requires consequential preprocessing choices, including handling missing values and selecting or discarding infusions in MIMIC-III.These choices can affect the cohort and evaluation even when code is available.