Source-linked AI summary
Large language models exhibit unreliable updating of clinical judgment as patient evidence evolves
Min Zeng, Rui Zhang
TL;DR
Clinical LLM evaluations often treat outputs as isolated decisions, leaving unclear whether models revise judgments appropriately as patient evidence changes. Using matched ICU trajectories and controlled interventions, the study finds harmful longitudinal dependence, asymmetric responses to worsening versus improving evidence, and sensitivity to supplied prior beliefs; evidence validation identifies a more reliable but less frequently retained subset of revisions.
Problem
Many clinical LLM evaluations treat outputs as isolated decisions, leaving limited evidence about whether models appropriately revise judgments as patient evidence evolves.
Source: Introduction · p. 2Method
The study compares independent and prior-conditioned assessments on matched ICU trajectories, uses controlled interventions to vary respiratory evidence and prior beliefs separately, and tests prompting and EVLU.
Source: Introduction · pp. 2–3Results
Across controlled tests, worsening evidence elicited stronger revisions than matched improving evidence and supplied prior beliefs influenced estimates; EVLU-2 reduced the harmful fraction among changed predictions from 60.4% to 46.9%, retaining 5.63% of post-initial predictions.
Source: Discussion · pp. 14–16Takeaways & Limitations
Longitudinal belief updating is a distinct reliability dimension, and evidence validation can identify more reliable revisions while substantially reducing coverage.
Source: Discussion · pp. 16–17EVLU is a conservative proof of concept: even EVLU-2 did not improve mean absolute error over independent inference on average, and EVLU-3 retained only 1.68% of post-initial predictions.
Source: Discussion · p. 16
Abstract
from arXiv · showhide
Large language models (LLMs) are increasingly explored for clinical reasoning, but whether they appropriately revise judgments as patient evidence evolves remains unclear. We evaluated longitudinal belief updating using matched intensive-care trajectories from electronic health records. Across diverse LLMs, conditioning on a preceding judgment more often increased than reduced prediction error when estimates changed, replicated for a second endpoint. Controlled interventions revealed two failure modes. First, with preceding assessment fixed, models responded more strongly to worsening than matched improving respiratory evidence; this asymmetry persisted after headroom normalization at moderate and strong evidence levels. Second, with current evidence fixed, increasing prior risk from 10% to 90% shifted estimates by 26.2 percentage points, demonstrating causal influence of prior model beliefs. Prompting did not restore reliable updating. Evidence-Validated Longitudinal Update (EVLU) identified fewer, more reliable revisions, revealing a reliability-coverage trade-off. These findings establish longitudinal belief updating as a distinct dimension of LLM reliability.
Introduction
Clinical LLM evaluations often measure isolated decisions, although care requires judgments to be maintained or revised as patient evidence changes. This study uses matched ICU trajectories and controlled interventions to distinguish evidence-responsive updating from influence of a model’s prior judgment.
Introduction
- Static or aggregate clinical benchmarks do not establish whether models appropriately revise judgments as patient evidence evolves.Longitudinal evaluations using real patient-care data remain uncommon, and accuracy has been the dominant evaluation dimension.Source: Introduction · p. 2
- A prior model assessment can shape later judgments beyond what current patient evidence supports, potentially propagating inaccurate assessments across time.Appropriate updating requires maintaining a supported judgment, revising it when contradicted, and adjusting revision to evidence strength.Source: Introduction · p. 2
- Existing evaluations do not disentangle evolving patient evidence from the model’s preceding judgment because both change together in natural trajectories.Independent manipulation of new evidence and prior judgment can reveal updating patterns that snapshot-level evaluations miss.Source: Introduction · p. 2
- The study compares independent and prior-conditioned assessments across diverse LLMs and replicates the endpoint analysis for vasopressor initiation.Controlled interventions separately manipulate respiratory evidence and supplied prior beliefs; the study also tests prompting and EVLU.Source: Introduction · pp. 2–3
Results
Across matched ICU trajectories, the study evaluates longitudinal predictions, controlled changes to respiratory evidence and prior beliefs, and mitigation strategies. Longitudinal context often harmed paired prediction accuracy, while controlled interventions exposed asymmetric evidence response and sensitivity to supplied priors; evidence validation retained a smaller, more reliable subset of revisions.
Longitudinal self-context more often worsens than improves paired prediction accuracy
- 46.20% of paired snapshots had increased absolute prediction error under longitudinal context, versus 30.28% with reduced error, raising mean absolute error by 3.62 percentage points.Across 6,580 snapshots, AUROC was 0.545 under independent evaluation and 0.541 under longitudinal evaluation; Brier score rose from 0.281 to 0.284.Source: Longitudinal self-context more often… · p. 3, Figure 2 · p. 5
Harmful longitudinal dependence replicates in hemodynamic deterioration
- Across the hemodynamic cohort, mean absolute prediction error rose by 3.44 percentage points, and longitudinal inference increased error in 46.7% of cases; all eight models showed increased error.In the hemodynamic cohort, AUROC nevertheless increased from 0.574 to 0.582; across models, harmful-update fractions exceeded 50%.Source: Harmful longitudinal dependence… · pp. 4–6
Belief revision is directionally appropriate and dose-responsive to controlled evolving evidence
- Worsening evidence raised predicted risk by 16.43, 30.86, and 45.21 percentage points at subtle, moderate, and strong strengths; matched improving evidence produced smaller reductions.Revision magnitude increased with evidence strength in 99.93% of worsening and 97.58% of improving transitions; raw asymmetry reached 19.65 percentage points under strong evidence.Source: Belief revision is directionally… · p. 6, Figure 3 · p. 7, Belief updating exhibits… · p. 8
Directional asymmetry persists after accounting for probability headroom and preceding prediction error
- After headroom normalization, asymmetry remained deterioration-directed for moderate and strong evidence and positive across prior-risk and preceding-error strata.Strong-evidence asymmetry was positive across all four prior-risk strata and low-, intermediate-, and high-error strata, including relatively accurate prior estimates.Source: Directional asymmetry persists after… · p. 8, Figure 4 · p. 9
Neutral and redundant evidence elicit little belief revision on average
- Neutral evidence produced no change, while redundant evidence yielded a mean absolute revision of 0.74 percentage points and no numerical change in 96.99% of transitions.Strong-evidence asymmetry was also deterioration-directed in both ventilation-positive and ventilation-negative trajectories.Source: Neutral and redundant evidence elicit… · p. 10, Directional asymmetry is consistent… · p. 10
Controlled prior-belief interventions causally shift subsequent risk estimates
- With current evidence fixed, raising supplied prior risk from 10% to 90% shifted updated ventilation risk by 26.18 percentage points and increased error when the prior was outcome-misaligned.The prior-belief effect replicated for vasopressor risk: a 10% to 90% prior shift changed updated risk by 24.48 percentage points.Source: Controlled prior-belief interventions… · pp. 10–11, Figure 5 · p. 11
Simple prompting does not eliminate harmful longitudinal dependence
- Evidence-first and anti-anchoring prompts reduced prior-directed shifts but did not improve reliability; anti-anchoring increased harmed snapshots and mean absolute prediction error.Evidence-first prompting did not significantly reduce harmful-update asymmetry or mean absolute error.Source: Simple prompting does not eliminate… · p. 12
Evidence validation identifies more reliable longitudinal revisions
- EVLU-2 reduced the harmful fraction among changed predictions to 46.90%, but retained the longitudinal branch for only 5.63% of post-initial predictions.EVLU-3 retained 1.68% of predictions and was not statistically distinguishable from EVLU-2 on harmful fraction, asymmetry, or mean absolute error.Source: Evidence validation identifies more… · pp. 12–13, Figure 6 · p. 14
Discussion
Longitudinal self-context more often worsened than improved prediction accuracy when it changed estimates, while controlled tests exposed prior-belief influence and asymmetric responses to evidence. Evidence validation selected more reliable revisions, but at substantial cost to coverage.
Scope and implications
- The study identifies longitudinal belief updating as a distinct reliability challenge, with harmful self-context effects across model families and replication for a second endpoint.The authors caution that their cohorts came from MIMIC-IV, trajectories primarily used structured variables, and mechanistic, prompting, and evidence-validation experiments focused on the primary model.Source: Discussion · pp. 14, 16, 17
Prior assessments as longitudinal context
- With current evidence fixed, raising supplied prior risk from 10% to 90% shifted estimates by 26.18 percentage points for mechanical ventilation and 24.48 percentage points for vasopressor initiation.The similar shifts across the two endpoints demonstrate causal input–output dependence, not a human-like cognitive anchoring mechanism.Source: Discussion · pp. 14–15
Directional evidence responsiveness
- Worsening respiratory evidence produced larger risk revisions than matched improving evidence, even after headroom normalization at moderate and strong evidence levels.The asymmetry persisted across preceding-error and outcome strata; probability-scale constraints, inaccurate preceding predictions, and outcome composition were not simple explanations.Source: Discussion · p. 15
Prompting
- Evidence-first and anti-anchoring prompts reduced prior-directed shifts but did not improve reliability; anti-anchoring increased harmed snapshots and mean absolute error.Generic instructions do not provide case-specific validation that a revision is justified by current evidence.Source: Discussion · p. 15
Evidence validation
- With EVLU-2, the harmful fraction among accuracy-changing predictions fell from 60.4% to 46.9%, while only 5.63% of post-initial predictions retained the longitudinal branch.EVLU-2 revisions were more reliable than rejected revisions beyond simply restricting revision frequency; random gates retaining the same number of revisions did not show the same shift.Source: Discussion · p. 16
Limitations
- EVLU remains a conservative proof of concept: EVLU-3 retained only 1.68% of post-initial predictions, and EVLU-2 did not improve mean absolute error over independent inference on average.EVLU relies on manually specified respiratory evidence rules, and its more reliable operating points retained only a small fraction of revisions.Source: Discussion · pp. 16–17
Scope and implications
- Longitudinal systems could validate whether context-induced revisions agree with independently evolving clinical evidence before propagating them.Evaluation should assess revision direction, magnitude, evidence dependence, and consequences, not only discrimination and calibration.Source: Discussion · p. 16
Methods
The study used matched MIMIC-IV adult ICU cohorts to evaluate 24-hour deterioration risk from serial structured clinical snapshots. It constructed a respiratory endpoint cohort and represented each snapshot with current-state and newly observed evidence.
Data source and study cohort
- MIMIC-IV adult ICU stays required age 18 years or older and at least 30 hours in the ICU; invasive ventilation was the primary endpoint, with vasopressor initiation as a replication endpoint.The vasopressor analysis used corresponding hemodynamic fields and otherwise kept the independent and longitudinal inference protocol unchanged.Source: Data source and study cohort · p. 17
Invasive ventilation outcome and matched cohort construction
- The matched respiratory cohort included 1,000 ventilation-positive and 1,000 negative ICU stays, with positive cases required to have at least two snapshots before ventilation onset.Positive events occurred 12–72 hours after ICU admission; matching used sex, first ICU care unit, and age within 10 years, and snapshots predicted ventilation in the subsequent 24 hours.Source: Invasive ventilation outcome and… · p. 18, Data source and study cohort · p. 18
Longitudinal clinical trajectory construction
- Each snapshot represented current state from measurements in the preceding 24 hours and new evidence observed since the prior snapshot, including vital signs, respiratory measures, laboratory results, and respiratory-support events.Measurements were summarized with the most recent value, minimum, maximum, median, and measurement count; the resulting cohort had 6,580 snapshots, all positive-stay snapshots before ventilation onset.Source: Longitudinal clinical trajectory… · pp. 18–19
Independent vasopressor cohort
The study constructed a matched vasopressor-initiation cohort and compared independent with longitudinal model assessments, then repeated the evaluation across model families and scales.
Independent vasopressor cohort
- The vasopressor cohort comprised 1,000 matched positive-negative pairs, with prediction targeting continuous vasopressor initiation within 24 hours.Positive stays had first qualifying initiation 12–72 hours after ICU admission; matching used sex, first ICU care unit, age, and follow-up coverage.Source: Independent vasopressor cohort · p. 19
Independent and longitudinal LLM evaluation
- Independent assessments used each snapshot alone, whereas longitudinal assessments also received the preceding structured judgment and were instructed to maintain or revise it using new evidence.The hemodynamic schema included a risk estimate and evidence for and against deterioration; Qwen3-8B was the primary model.Source: Independent and longitudinal LLM… · p. 20
Cross-model evaluation
- The cross-model evaluation applied the same snapshots, prediction target, and independent-versus-longitudinal prompt structure to seven additional general-purpose and medically adapted LLMs.Models were evaluated locally with deterministic decoding and model-specific chat templates.Source: Cross-model evaluation · p. 20
Controlled respiratory evidence interventions
Controlled respiratory interventions varied newly presented evidence while fixing prior patient state and model assessment; separate interventions varied supplied prior risk while holding current evidence fixed.
Controlled respiratory evidence interventions
- For respiratory transitions, the preceding objective state and model assessment were fixed while newly presented respiratory evidence was varied across worsening and improving directions at three strengths.Interventions changed respiratory rate, oxygen saturation, and supplemental oxygen flow by prespecified magnitudes, subject to physiological bounds.Source: Controlled respiratory evidence… · pp. 20–21
- Redundant and neutral controls tested whether revisions reflected directional clinical evidence rather than merely the presence of additional prompt content.The redundant condition restated prior measurements; the neutral condition specified that no new respiratory measurements or directional evidence were available.Source: Controlled respiratory evidence… · p. 21
Controlled prior-belief interventions
- Prior-belief interventions held current evidence and other prompt content constant while varying only supplied risk across five levels from 10% to 90%.Analyses measured dose response, the within-snapshot difference between extreme priors, monotonicity, and error under outcome-aligned versus outcome-misaligned priors.Source: Controlled prior-belief interventions · pp. 21–22
Prompt-based mitigation
The authors tested two prompting strategies intended to reduce dependence on previous model assessments, comparing them with standard longitudinal prompting against independent predictions.
Prompt-based mitigation
- The evidence-first prompt required an initial assessment of the current snapshot before reviewing the prior assessment as historical context.The anti-anchoring prompt warned that the prior could be inaccurate, outdated, or overconfident and should not be treated as evidence about the current patient.Source: Prompt-based mitigation · p. 22
- Both strategies emphasized that current clinical evidence should take precedence when it conflicted with the previous assessment.Source: Prompt-based mitigation · p. 22
- Both strategies were evaluated on the same 4,580 snapshots and compared with standard longitudinal prompting using independent current-snapshot predictions as the reference.Outcomes included harmful context effects, error changes, normalized asymmetry, and prior-directed movement.Source: Prompt-based mitigation · p. 22
Evidence-validated longitudinal updating
EVLU is a post hoc rule for retaining longitudinal revisions only when evolving respiratory evidence supports them, with thresholds used to examine the reliability-coverage trade-off.
Evidence-validated longitudinal updating
- The respiratory evidence score aggregates directional votes from seven available variables, using prespecified change thresholds to distinguish improvement, stability, and deterioration.The variables include SpO2, FiO2, oxygen flow, respiratory rate, PaO2, PaCO2, and pH.Source: Evidence-validated longitudinal updating · p. 23
- EVLU retains a non-zero context-induced revision only when its direction agrees with respiratory evidence and the evidence score reaches the selected threshold; otherwise, it reverts to the current-snapshot estimate.It is a rule-based post hoc procedure requiring neither additional model training nor LLM inference.Source: Evidence-validated longitudinal updating · pp. 22–23
- Concordance thresholds k = 1, 2, and 3 were evaluated as a sensitivity analysis, comparing EVLU with unrestricted longitudinal and independent inference.The evaluation tracked harmful retained changes, normalized harmful-update asymmetry, mean absolute error change, and retention of longitudinal updates.Source: Evidence-validated longitudinal updating · pp. 23–24
Evaluation metrics
The evaluation combines predictive-performance measures with measures of how preceding model context and controlled evidence interventions change risk estimates. It also assesses whether those changes align with evidence direction and affect prediction error.
Evaluation metrics
- The natural longitudinal evaluation compares AUROC, Brier score, and MAE for predicted invasive-ventilation risk against the 24-hour outcome.Paired differences between longitudinal and independent risk estimates quantify the effect of prior model context.Source: Evaluation metrics · p. 24
- A context-induced change is classified as helpful, harmful, or having no effect according to whether it reduces, increases, or leaves absolute prediction error unchanged.The harmful fraction is calculated among snapshots with a non-zero change in absolute prediction error.Source: Evaluation metrics · p. 24
- A prior-directed shift measure quantifies attraction to the supplied prior belief and is also used in prompting analyses.Positive normalized asymmetry indicates stronger deterioration-directed updating after accounting for unequal probability headroom.Source: Evaluation metrics · p. 25
- Belief revision is measured as the change from the model’s prior risk estimate to its updated estimate after a controlled evidence intervention.Positive change is directionally appropriate for worsening evidence; negative change is appropriate for improving evidence.Source: Evaluation metrics · p. 25
- Evidence-strength sensitivity is assessed across subtle, moderate, and strong interventions, with monotonicity defined as non-decreasing revision magnitude as evidence strength increases.Directional asymmetry is evaluated on the raw probability scale and after normalizing for available upward or downward probability headroom.Source: Evaluation metrics · p. 25
Statistical analysis
Primary comparisons use paired clinical snapshots or transitions, with uncertainty estimated by resampling at the ICU-stay level. The analyses compare natural longitudinal predictions, controlled interventions, prompting strategies, and EVLU policies.
Statistical analysis
- Primary comparisons are paired at the clinical-snapshot or transition level, and 95% confidence intervals generally use 10,000 ICU-stay-level bootstrap replicates.Stay-level resampling preserves dependence among multiple observations from the same ICU stay.Source: Statistical analysis · p. 26
- Natural longitudinal analyses compare independent and longitudinal predictions at matched snapshots and classify context-induced changes by their effect on absolute prediction error.Cross-model uncertainty for harmful-update asymmetry is estimated by ICU-stay-level bootstrap resampling.Source: Statistical analysis · p. 26
- Controlled respiratory interventions pair worsening and improving conditions within each transition and assess directional asymmetry on raw and headroom-normalized scales.The analyses also stratify normalized asymmetry by preceding risk, preceding prediction error, and clinical outcome.Source: Statistical analysis · p. 26
- Controlled prior-belief conditions are paired within each snapshot while current evidence is fixed, with susceptibility assessed across five supplied prior-risk levels.Bootstrap uncertainty covers the 10-point increase, 90%-versus-10% contrast, monotonicity, and outcome alignment of priors.Source: Statistical analysis · p. 26
- Prompting comparisons use the same post-initial snapshots and independent predictions as the reference, while EVLU policies are compared with unrestricted longitudinal and independent inference.EVLU-2’s 258 retained revisions are tested against random retention of 258 among 3,503 nontrivial revisions using 10,000 assignments.Source: Statistical analysis · p. 26
- Exploratory subgroup and sensitivity analyses receive no multiple-comparison adjustment and are interpreted descriptively rather than as independent confirmatory tests.Primary analyses emphasize paired effect estimates with confidence intervals.Source: Statistical analysis · p. 27
Data Availability
The study’s clinical data come from MIMIC-IV, a publicly available, de-identified electronic health record database.
Data Availability
- MIMIC-IV access is available through PhysioNet to credentialed users who complete required training and a data-use agreement.The study’s derived cohorts can be reconstructed from MIMIC-IV using the described cohort-construction procedures and accompanying code.Source: Data Availability · p. 27
Funding
The work was supported by six U.S. federal agencies, and the funders had no role in the study or manuscript preparation.
Funding
- Support came from NCCIH, NIA, NCI, NIDDK, NIMHD, and the FDA through the listed grants.The funders had no role in study design, data collection and analysis, the decision to publish, or manuscript preparation.Source: Funding · p. 27