PaperlayerView exact revision on arXiv

Source-linked AI summary

Unknown is not normal: separating language-model extraction from rule-based decision logic for clinical risk scores

Nicolás Vera Zúñiga

arXiv:2609.34112v1cs.CLcs.AI

TL;DR

Undocumented findings can be mistaken for normal in clinical risk-score calculations, despite notes often not determining the decision category. The paper separates three-state LLM extraction from code that computes bounds and asks only decision-changing questions. Bounds matched ask-all accuracy with about half the questions, while missing-as-normal under-triaged patients and real case reports often lacked enough information to determine a category.

  • Problem

    When clinical notes omit findings, treating them as normal can misclassify risk, while LLMs may commit before a category is determined or abstain when it is determined.

    Source: §1 · p. 1

  • Method

    An LLM extracts present, absent, or unknown facts, and deterministic code computes score bounds and asks about unknowns only when they could change the decision category.

    Source: §1 · p. 2, §2 · p. 3

  • Results

    99.4% accuracy for bounds and ask-all, with 0.92 vs 1.78 questions per case; missing-as-normal accuracy was 91.2–91.8%, with 8.2–8.8% under-triage.

    Source: §3 · p. 4

  • Takeaways & Limitations

    The discussion reports that bounds avoided premature commitment and irrelevant questions, halved questions, and worked with a 9B local extractor; only 52% of real reports determined the category.

    Source: §3 · p. 5, §4 · p. 7

    The synthetic notes were cleaner and more complete than real notes and were written by a model from the Claude extractor’s family, so absolute accuracies are optimistic.

    Source: §4 · p. 7

Abstract

from arXiv · show

Large language models (LLMs) are increasingly used to compute clinical risk scores from free-text notes. Notes are often incomplete, and treating undocumented findings as normal can silently misclassify patients. We test whether separating three-state extraction (present, absent or unknown, by an LLM) from decision logic (deterministic code computing score bounds over unknown inputs) lets a system ask only questions that can change the decision. On 1,200 synthetic emergency cases across six calculators (HEART, CURB-65, qSOFA, PERC, Wells, Cockcroft-Gault), with a simulated clinician answering questions, we compared this bounds policy with asking for every missing input, a missing-equals-normal schema, and an end-to-end LLM agent (Claude Opus 5.5). With Claude Haiku 4.5 as extractor, the bounds policy matched ask-all accuracy (99.4% vs 99.4%) with half the questions (0.92 vs 1.78 per case) and no irrelevant ones. Treating missing as normal dropped accuracy to 91.2% and under-triaged 8.5% of patients (95% CI 7.1-10.2), and under-triage persisted under messy notes and a noisy clinician. The agent was equally accurate under ideal conditions (99.6%) but 9.5% of its questions were irrelevant; with a noisy clinician it was less accurate than the bounds policy (83.5% vs 87.0%, p<0.001) and committed prematurely in 2.7% of cases (bounds: 0%). A 9B local model as extractor reached oracle-level accuracy (99.8%). In 584 real case reports from MedCalc-Bench, only 52% contained enough information to determine the category (HEART 13%). Routing decisions through code that reasons explicitly about unknowns avoids premature commitment and irrelevant questions, halves the questions asked, and works with small local models.

1. Introduction

Clinical score calculators use documented inputs to assign decision categories, but treating unmentioned findings as normal can place patients in lower-risk categories. The paper tests a division of labour in which an LLM extracts three-state facts and code determines whether unknowns can change the category before asking questions.

1. Introduction

  • Clinical calculators have explicit input thresholds, but the decision category—not the exact score—is usually what matters clinically.For HEART, scores 0–3 are low risk, 4–6 moderate, and 7 or more high.Source: §1 · p. 1
  • Treating an undocumented finding as absent can lower a patient’s risk category, while LLMs may also commit when the category is undetermined or abstain when it is determined.The paper frames the challenge as avoiding both premature commitment and unnecessary abstention.Source: §1 · p. 1
  • The proposed pipeline has an LLM extract present, absent, or unknown facts, then code computes score bounds and asks only about unknown inputs that could change the category.Each clinician answer updates the bounds.Source: §1 · p. 2, Figure 1 · p. 2
  • The study compares bounds-based questioning with asking about every missing input, treating missing as normal, and an end-to-end LLM agent.Its four hypotheses address question reduction at equal accuracy, agent questioning and premature answers, missing-as-absent errors, and confirmation of low-confidence critical values.Source: §1 · p. 2

2. Methods

The methods implement six calculators with explicit decision categories, then compare policies that handle unknown inputs differently. Evaluation uses synthetic notes, several extractors and clinician conditions, plus a real-note analysis of whether documented facts determine the category.

2. Methods

  • Code enumerates achievable scores over unknown inputs and marks a category determined when only one category remains possible.An unknown is decision-relevant if some completion of the other unknowns makes its value change the category; property-based tests checked the bounds and relevance labels.Source: §2 · p. 2
  • The synthetic cohort sampled 200 patients per calculator, left each input undocumented with probability 0.3, and resampled cases so half had an undetermined category.Notes included traps such as negations, other units, prior-visit values, superseded readings, and medication-implied comorbidities; 559 messy-note cases passed validation.Source: §2 · pp. 2–3
  • Extractors returned a status, value, unit, evidence quote, and confidence; code rejected unsupported quotes and implausible values, converted units, and evaluated oracle, Haiku, and local Qwen models.Confidence was calibrated on a 30% development split.Source: §2 · p. 3
  • The policies were ask-all, bounds-based questioning, bounds plus low-confidence decision-critical checks, missing-as-normal, and an end-to-end Claude Opus 5.5 agent.Bounds recomputes after each answer; when no question can settle the category, policies abstain.Source: §2 · p. 3
  • Primary outcomes were decision-category accuracy and questions per case, with under-triage, over-triage, premature commitment, irrelevant questions, and extraction performance among secondary outcomes.A simulated clinician sometimes could not answer, answered incorrectly, or supplied a numeric range; the real-note analysis used 584 training notes and assessed category determination and missing-as-normal accuracy.Source: §2 · pp. 3–4

3. Results

Bounds-based questioning matched ask-all accuracy while asking substantially fewer questions, and the missing-as-normal policy under-triaged patients. Under noisy answers, the agent was less accurate and sometimes committed prematurely; real reports often lacked enough documented information to determine a category.

3. Results

  • 99.4% accuracy for both bounds and ask-all, with 0.92 vs 1.78 questions per case, respectively; bounds asked no irrelevant questions.This comparison used Claude Haiku 4.5 on clean notes; about half of ask-all questions could not change the decision.Source: §3 · p. 4, Table 1 · p. 5
  • With a noisy clinician, the agent reached 83.5% accuracy versus 87.0% for bounds and committed prematurely in 2.7% of cases, while code policies never did.A noisy clinician reduced accuracy across policies by 12–16 points, mostly through abstentions when the decisive answer was unavailable.Source: §3 · p. 4
  • On messy notes, Haiku bounds accuracy was 98.6%, bounds plus checks reached 99.1%, and Qwen reached 99.5%.The agent reading notes directly reached 99.8% versus 98.6% for bounds with Haiku (p=0.047).Source: §3 · p. 4
  • Only 52% of 584 real training case reports had a category determined by documented facts, ranging from 90% for PERC to 13% for HEART.Under missing-equals-normal, categories were correct for 69% of notes, including 22% for HEART; truth stayed within bounds for 93–95%.Source: §3 · p. 5

4. Discussion

Treating undocumented findings as normal under-triaged about one patient in twelve, while routing decisions through code avoided premature answers and irrelevant questions. The discussion emphasizes the limits of the evaluation and the need to report whether notes determine categories.

4. Discussion

  • Missing-as-normal under-triaged about one patient in twelve in every condition, including clean notes with a perfect clinician, and gave the correct category for only two-thirds of real case reports.The authors note that headline accuracy can obscure these safety differences.Source: §4 · p. 5
  • On real case reports, missing-as-normal gave the correct category for only two-thirds; Figure 4 contrasts note-determined categories with correctness under that convention.The authors say the analysis suggests MedCalc-Bench's convention overstates how often a category can safely be computed from the note alone.Source: §4 · pp. 5, 7, Figure 4 · p. 6
  • Figure 3 compares under- and over-triage across four conditions, assigning undetermined cases the highest-risk category still possible.Its horizontal lines show Wilson 95% confidence intervals.Source: Figure 3 · p. 6
  • The code-based policy halved questions and never answered early or asked irrelevant questions; a 9B local extractor matched perfect extraction, while the authors do not claim greater accuracy than a capable agent.The agent was as accurate under ideal conditions and more accurate than a small extractor on messy notes; the modular design's stated advantages include determinism and auditability.Source: §4 · p. 7
  • The authors position the determinacy check alongside active feature acquisition, selective prediction, and partial evaluation, and identify executable guideline standards as a natural home.They distinguish their work from ClinDet-Bench by adding question asking and comparison with an agent that can ask.Source: §4 · p. 7
  • Synthetic notes were cleaner and more complete than real notes, and shared a model family with the Claude extractor; absolute accuracies may therefore be optimistic.The authors consider question counts and safety differences more transferable; the clinician was simulated with a simple noise model.Source: §4 · p. 7
  • The real-note analysis did not include question asking and used labels with known errors; accuracy comparisons were also underpowered near the ideal-condition ceiling.Calculator readings and priors were reviewed by one physician.Source: §4 · p. 7
  • The authors recommend keeping unknown distinct from absent, asking only about decision-changing inputs, and reporting how often notes determine categories.If information cannot be obtained, they recommend falling back to the higher-risk category.Source: §4 · p. 7

Declarations

The authors report the code and its archive, describe which materials are regenerated or unavailable, and disclose funding, competing interests, AI use, and ethics status.

Declarations

  • Code is available under an MIT licence and archived at Zenodo; synthetic data are regenerated from configuration and seed, while notes, extractions, and response cache are not released.The paper reports no funding or competing interests and describes AI use and the ethics basis for using synthetic data and published de-identified reports.Source: Declarations · p. 7

Supplementary material

The supplementary material points to additional methods and results, including cohort priors, missingness sensitivity, confidence calibration, real-case tables, and sensitivity to implausible synthetic cases.

Supplementary material

  • The full supplement includes calculator readings, full result grids and paired comparisons, and exact prompts.It is identified as paper/supplement.pdf in the code repository.Source: Supplementary material · p. 10

S2. Synthetic cohort and population priors

  • The synthetic cohort contains 1,200 cases, with undetermined categories comprising 50% by design; calculator parameters were sampled from intended-use populations and reviewed by the author.Parameters were sampled independently except for a blood-pressure constraint, and the PERC cohort represents low gestalt pre-test probability.Source: S2. Synthetic cohort and population… · p. 10

S3. Missingness sensitivity (oracle extraction, natural share of undetermined cases)

  • Missingness sensitivity uses fresh 200-case-per-calculator cohorts at each missingness level and reweights strata to the natural share of undetermined cases.The accompanying table headings identify questions per case and undetermined share.Source: S3. Missingness sensitivity (oracle… · p. 11

S4. Confirmation threshold sweep (bounds + checks policy, clean notes, ideal clinician)

  • The bounds-plus-checks policy confirms decision-critical extracted values when calibrated confidence is below a threshold; the sweep reports accuracy, questions, and confirmations per case.The supplied table headings include the confirmation-confidence threshold and those three measures.Source: S4. Confirmation threshold sweep… · p. 11

S5. Calibration of extraction confidence

  • Calibration fits isotonic and temperature methods on a seeded 30% development split and evaluates them on the remainder; isotonic ECE is 0.002 for both Haiku and Qwen3.5-9B.Raw ECE is 0.014 for Haiku and 0.048 for Qwen3.5-9B; temperature-calibrated ECE is 0.003 and 0.033, respectively.Source: S5. Calibration of extraction confidence · p. 12

S8. Real case reports, all splits and extractors

S10. Sensitivity to implausible synthetic cases

  • Twelve of 1,200 synthetic cases were flagged for implausible HEART characteristics, and the main results were recomputed without them.The flagged patterns included established atherosclerotic disease with no risk factors, or age under 40 with at least three risk factors or atherosclerotic disease.Source: S10. Sensitivity to implausible… · p. 13
Loading 2609.34112v1…