Source-linked AI summary
Foundation Agents Meet Agentic Deep Research: Evidence-Grounded Clinical Code Forecasting
Junda Wang, Meysam Ghaffari, Akshat Choube, Mohsen Sharifi Renani, Hong Yu, Carlos Morato
TL;DR
Next-encounter ICD forecasting must predict a future, multi-label code set from records available beforehand. ICD-Deepresearch combines foundation-model candidate generation with bounded evidence expansion and fixed-budget selection, outperforming local comparators while retrieving more useful evidence in physician audits.
Problem
Next-encounter ICD forecasting must project a patient's longitudinal trajectory into a future, multi-label code set when the target encounter is unavailable.
Source: §1 · p. 1Method
ICD-Deepresearch combines EHR-prior and independent language-model candidate generation with bounded medical search, code verification, and joint fixed-budget selection.
Source: §1 · p. 1Results
ICD-Deepresearch outperforms candidate-generation, standalone GPT-5 web-search, isolated-path, and Medical Deep Research comparators, while physician audits find a higher proportion of useful evidence.
Source: §1 · p. 2Takeaways & Limitations
The results support controlling candidate expansion, evidence acquisition, and fixed-budget selection rather than treating search as retrieval of the future answer.
Source: §1 · p. 2Reason evaluation uses a gold-aware GPT-5 semantic judge and is not independent clinical validation; dual-clinician annotation would strengthen the evaluation.
Source: Limitations and Ethics · p. 22
Abstract
from arXiv · showhide
Next-encounter ICD forecasting predicts which standardized diagnosis codes will be documented at a future visit from the longitudinal record available beforehand. The task is prospective and multi-label: the target note does not yet exist, and several codes may be correct. Structured EHR foundation models capture recurrence and temporal progression, whereas language foundation models generate flexible diagnostic hypotheses. We introduce ICD-Deepresearch, a DeepResearch workflow that composes these predictive foundation models with medical search and ICD dictionaries. Because no source reveals the future code set, research evaluates candidate transitions by linking patient evidence, external clinical relations, and exact code semantics under a fixed top-K budget. Candidate Generation uses SparseEHR to produce an EHR Prior that initializes two bounded Research Expansion rounds; an independent GPT-5 Direct Forecast supplies complementary candidates. Final Selection validates, deduplicates, and jointly ranks both paths, after which a separate module writes rationales without changing predictions. Finally ICD-Deepresearch achieves patient-averaged precision/recall of 24.60/35.09% on MIMIC-III and 25.14/48.32% on MIMIC-IV. Physicians rate 51% and 68% of its retrieved documents useful, compared with 22% and 39% for standalone GPT-5 web search and 32% and 41% for Medical Deep Research. ICD-Deepresearch therefore improves over the registered local comparators while retrieving evidence with higher physician-rated usefulness than the standalone research systems
1 Introduction · 2 Related Work · 3 Method
ICD-Deepresearch forecasts next-encounter ICD codes by combining structured-EHR and language-model candidates with patient-anchored research, exact-code verification, and fixed-budget joint selection. It outperforms local comparators while retrieving fewer, more useful documents according to physician audits.
1 Introduction
- Next-encounter forecasting is prospective and multi-label: models rank the complete future ICD code set using only the pre-target longitudinal record.The target encounter’s symptoms, findings, tests, procedures, documentation, and codes are unavailable at prediction time.Source: §1 · p. 1
- ICD-Deepresearch outperforms Candidate Generation Only, standalone GPT-5 Web Search, isolated candidate paths, and Medical Deep Research, while retrieving fewer documents with a higher useful-evidence proportion.The system is evaluated on MIMIC-III and MIMIC-IV, and physician audits assess retrieved evidence usefulness.Source: §1 · p. 2
2 Related Work
- Prior work covers longitudinal multi-label forecasting, adaptive retrieval, clinical agents, Medical Deep Research, and retrospective coding, but ICD-Deepresearch links patient trajectories to candidate-specific transitions and exact code semantics.Unlike retrospective coding or query-driven clinical retrieval, forecasting begins without an explicit future diagnosis or target note.Source: §2 · p. 2
System Overview
- The three-stage pipeline generates candidates, performs exactly two bounded research rounds, jointly selects vocabulary-valid top-K codes, and writes rationales only after predictions are fixed.Research Expansion is initialized only by the EHR Prior, while Direct Forecast remains independent until Final Selection.Source: System Overview · pp. 2–3, Figure 2 · p. 3
Task, Inputs, and Outputs
- The task maps each patient’s pre-target encounters, events, demographics, and available notes to an ordered list of unique, vocabulary-valid diagnosis codes evaluated at K=10 and K=20.A single ranking supports both reported cutoffs, while inference-time generation, tools, ranking, and explanation use only the pre-target record.Source: Task, Inputs, and Outputs · p. 3
Two Candidate Paths
- The two candidate paths pair SparseEHR’s structured-EHR EHR Prior with an independent GPT-5 Direct Forecast, preserving language-model hypotheses without letting them determine the research agenda.Candidate Generation is target-blind and fixed before Research Expansion; Direct Forecast can use structured events and pre-target notes but joins only at Final Selection.Source: §1 · p. 1, Two Candidate Paths · pp. 3–4
Research Expansion
- Research Expansion runs Plan → ToolAct → ReadBind → Verify → UpdateRank twice, keeping patient clues, external transitions, and exact ICD meanings distinct while updating candidate rankings.Patient-anchored queries use Google or fixed PMC/arXiv corpora plus an ICD dictionary, and the second round conditions on the complete first-round state.Source: Research Expansion · p. 4
Joint Final Selection
- Final Selection canonicalizes and deduplicates the direct and researched candidates, then the Evidence-Aware Reranker jointly scores them using the record, generator ranks, metadata, and bound evidence.The RRF control changes only the selection rule, while post-selection explanations distinguish patient clues, clinical relations, ICD meanings, and uncertainty without changing predictions.Source: Joint Final Selection · p. 4, Post-Selection Explanation · p. 4
4 Experimental Design
The experiments assess forecasting performance, component controls, retrieval quality and efficiency, and post-selection explanations on held-out MIMIC-III/IV admissions. They use controlled local comparators, published reference values, and task-specific metrics and audits.
Evaluation logic
- Evaluation covers forecasting, component and selector controls, retrieval quality and efficiency, and post-selection explanations across four research questions.MIMIC-III and MIMIC-IV use held-out final admissions with ICD-9-CM and ICD-10-CM targets.Source: §4 · p. 4
Experimental specification
- Supplementary protocols specify cohort sizes, splits, outcome construction, prompts, retrieval budgets, model snapshots, seeds, implementation details, and controlled states.The published block’s baseline sources and numerical provenance are also documented in the supplement.Source: §4 · pp. 4–5
Comparators and controls
- Local comparisons include Candidate Generation Only, the full system, standalone GPT-5 Web Search, and controls isolating Direct and Research paths.Research-context variants use a fixed candidate pool; Medical Deep Research is a second standalone comparator without the EHR Prior, research ledger, or selector.Source: §4 · p. 4
Metrics and audits
- Table 1 reports patient-averaged P@K/R@K, while Table 2 reports pooled micro-P/R/F1 and patient-macro F1 within separate dataset-specific comparison blocks.Published THCM-CAL-protocol values are transcribed from Table 2 of Zhang et al. 2025 and form a separate block; comparisons remain within each table and dataset.Source: §4 · pp. 4–5, Table 1 · p. 5
- Retrieval audits measure source composition, metadata support and forecast usefulness, and blinded physician usefulness, while explanation audits use five reason dimensions and coverage-adjusted Reason5.Supplementary specifications define source sampling, efficiency accounting, explanation judge contracts, metrics, and inference.Source: §4 · p. 5
5 Results and Discussion
Across four research questions, ICD-Deepresearch improves forecasting, retrieval usefulness, and explanation semantics through joint candidate selection and patient-conditioned research, while search depth and temporal grounding remain state-dependent limitations.
RQ1: Complementary Candidate States Require Joint Selection
- 24.60/35.09% and 25.14/48.32% P@20/R@20 on MIMIC-III/IV, respectively, exceed GPT-5 + Web Search and Candidate Generation Only, while also leading both at K = 10.Relative gains over GPT-5 + Web Search are +1.83/+2.61 and +2.96/+16.76 precision/recall points; gains over Candidate Generation Only are +3.68/+5.25 and +1.79/+15.10.Source: RQ1: Complementary Candidate States… · p. 5
- 29.25 and 35.44 micro-F1 on MIMIC-III/IV, respectively, exceed every isolated path and standalone comparator, while the Evidence-Aware Reranker adds 2.52 and 0.59 points over RRF.The joint system’s superiority is evaluated within separate diagnostic cohorts and pooled metrics, not against the scale-cohort values.Source: RQ1: Complementary Candidate States… · p. 5
RQ2: Search Depth Is a State-Dependent Allocation Problem
- Two expansion iterations recover 144 additional EHRSHOT targets but enlarge the mean pool from 19.8 to 51.0 and reduce micro-F1 from 25.82% to 16.34%, showing that coverage gains can impose cardinality costs.Initial expansion improves MIMIC-III/IV micro-F1 by +3.06/+9.04 points, whereas follow-up changes it by −0.98/−0.004; MIMIC-IV follow-up yield is 12.96%, near its 13.01% break-even requirement.Source: RQ2: Search Depth Is a State-Dependent… · p. 5, Figure 4 · p. 7
- 26 of 28 MIMIC-IV follow-up matches occur in 13 patients, while 52 patients worsen; filtering retains 21 matches, shrinks the mean pool from 38.19 to 36.64, and raises micro-F1 from 26.02% to 26.54%.These concentrated gains motivate learning marginal expansion value from the current state and stopping when expected yield falls below cardinality cost.Source: RQ2: Search Depth Is a State-Dependent… · pp. 5–6
RQ3: Focused Retrieval Improves Physician-Rated Usefulness
- 51% and 68% physician-rated usefulness on MIMIC-III/IV, respectively, surpass GPT-5 + Web Search and Medical Deep Research despite 1,410 and 11,124 retrieved source occurrences.On matched cohorts, Research Expansion also uses 7.3×/3.4× fewer unique queries and records 16.8×/21.9× higher batched throughput, with patient-linked transition searches rather than generic code lists.Source: RQ3: Focused Retrieval Improves… · pp. 6–7, Table 3 · p. 6
RQ4: Explanation State Improves Semantics More Than Temporal Grounding
- ICD-Deepresearch improves exact ICD alignment by 0.07 and evidential calibration by 0.22 on 441 shared MIMIC-III hits, while MIMIC-IV quality gains are clearest in code semantics and calibration rather than next-encounter relevance.Coverage-adjusted quality rises from 1.36 to 1.57 on MIMIC-IV, but next-encounter relevance changes by only +0.03; temporal grounding remains weak.Source: RQ4: Explanation State Improves… · p. 7, Figure 3 · p. 7
- The remaining failures include weak temporal relations, query–candidate drift, definition–forecast conflation, semantic mismatches, horizon-polarity errors, and uncontrolled cardinality.These motivate verification linking an immutable pre-target patient span, a source-entailed directional relation, and an exact code expressing the same entity at the immediate forecasting horizon.Source: RQ4: Explanation State Improves… · p. 7
6 Conclusion
Next-encounter ICD forecasting is framed as budgeted hypothesis search: expansion improves coverage but increases Final Selection’s resolution burden. The findings motivate a state-aware controller that expands only when anchored evidence and verified code semantics justify the expected marginal value.
- 8.28 micro-F1 points of joint-pool selection headroom remain on MIMIC-III, while research context changes micro-F1 by +0.12 on MIMIC-III and +1.90 on MIMIC-IV.Independent relation-source coverage is only 11.62% on MIMIC-III and 1.39% on MIMIC-IV.Source: §6 · p. 7
- Candidate expansion broadens coverage but enlarges the pool that Final Selection must resolve, making forecasting a budgeted hypothesis-search problem.Source: §6 · p. 7
- Candidate-conditioned retrieval yields denser physician-rated evidence, while the richer post-selection state aligns more strongly with exact-code alignment and calibration than immediate-horizon relevance.Source: §6 · p. 7
- A state-aware controller should expand only when a patient anchor, directional clinical relation, and verified code semantics justify the expected marginal value; otherwise, it should retain or stop.Source: §6 · p. 7
S1 Guide to the Supplementary Materials
The supplement follows the main paper’s definitions and supplies implementation details, diagnostics, prompt contracts, case traces, and artifact provenance. It organizes these materials across Sections S2–S9, including methods, evaluation, results, interpretation, provenance, prompts, case traces, and error analysis.
Supplement scope
- The supplement preserves the main paper’s method definitions, component names, information boundary, evaluation contracts, and reported claims while adding implementation and diagnostic detail.Unqualified ICD-Deepresearch denotes the full system with the Evidence-Aware Reranker; controlled variants and post-selection explanation audits are explicitly labeled.Source: S1 Guide to the Supplementary Materials · p. 10
Supplement structure
- Sections S2–S5 cover the workflow, evaluation contracts, method-aligned results, retrieval efficiency, mechanism diagnostics, interpretation, limitations, and ethics.S2 specifies the EHR Prior, Direct Forecast, Research Expansion, Final Selection, and explanation writer; S3 covers cohorts, comparisons, metrics, sources, efficiency, uncertainty, and audits.Source: S1 Guide to the Supplementary Materials · p. 10
Artifacts and provenance
- Sections S6–S9 provide exact figure values, executable-call provenance, prompt contracts, case traces, and a cross-case error taxonomy, with quoted instructions transcribed from executable sources.Long JSON contracts are shown verbatim or as complete required-field lists, while dynamic fields and runtime truncation or retry suffixes are recorded.Source: S1 Guide to the Supplementary Materials · p. 10
S2 Full Method Specification
This section specifies ICD-Deepresearch’s target-blind, EHR-prioritized pipeline for forecasting held-out encounter codes, combining bounded evidence and ICD-mapping research with an independent direct forecast. It details deterministic candidate selection, optional patient-specific reranking, prediction-preserving explanation, and retrospective diagnostics.
Task and Information Boundary
- Inference uses only the pre-target record and fixed resources, with target codes and target-encounter notes withheld until evaluation.Call-specific serializations preserve the same information boundary, and target identifiers are removed from search phrases.Source: Task and Information Boundary · p. 10
Research Expansion (internal label DR-CG)
- Research Expansion runs exactly two bounded iterations that generate patient-anchored hypotheses, retrieve code-bound evidence, validate clinical bridges and exact mappings, and retain only accepted candidates.The second iteration uses discovery and validation retrieval, with acceptance requiring all three checks, resolvable evidence, and a confidence threshold of 0.68 under the default contract.Source: Task and Information Boundary · p. 11, Research Expansion (internal label… · pp. 11–13
Final Selection
- Final Selection compares deterministic Union+RRF with an Evidence-Aware Reranker over the same frozen, target-blind proposal artifact combining research candidates and Direct Forecast outputs.The artifact preserves candidate provenance, ranks, evidence status, patient clues, transition hypotheses, and related metadata before selector comparison.Source: Task and Information Boundary · p. 11, Final Selection · p. 13
EHR Prior and Direct Forecast
- SparseEHR produces the ordered 20-code EHR Prior, while Direct Forecast independently ranks target-vocabulary candidates without receiving research outputs or retrieved evidence.Research Expansion is initialized only from the EHR Prior seed set; the direct path joins the research path only at Final Selection.Source: EHR Prior and Direct Forecast · p. 11, Task and Information Boundary · p. 11
Final Selection
- Final Selection uses either reciprocal-rank fusion or an evidence-aware reranker over the same frozen candidate union, with deterministic tie-breaking and a fixed top-K prefix.RRF rewards candidates appearing in both branches but permits high-ranked branch-exclusive candidates; the full reranker prioritizes patient-specific anchors and exact-code support over generic prevalence.Source: Final Selection · p. 14
Post-Selection Explanation Module (RRF-Selected Audit)
- Post-selection explanations produce exactly one structured, code-bound rationale per selected code while preserving identities and order and leaving prediction metrics unchanged.Rationales separate patient anchors, population relations, exact ICD meanings, and next-encounter hypotheses; the audit therefore evaluates RRF-selected outputs rather than the full Evidence-Aware selector.Source: Post-Selection Explanation Module… · p. 14
Retrospective Proposal Diagnostics
- Retrospective diagnostics evaluate accepted Research Expansion additions before top-K ranking, and the registered executor remains fixed at two expansion iterations followed by selection.The break-even condition applies to pooled micro-F1, whereas patient-macro F1 weights additions differently; both diagnostics are computed only after forecasts are finalized.Source: Retrospective Proposal Diagnostics · pp. 14–15
S3 Full Experimental Protocol
The protocol evaluates ICD-Deepresearch prospectively on held-out patient trajectories using distinct scale-cohort and registered subset contracts. It separately audits forecasting, retrieval usefulness, efficiency, and explanation quality under fixed data, candidate, metric, and comparison conditions.
Study Design
- The study compares full ICD-Deepresearch with component, standalone-search, reranker, iteration, source, efficiency, and explanation analyses under registered contracts.Primary forecasting uses completed MIMIC-III and MIMIC-IV exports; component and selector analyses use separately registered diagnostic cohorts.Source: Study Design · p. 15
Cohorts and Outcome Construction
- MIMIC-III targets unique ICD-9-CM diagnoses from each patient’s final admission, while MIMIC-IV uses the corresponding ICD-10-CM construction.Scale cohorts contain 7,496 MIMIC-III and 62,537 MIMIC-IV trajectories; component analyses use fixed 100-patient cohorts with three independent runs.Source: Cohorts and Outcome Construction · p. 15
Primary Comparison and Method-Aligned Ablations
- Supplementary Table S2 evaluates Candidate Generation Only, Direct Forecast Only, Research Path Only, reranker removal, the full system, and standalone search comparators.Panel C separately fixes candidate identities and display order while varying Research Expansion context and bounding selection with a target-conditioned pool oracle.Source: Primary Comparison and Method-Aligned… · p. 16, Table S1 · p. 16
Implementation Record
- Implementation combines SparseEHR candidate generation, concurrent Direct Forecast and bounded Research Expansion, dual retrieval channels, validation, and GPT-5 evidence-aware reranking.Online retrieval uses serialized, rate-limited Google search with parallel URL reading, while offline retrieval searches fixed PMC and arXiv snapshots.Source: Implementation Record · p. 17
Metrics, Inference, and Audit Protocol
- Primary outputs use patient-averaged exact-K metrics, whereas component tables use pooled micro counts; repeated diagnostic rows average three runs over the same 100 patients.Patient-averaged recall is not pooled recall, so the protocol does not combine primary P@K/R@K values to infer micro-F1.Source: Metrics, Inference, and Audit Protocol · p. 18
Retrieved-Source Metadata and Physician Audit
- The source audit samples 50 source occurrences per system–dataset cell, with two physicians independently assigning binary usefulness labels.Physician usefulness is the mean of the two physician-specific rates, and the audit unit is a retrieved source occurrence.Source: Retrieved-Source Metadata and… · p. 19
Online-Retrieval Efficiency Accounting
- Efficiency compares matched 100-patient cohorts using system-specific definitions for queries, retrieval operations, source accesses, tokens, costs, throughput, and pooled Micro-F1@20.Because telemetry paths do not represent calls, retries, nested calls, and usage records identically, total model-call counts are omitted.Source: Online-Retrieval Efficiency Accounting · p. 19
Explanation Evaluation
- The explanation experiment evaluates K = 20 explanations from ICD-Deepresearch and three direct-model systems using blinded GPT-5 judging across five 0–5 dimensions.Judging covers clinical validity, patient grounding, next-encounter relevance, exact ICD alignment, and evidential calibration, with identity and prediction details withheld.Source: Explanation Evaluation · p. 19
S4 Additional Results and Diagnostics
Supplementary diagnostics show that ICD-Deepresearch’s evidence-aware joint selection improves diagnostic-cohort micro-F1, while research expansion increases coverage at substantial cardinality and efficiency trade-offs. Audits find stronger semantic scores but expose limitations in automated evaluation and unsupported patient facts requiring anchor verification.
Component, Selector, and Fixed-Pool Controls (Main RQ1–RQ2)
- 35.44 micro-F1 on MIMIC-IV, with ICD-Deepresearch exceeding Direct Forecast Only by 4.08 points, Research Path Only by 2.54, RRF by 0.59, and standalone GPT-5 web search by 2.07.On MIMIC-III, it exceeds the corresponding baselines by 4.02, 3.69, 4.98, 2.52, and 2.41 micro-F1 points, respectively; these are descriptive five-run means.Source: Component, Selector, and Fixed-Pool… · p. 19
Secondary Diagnostics
- 144 additional held-out matches were recovered on EHRSHOT, but mean set size more than doubled and micro-F1 fell from 25.82% to 16.34%.Initial expansion cleared the micro-F1/2 boundary on both MIMIC diagnostics, whereas Research Round 2 fell below it in all three settings; the support-score rule retained 21 of 28 Round 2 matches.Source: Component, Selector, and Fixed-Pool… · p. 19, Secondary Diagnostics · p. 21
RQ3: Research Expansion Source Quality
- 51%/68% of Research Expansion documents were physician-rated useful on MIMIC-III/IV, versus 32%/41% for Medical Deep Research; Research Expansion also had the highest support proxy on both datasets.In the fixed-pool research-context control, mean micro-F1 changed by +0.12 points on MIMIC-III and +1.90 on MIMIC-IV relative to the no-context GPT-5 arm.Source: RQ3: Research Expansion Source Quality · p. 20, Fixed-Pool Research-Context Control… · p. 20
RQ3: Research Expansion Efficiency
- $0.020–0.039 and $0.021–0.049 per patient were Research Expansion’s visible-token cost ranges on MIMIC-III/IV, versus $0.458 and $0.317 for standalone GPT-5 web search.Research Expansion also used fewer queries, source accesses, and visible tokens while recording 16.8×/21.9× higher batched throughput; micro-F1@20 was lower on both datasets.Source: RQ3: Research Expansion Efficiency · p. 20, Table S4 · p. 20
RQ4: Post-Selection Semantic and Safety Audit
- 0.07 higher ICD alignment and 0.22 higher evidential calibration were achieved by ICD-Deepresearch on 441 MIMIC-III shared hits; MIMIC-IV scores were higher across four reported dimensions.Cohort-level CA-RQ5 changed from 1.29 to 1.33 on MIMIC-III and from 1.36 to 1.57 on MIMIC-IV, but only 11.62% and 1.39% of true-positive explanations had independent clinical-relation sources.Source: RQ4: Post-Selection Semantic and… · p. 20
Secondary Diagnostics
- Unsupported patient facts, including LDL 162 mg/dL and atorvastatin use, appeared in saved traces despite being absent from structured pre-target input, motivating immutable source-span grounding and anchor verification.The semantic audit was automated and gold-aware; GPT-5 saw the held-out gold set only after predictions and explanations were frozen, and judge–generator dependence remained possible.Source: RQ4: Post-Selection Semantic and… · p. 21, Secondary Diagnostics · p. 22
S5 Extended Interpretation, Limitations, and Ethics
The benchmark forecasts next-encounter documentation rather than disease state, while audits clarify candidate control and evidence roles. Interpretation is limited by replay, sampling, validation, reproducibility, and clinical-use constraints.
Functional Interpretation
- The target is the future encounter’s documented code set, shaped by disease state, care setting, follow-up, and coding practice; K = 20 can create multiple false positives.Forecasting metrics measure recovery of documentation, not clinical benefit, which requires severity grading, workflow simulation, and prospective evaluation.Source: Functional Interpretation · p. 22
- Component audits distinguish candidate coverage, fixed-pool research context, target-conditioned ranking, and support-score retention as separate controls for future stopping policies.The audit compares the EHR Prior, Direct Forecast, Research Expansion, and registered final selectors at their respective stages.Source: Functional Interpretation · p. 22
Limitations and Ethics
- The primary analysis uses an Evidence-Aware Reranker on saved target-blind states, whereas explanation auditing uses an archived RRF-selected prefix and does not audit the full output.MIMIC-III and MIMIC-IV share one health-system setting, and EHRSHOT adds a mixed-vocabulary diagnostic.Source: Limitations and Ethics · p. 22
- Source usefulness rates are descriptive: physicians independently label sampled documents, without adjudicated consensus or uncertainty intervals; independent relation coverage remains low.Dictionary citations establish code meaning but not next-encounter likelihood, and passage-level entailment is evaluated through saved candidate and case traces.Source: Limitations and Ethics · p. 22
- Reason quality is judged by a gold-aware GPT-5 evaluator after outputs are frozen, but GPT-5 also generates compared outputs, so this is not independent clinical validation.Dual-clinician annotation and adjudication would strengthen evaluation; clinical deployment additionally requires privacy, security, deterministic verification, severity-aware analysis, and oversight.Source: Limitations and Ethics · p. 22
S6 Prompt, Figure, and Case Reproducibility Archive
The archive documents exact explanation-figure values, uncertainty, source coverage, and the provenance of executable forecasting calls. It distinguishes the RRF-selected explanation output from the full Evidence-Aware ICD-Deepresearch system and maps legacy interfaces to reader-facing functions.
Exact Values for the Explanation Figures
- Table S5 records automated gold-aware GPT-5 judge scores for pooled true-positive Reason5 means and CA-RQ5 at K = 20, with values rounded to two decimals.Figure differences and intervals use unrounded item-level values.Source: Table S5 · p. 24
- CA-RQ5 differences for RRF-selected ICD-Deepresearch versus GPT-5 had 95% confidence intervals of −0.02 to +0.09 on MIMIC-III and +0.15 to +0.27 on MIMIC-IV.The comparison covers complete explanation configurations and does not isolate any single evidence field’s causal contribution.Source: Exact Values for the Explanation Figures · p. 24
- Independent clinical-relation sources covered 58/499 (11.62%) and 8/577 (1.39%) RRF-selected ICD-Deepresearch true-positive explanations on MIMIC-III and MIMIC-IV, respectively.Source: Exact Values for the Explanation Figures · p. 24
Prompt and Artifact Provenance
- The explanation archive uses the RRF-selected AF/Union+RRF output, which is distinct from the full Evidence-Aware ICD-Deepresearch output.FR denotes the full system’s Evidence-Aware selector, whereas AF/Union+RRF is the selector replacement reported as ICD-Deepresearch w/o Evidence-Aware Reranker.Source: Prompt and Artifact Provenance · p. 24, Table S5 · p. 24
- The provenance mapping links legacy FCG, DCG, DR-CG, JCS, AF, and FR interfaces to EHR Prior, Direct Forecast, Research Expansion, Final Selection, and ICD-Deepresearch.Historical Stage 1 and Stage 2 source identifiers correspond to Iterations 1 and 2 of Equation 12.Source: Prompt and Artifact Provenance · p. 24
- Table S6 inventories each executable LLM, deterministic, and API call, including its forecasting-function mapping, line ranges, and archived execution boundary.Source: Table S6 · p. 25
Post-selection explanation
Post-selection explanation uses search and reading tools to gather candidate-bound observations, while deterministic ICD dictionary processing normalizes and validates target-code identity. The execution inventory records provenance across selection, explanation, and audit functions.
Post-selection explanation
- The workflow combines Google or SearchCans, PubMed or fixed PMC/arXiv workers, and a local ICD dictionary.Source: Prompt and Artifact Provenance · p. 25
- Search engines and readers return candidate-bound observations, while deterministic dictionary processing normalizes and validates target-code identity.Source: Prompt and Artifact Provenance · p. 25
- The execution inventory covers LLM calls, search and page-reading APIs, ICD lookup, validation gates, RRF, explanation writers, and audit judges reached by the reported paths.Source: Prompt and Artifact Provenance · p. 26
S7 Prompt Design and Internal Call Contracts
The section specifies ICD-Deepresearch’s prompt contracts, ranking safeguards, candidate schemas, and branch-specific interfaces. Its design enforces patient-anchored exact-code reasoning, bounded outputs, and separation between prediction, selection, and explanation.
Research Expansion contracts
- Research Expansion uses separate formulation, add-judge, query-planning, mapping, judging, and repair calls, with every mapped code still bound to the pre-target patient record.Archived prompts distinguish candidate formulation from add-judging and support retry or completion calls when verdicts are omitted.Source: Table S7 · p. 26, Table S9 · p. 28, Table S10 · p. 29
Ranking safeguards
- Patient-specific evidence dominates prevalence and generic literature, while research-only candidates require both a concrete patient anchor and an exact temporal transition.Supported sources establish general relations but do not prove that a patient will receive a code next; frequency-only and unsupported research-only candidates are heavily constrained.Source: S7 Prompt Design and Internal Call… · p. 27
- The reranker validates exact specificity, preserves plausible acute candidates, avoids inferring diagnoses from generic risks, and jointly calibrates every supplied candidate without adding codes.It prioritizes durable statuses, repeated active chronic diseases, follow-up plans, and recurrently coded conditions, while treating absent web support for acute diagnoses as neutral.Source: S7 Prompt Design and Internal Call… · p. 27
Output contracts
- The shared response contract requires one score for every candidate, confidence in [0, 100], explicit patient-anchor and exact-code-support categories, and a patient-specific EOV threshold.MIMIC-IV returns candidates meeting the threshold, whereas MIMIC-III additionally records include_before_eov; both require exact supplied-code coverage and JSON-only output.Source: S7 Prompt Design and Internal Call… · pp. 28, 34
Branch separation
- The Direct branch is independent of retrieval and candidate pools, RRF combines its rank with research rank, and explanations are generated only after predictions and order are finalized.RRF can promote complementary mid-ranked codes, but its score is not a probability; the evidence-linked explanation writer cannot improve prediction metrics.Source: Table S15 · p. 34, Table S16 · p. 35, Table S19 · p. 38
S8 Case Studies
The case studies illustrate how candidate expansion, joint ranking, explanation alignment, and safety failures arise in ICD-Deepresearch traces. Complementary forecasting paths recover different true positives, while unresolved patient anchors can degrade expansion precision.
S8 Case Studies
- Selected traces preserve row identifiers, functional roles, prediction outcomes, and quoted or reconstructed outputs, while marking omitted candidate lists and distinguishing archival RRF-selected explanations from the full selector.The qualitative trace table compares direct-only and RRF-selected outputs, productive and failed expansion, and pathway-specific true positives.Source: S8 Case Studies · p. 42, Table S24 · p. 42
- In a failed MIMIC-IV expansion, all nine Research Round 2 additions were absent from the held-out encounter, reducing precision .224→.190 and F1 .355→.310 despite natural recall .846.The set expanded from 49→58 candidates, and several additions relied only on dictionary definitions or reversed the immediate-horizon direction.Source: S8 Case Studies · p. 43
- The failure attached medically credible relations to invented anchors, including absent claims of chronic heartburn and multi-year low-back pain.The case also lists six further false additions: J02.9, E66.09, R07.9, E11.9, I10, and G47.00.Source: S8 Case Studies · p. 43
- The required correction is to resolve every candidate anchor to an immutable event identifier or exact pre-target record span, then separately verify source entailment and immediate-horizon direction.This correction directly addresses unresolved patient claims and temporal-direction errors in research expansion.Source: S8 Case Studies · p. 43
D0 ICD-Deepresearch: Direct Forecast Only
- Research ranks F32.9 and Z87.891 5th and 10th, versus 28th and 27th by Direct Forecast Only; it obtains 6 true positives and F1@20=.343.Direct Forecast Only obtains 5 true positives and F1@20=.286; E27.40 is ranked 19th by Direct Forecast Only and 25th by Research Path Only.Source: D0 ICD-Deepresearch: Direct Forecast… · p. 43
D2 ICD-Deepresearch w/o Evidence-Aware Reranker
This section examines ICD-Deepresearch without the evidence-aware reranker, focusing on prediction performance, complementary direct and research candidate paths, and the alignment and usefulness of retrieved evidence and explanations. It also characterizes post-hoc false positives and the quality of research sources selected by the method.
D2 ICD-Deepresearch w/o Evidence-Aware Reranker
- E27.40 entered through direct ranking, F32.9 and Z87.891 through research, and E87.0 rose to rank 20 through agreement between component ranks 25 and 23.The trace illustrates how direct and research paths contributed distinct candidates while agreement promoted a shared candidate.Source: D2 ICD-Deepresearch w/o Evidence-Aware… · p. 43
- P/R/F1=.400/.533/.457 with 8 true positives for ICD-Deepresearch without the evidence-aware reranker.The case trace highlights complementary candidate sources and ranking behavior.Source: D2 ICD-Deepresearch w/o Evidence-Aware… · p. 43
Complete Paired Explanation Output
- The paired MIMIC-IV explanation output used the same pre-target record but differed in prediction and evidence states, and its listed rationales generally marked uncertain future conditions as post-hoc false positives.The output includes conditional explanations for recurrent pregnancy, infection, anemia, and delivery-related codes; among the shown labels, Z37.0 and D57.3 were post-hoc true positives while many others were false positives.Source: Table S25 · p. 44, Complete Paired Explanation Output · pp. 44–45
- 4.80 versus 3.47: ICD-Deepresearch’s RRF-selected explanations received the higher five-axis Reason5 mean, although both systems recovered the same three gold codes at K = 20.The score difference concentrated on code alignment and evidential calibration, while not isolating any single additional input field.Source: Complete Paired Explanation Output · p. 49
B1 Archived research-path metadata
- Research Expansion’s archived glycemic-control source was judged credible, patient-relevant, and forecast-useful, whereas standalone search returned a noncredible code-oriented forum source judged irrelevant and unusable.The useful source linked glycemic control with wound-healing delay; the forum query listed ICD-10-CM definitions without a concrete next-encounter transition.Source: B1 Archived research-path metadata · p. 50
C1 Archived research-path metadata
- Research Expansion’s catheter-infection surveillance source was judged credible, patient-relevant, and directly useful, while standalone search’s AskDocs forum source was judged noncredible, irrelevant, and unusable.The metadata judge identified catheter-infection surveillance as potentially relevant but found the forum source lacked authoritative next-visit guidance.Source: C1 Archived research-path metadata · p. 50, C4 Archived proxy label · p. 50
S9 Cross-Case Error Analysis
Cross-case traces diagnose Research Expansion failures through a three-link contract connecting a resolvable pre-target anchor, a source-entailed directional relation, and an exact immediate-horizon ICD candidate. The taxonomy localizes errors in anchoring, relation direction, code mapping, and candidate retention while treating the RRF observation as a single-case illustration.
Failure taxonomy
- The three-link contract requires a resolvable pre-target anchor, a source-entailed directional relation, and an exact immediate-horizon ICD candidate.Failure mechanisms are paired with corresponding validation requirements.Source: S9 Cross-Case Error Analysis · p. 51, Table S28 · p. 51
Case diagnoses
- Row 52 recovers candidates but reveals mapping errors and definition-only support, whereas row 59 breaks the anchor and horizon links.Row 62 shows the RRF selector retaining true positives available at complementary ranks across the two branches, but this is not a general estimate of RRF effectiveness.Source: S9 Cross-Case Error Analysis · p. 51