PaperlayerView exact revision on arXiv

Source-linked AI summary

Trajectory-Aware Clinical Risk Prediction via Severity-Grounded Knowledge Graphs and Retrieval-Augmented Generation

Kyunghoon Jeon, Youmin Ko, Woohwan Jung, Hyunjoon Kim

arXiv:2607.18270v1cs.AIcs.CL

TL;DR

Sparse, irregular EHR histories and underused clinical notes leave disease severity and progression insufficiently captured. TRACER combines severity-weighted knowledge-graph trajectories, clinical-note evidence, and similar peer cases; it reports gains across MIMIC-III and MIMIC-IV tasks, including a 28.5% sensitivity improvement over KARE for MIMIC-III mortality.

  • Problem

    Clinical risk prediction must infer outcomes from sparse, irregular EHR histories, while existing approaches may miss temporal turning points, diagnosis severity, and nuanced clinical-note evidence.

    Source: §1 · p. 1

  • Method

    TRACER combines literature-grounded diagnosis severity in a medical knowledge graph, retrieves cross-visit trajectories and similar patient cases, and adds salient evidence from clinical notes.

    Source: §1 · p. 2

  • Results

    TRACER reports consistent substantial gains across MIMIC-III and MIMIC-IV mortality and readmission tasks; for MIMIC-III mortality, sensitivity improves by 28.5% versus KARE.

    Source: §1 · p. 2, §5.2 · p. 7

  • Takeaways & Limitations

    The framework provides path-level, clinically grounded explanations that the authors say enhance interpretability and trust.

    Source: §1 · p. 2

    EMERGE and KerPrint results are omitted because their preprocessing codes are not public and their reported results use incorrect preprocessing.

    Source: §5.1 · p. 6

Abstract

from arXiv · show

While Electronic Health Records (EHRs) offer a wealth of clinical data, effectively augmenting a patient's records with heterogeneous external knowledge to predict the patient's clinical risk remains a significant challenge. Existing methods fail to capture disease severity, treatment responses, and nuanced clinical progression, due to data sparsity and the underutilization of unstructured clinical notes. To address these challenges, we propose TRACER (a trajectory-aware and clinically grounded prediction framework) that (1) constructs a medical knowledge graph enriched with severity information from medical literature, (2) retrieves clinically relevant, severity-weighted paths of a patient's progression from the knowledge graph, (3) extracts clinically relevant events from unstructured clinical notes, and (4) augments patient context with similar peer cases. Experiments on the MIMIC-III and MIMIC-IV datasets demonstrate large gains over state-of-the-art baselines, with up to 28.5% increase in Macro F1 score for the mortality prediction task, and 19.7% increase for the readmission prediction task.

1 Introduction

TRACER combines severity-aware medical trajectories, clinical-note evidence, and similar-patient cases to address sparse patient histories and underused narrative data. The paper reports gains across two datasets and tasks, with path-level explanations.

1 Introduction

  • Sparse, irregular visit histories can hinder models’ capture of temporal dynamics and critical disease-progression turning points.In MIMIC-III, 68.6% of patients have one visit and 95.7% have fewer than five; cited baselines show limited-history mortality Macro F1 below 0.7 and sensitivity below 0.5, except KARE at five visits.Source: §1 · pp. 1–2
  • Prior approaches may treat diagnoses equally, missing severity differences, plausible progressions across visits, and clinically critical red flags.Source: §1 · pp. 1–2
  • Clinical notes contain assessments, suspected complications, treatment responses, and patient-reported symptoms that structured codes may omit or oversimplify.Source: §1 · p. 2
  • When personal histories are sparse, relying only on an individual’s record can leave relevant experience from historically similar patient cohorts unused.Source: §1 · p. 2
  • TRACER integrates severity-aware knowledge-graph trajectories, clinical-note evidence, and similar-patient cases in a retrieval-augmented framework for clinical risk prediction.The trajectories connect concepts across visits, while similar cases contribute cohort-level experience.Source: §1 · p. 2
  • The authors report consistent, substantial gains on MIMIC-III and MIMIC-IV for mortality and readmission prediction, with path-level explanations intended to enhance interpretability and trust.Source: §1 · p. 2

2 Related Work

Prior clinical risk prediction work models visit sequences, builds medical knowledge graphs, or retrieves external text. The paper positions TRACER against limits in severity modeling and use of clinical-note information.

2 Related Work

  • Early methods model temporal visit sequences without external knowledge, while later work uses medical knowledge graphs or retrieves external text to complement EHRs.Source: §2 · p. 2
  • Graph approaches include GRAM and KAME for patient embeddings, GRASP for patient-group relationships, and personalized KGs in KerPrint and GraphCare.Source: §2 · p. 2
  • The reviewed KG methods may use only 1-hop neighbors and ignore concept severity, while text-based methods may neglect clinical notes or their richer textual information.The paper says these gaps can leave high-risk events and hidden progressions uncaptured.Source: §2 · pp. 2–3
  • LLM-related approaches include KARE, DearLLM, and CPLLM [38], alongside methods for retrieving documents or extracting concepts from clinical notes.Source: §2 · pp. 2–3

3 Preliminary

The preliminary section specifies the EHR records used to represent patients and the outcomes predicted from those records.

3.1 Electronic Health Records

  • An EHR comprises temporally ordered visits of diagnoses, procedures, and medications, along with visit-level demographics and clinical notes.The notes may include progress notes, discharge summaries, or physician notes.Source: §3.1 · p. 3

3.2 Problem Formulation

  • The task predicts mortality status or readmission within 15 days after discharge from the latest historical visit, using EHR data through that visit.Source: §3.2 · p. 3

4 Methodology

TRACER forms patient profiles from severity-grounded knowledge-graph trajectories, clinical-note evidence, and demographics, then adds similar-patient context for LLM mortality or readmission prediction.

4.1 Overview of the TRACER Framework

  • TRACER proceeds through patient medical-profile retrieval, similar-patient retrieval, and LLM prediction of mortality or readmission using target and peer context.Source: §4.1 · p. 3

4.2 Severity-weighted Medical KG Construction

  • SMKG adds diagnosis-level 1–20 scores generated from retrieved PubMed and Wikipedia text while retaining the unified KG’s node and edge sets.The scores are retrieval priors, not standalone prognostic labels.Source: §4.2 · pp. 3–4

4.3 Patient Medical Profile Retrieval

  • The patient medical profile augments sparse EHRs with disease progression, treatment responses, health-status descriptions, and demographics, organized into risk and protective profiles.Source: §4.3 · p. 4

4.3.1 Trajectory Retrieval.

  • Candidate trajectories concatenate shortest SMKG paths of up to three hops between concepts in consecutive visits; risk and protective queries retrieve severity- and relevance-weighted paths.Source: §4.3.1 · pp. 4–5

4.3.2 Trajectory Refinement.

  • MMR reranking balances query relevance and novelty, while NLI entailment-based stopping leaves Key Supporting Trajectories as LLM context without additional model training.Source: §4.3.2 · p. 5

4.3.3 Clinical Note Retrieval.

  • To avoid irrelevant notes, TRACER splits historical clinical records into passages and retrieves top-n cosine-matched evidence separately for risk and protective queries.Source: §4.3.3 · p. 5

4.4 Retrieval-augmented Clinical Risk Prediction

  • The prompt combines target profiles and top-m peers’ KSTs and demographics, repacks relevant evidence near the task, and asks the LLM for binary predictions with interpretable evidence chains.Source: §4.4 · p. 5

5 Experiments

TRACER outperforms the compared baselines on MIMIC-III and MIMIC-IV, while ablations show that severity grounding and multiple evidence sources contribute complementary gains.

5.1 Experimental Setup

  • Experiments use MIMIC-III and MIMIC-IV, four prediction metrics, and state-of-the-art clinical models plus ETHOS and a prompt-only zero-shot LLM baseline.GPT-4o-mini is the common backbone for LLM-based models; the reported results exclude EMERGE and KerPrint because their preprocessing codes were unavailable and reported results used incorrect preprocessing.Source: §5.1 · p. 6

5.2 Main Results

  • On MIMIC-III mortality prediction, TRACER improves sensitivity by 28.5% and Macro F1 by 19.7% over KARE, and it leads across all metrics on both tasks.On MIMIC-IV, it leads on all mortality metrics except specificity, where it trails the best competitor by 0.8%; it also exceeds competitors on readmission specificity by at least 15.5%.Source: §5.2 · p. 7

5.3 Ablation Study

  • Removing severity scores lowers mortality sensitivity from 0.7128 to 0.4950, while removing all trajectories lowers mortality Macro F1 from 0.8197 to 0.6302.Removing clinical notes lowers readmission Macro F1 from 0.8709 to 0.7981 and sensitivity from 0.7809 to 0.6762; combined ablations indicate the system uses complementary components.Source: §5.3 · p. 7
  • For mortality prediction, removing risk trajectories causes the largest sensitivity drop, while readmission prediction is more sensitive to clinical-note and similar-patient evidence.The authors associate note evidence with short-term deterioration and discharge-related cues not fully represented by structured codes.Source: §5.3 · p. 7

6 Analysis

The analyses examine severity-score consistency, graph severity, inference efficiency, retrieval settings, backbone robustness, and trajectory path length. Results identify settings and evidence choices associated with stronger or more efficient prediction.

6.1 Consistency of Severity Score Generation

  • Severity S.D. falls from 2.02 with GPT-4o-mini to 1.59 with GPT-5-nano and 1.16 with GPT-o3; GPT-o3 also improves mortality Macro F1 by over 3% versus GPT-4o-mini.The authors report that severity variability remains low even with GPT-4o-mini.Source: §6.1 · p. 7, Table 4 · p. 8

6.2 Effectiveness of SMKG

  • On MedQA, replacing MedGraphRAG's UMLS graph with SMKG increases accuracy by 3.94 percentage points, with improvements also reported on MedMCQA and PubMedQA.The authors attribute the gains to encoding disease severity to distinguish relative clinical risk and importance.Source: §6.2 · pp. 7–8, Table 5 · p. 8

6.3 Inference efficiency

  • TRACER is approximately 33% faster than KARE and achieves 29.9% higher Macro F1 and 84.1% higher sensitivity; trajectory retrieval takes 60 seconds versus 92 seconds for KARE subgraph retrieval.Inference time is measured as TRACER's total inference-stage duration and KARE's summed retrieval, reasoning-chain construction, and LLM prediction durations.Source: §6.3 · p. 8, Table 6 · p. 8

6.4 Hyperparameter Sensitivity

  • For MIMIC-III mortality prediction, similar-patient and clinical-note counts perform best at 5, MMR weight performs best at 0.7, and trajectory proportion performs best at 5.Counts above 5 and MMR weights above 0.7 reduce performance; counts below 5 also slightly reduce performance.Source: §6.4 · p. 8, Figure 5 · p. 8

6.5 Similar Patient Retrieval Strategy

  • Recent-visit Jaccard performs best across mortality and readmission tasks and retrieves patients approximately 19% faster per patient than the StageNet baseline.The strategy matches exact medical-concept overlap in the most recent two historical visits.Source: §6.5 · p. 9, Table 7 · p. 9

6.6 Robustness to LLM Backbones

  • TRACER remains effective across different LLM backbones; GPT-4o-mini performs best overall, LLaMA2-8B has the highest specificity, and Qwen3.5-2B retains reasonable sensitivity and accuracy.Similar-patient evidence is auxiliary; predictions remain primarily grounded in the target patient's trajectories, notes, and demographics.Source: §6.6 · p. 9, Table 8 · p. 9

6.7 Hop-limit Analyses

  • For mortality prediction, increasing the path limit from 1 to 3 raises Macro F1 from 0.7960 to 0.8197 and sensitivity from 0.6732 to 0.7128; sensitivity declines slightly at 4 hops.The authors identify three hops as the practical balance between path coverage and noise mitigation.Source: §6.7 · p. 9, Table 9 · p. 9

7 Conclusion

TRACER combines severity-aware trajectories, similar peer cases, and clinical-note evidence for mortality and readmission prediction. The implementation uses GPT-4o-mini prompting and prioritizes risk trajectories when context exceeds the prompt limit.

7 Conclusion

  • TRACER integrates severity-aware trajectories, similar patients selected by recent visit patterns, and unstructured clinical-note evidence for prediction and path-level explanations.Experiments on MIMIC-III and MIMIC-IV report improvements over state-of-the-art baselines in sensitivity, Macro F1, specificity, and accuracy.Source: §7 · p. 9

A Implementation Details

  • The implementation uses zero-shot GPT-4o-mini prompting with temperature 0.7 and top-p 1.0, and caps prompts at 2,048 tokens.When context exceeds the cap, risk trajectories take priority and protective trajectories are truncated first.Source: A Implementation Details · p. 12
  • Inference processes one patient at a time because of LLM API latency, averaging approximately 4–6 seconds per prediction; hyperparameters are tuned manually with sensitivity prioritized.Source: A Implementation Details · p. 12

A.1 Errors in Implementation of Label Definition:

The appendix corrects label construction, describes baseline families, and specifies dataset access, preprocessing, and sampling. These choices define the comparison protocol and the experimental cohorts.

A.1 Errors in Implementation of Label Definition:

  • Corrected preprocessing sorts mortality visits by discharge time and calculates readmission intervals from the prior discharge to the next admission, then recomputes labels and reruns baselines under the same protocol.In MIMIC-III, positive-label proportions changed from 5.42% to 10.83% for mortality and from 54.82% to 21.60% for readmission.Source: §A.1 · p. 12

B.1 Temporal Sequence Models

  • Temporal baselines model visit sequences, ranging from recurrent and convolutional models to attention and Transformer architectures, without explicitly encoding all medical relationships or severity.The comparison includes GRU, Transformer, RETAIN, Deepr, and TCN.Source: §B.1 · p. 12, B Baselines · p. 12

B.2 Knowledge Graph-based Models

  • Knowledge-graph baselines include ontology-based representations, similar-patient graphs, patient-specific graphs, and retrieval-augmented EHR models; the described approaches have limits in severity modeling or graph traversal.Examples include GRASP, StageNet, GraphCare, and RAM-EHR.Source: §B.2 · p. 12

B.3 Text and LLM-based Models

  • Text and LLM-based baselines include medical-text retrieval, LLM-generated concept correlations, and KARE reasoning chains, with KARE described as lacking explicit severity modeling.The comparison includes MedRetriever, KARE, and DearLLM.Source: §B.3 · pp. 12–13

C Dataset

  • The study uses publicly available MIMIC-III v1.4 and MIMIC-IV v2.0 EHR datasets hosted on PhysioNet.Dataset statistics are summarized in Table 10.Source: C Dataset · p. 13

C.1 Patient Privacy and Ethical Considerations

  • The data are de-identified, accessed by trained and approved researchers, and reported only in aggregate; patients are split across sets to prevent leakage.No additional linkage or re-identification processes are introduced.Source: §C.1 · p. 13

C.2 Dataset Preprocessing

  • The main cohort retains patients with at least two visits, uses the last visit as the target, maps diagnoses and procedures with CCS and medications with ATC-3, and excludes patients with more than 10 visits.Source: §C.2 · p. 13

C.3 Sampling Details

  • MIMIC-IV cohorts are sampled for mortality and readmission, then subsets are split 8:1:1 with patient-level separation and balanced outcome representation.The mortality sample includes 2,152 positive and 10,000 negative patients; the readmission sample includes 5,000 patients in each outcome group.Source: §C.3 · p. 13

D Evaluation Metrics

The evaluation prioritizes sensitivity and Macro F1 for imbalanced clinical prediction while reporting specificity and accuracy as complementary measures. TRACER outperforms all baselines across the reported metrics on both datasets and both prediction tasks.

D Evaluation Metrics

  • Sensitivity measures the proportion of actual positive cases correctly identified, and the evaluation prioritizes it because missed high-risk patients can have serious consequences under class imbalance.The paper gives missed ICU triage, delayed interventions, and preventable deterioration as possible consequences of false negatives.Source: D Evaluation Metrics · p. 13
  • Macro F1 is the unweighted average of positive- and negative-class F1 scores, so majority-class performance does not overshadow the minority class.The paper frames this balance as relevant to imbalanced classification and to avoiding false reassurance or over-alarm.Source: D Evaluation Metrics · p. 14
  • Specificity measures correctly identified true negatives and is reported alongside sensitivity to assess the trade-off between missed risks and false alarms.Accuracy is reported only as a reference because a classifier predicting every patient as negative can score highly on imbalanced datasets while missing all positive cases.Source: D Evaluation Metrics · p. 15

E Main Results

  • TRACER outperforms all baselines across Macro F1, sensitivity, specificity, and accuracy on MIMIC-III and MIMIC-IV mortality and 15-day readmission prediction.Tables 11 and 12 present the quantitative comparisons for MIMIC-III and MIMIC-IV, respectively.Source: Table 11 · p. 14, Table 12 · p. 14, E Main Results · p. 15

E.1 Mortality Prediction

For mortality prediction, TRACER improves sensitivity over KARE and leads in Macro F1; for readmission, it also outperforms the strongest baseline across metrics. The paper attributes these results to severity-aware trajectories, clinical notes, and similar-patient context.

E.1 Mortality Prediction

  • Sensitivity improves by up to 84.1% on MIMIC-III and 29.5% on MIMIC-IV over the strongest baseline, KARE, for mortality prediction.The paper relates higher sensitivity to fewer missed high-risk patients.Source: §E.1 · p. 15
  • TRACER leads all competitors in mortality Macro F1, balancing performance across death and survival classes.The paper describes this balance as important for preventing both over- and under-treatment.Source: §E.1 · p. 15
  • Severity-aware trajectory retrieval, path refinement, repacking, and similar-patient augmentation are identified as core innovations behind the mortality gains.The paper contrasts severity-aware trajectories with flat paths or 1-hop neighborhoods and says refinement and repacking focus LLM attention on informative paths and notes.Source: §E.1 · p. 15

E.2 Readmission Prediction

  • For readmission, TRACER exceeds the strongest baseline on all metrics and improves MIMIC-IV specificity by at least 15.5%.The paper connects higher specificity with avoiding excessive, costly, and often unnecessary readmissions.Source: §E.2 · p. 15
  • The paper identifies salient note retrieval and protective trajectories as contributors to readmission prediction, and describes the combined design as accurate, interpretable, and clinically relevant across both tasks.Notes capture discharge planning, unresolved symptoms, and social context; protective trajectories trace improvements and interventions that reversed deterioration.Source: §E.2 · p. 15

F Robustness to Data Sparsity across Visit Sequence Lengths

As visit sequences lengthen, TRACER’s Macro F1 declines more gradually than most baselines, while its sensitivity rises smoothly. The authors attribute this robustness to retrieving similar patients using recent visits.

F Robustness to Data Sparsity across Visit Sequence Lengths

  • TRACER’s Macro F1 decreases more gradually than most baselines as visit count increases, and its sensitivity rises smoothly while baseline sensitivity changes more abruptly or unstably.Figure 6 plots Macro F1 and sensitivity by visit count, with bars showing the percentage of patients at each sequence length.Source: F Robustness to Data Sparsity across… · pp. 15–16, Figure 6 · p. 16
  • The paper attributes TRACER’s more stable predictions for longer visit histories to retrieving similar patients based on the target patient’s recent visits.This retrieval focuses on relevant, up-to-date clinical status rather than simply accumulating historical records.Source: F Robustness to Data Sparsity across… · p. 16

G Impact of Repacking and Refinment

Ablations and sensitivity analyses examine trajectory refinement and repacking, severity-score granularity, baseline comparisons, knowledge snapshots, and six hyperparameters. The results favor refined trajectories, fine-grained severity scores, and balanced settings for retrieval and prompting.

G Impact of Repacking and Refinment

  • Removing trajectory refinement lowers mortality sensitivity by 7.2% (0.6831 →0.6336); omitting trajectory repacking lowers it by 5.8% (0.6831 →0.6435), versus 4.3% (0.6831 →0.6534) without similar-patient repacking.The authors conclude trajectory repacking is more effective than similar-patient repacking and that trajectory selection and ordering matter for LLM performance.Source: G Impact of Repacking and Refinment · p. 16

H Impact of Severity Score Granularity.

  • The 1–100 severity scheme leads in Macro F1, specificity, and accuracy, while 1–20 leads in sensitivity; the 1–5 scheme underperforms across metrics.The authors say coarse scoring may obscure clinically important differences and that fine-grained scores better differentiate high-risk patients.Source: H Impact of Severity Score Granularity · p. 16, Table 14 · p. 16

I.1 Comparison across Cutting-edge Clinical Risk Prediction Baselines

  • On MIMIC-IV mortality prediction, TRACER outperforms ETHOS and a prompt-only zero-shot LLM baseline across all reported metrics.The comparison reports Macro F1, sensitivity, specificity, and accuracy.Source: §I.1 · pp. 16–17, Table 15 · p. 16

I.2 Robustness across Knowledge Snapshots

  • Older PubMed snapshots produce only bounded degradation relative to the full setting, while the API snapshot retains competitive performance on MIMIC-III mortality prediction.The authors interpret this as robustness to temporal variation in retrieved biomedical knowledge and model versions.Source: Table 16 · p. 16, §I.2 · p. 17

J Hyperparameter Sensitivity

  • For risk-trajectory relevance, α=0.7 yields Macro F1 0.8284 and sensitivity 0.6831; for protective trajectories, β=0.3 yields Macro F1 0.8282 and sensitivity 0.6831.The paper associates lower α with underweighting severity and higher α with reduced semantic alignment; overemphasizing protective signals may dilute high-risk contrastive reasoning.Source: §J.1 · p. 17, §J.2 · p. 17
  • Using k=5% of trajectories and n=5 clinical-note passages yields Macro F1 0.8284 and sensitivity 0.6831 in each reported setting.The paper says lower k reduces coverage and higher k adds redundancy; too few notes underrepresent context, while too many dilute key signals, with n=7 lowering F1 to 0.8076.Source: §J.3 · p. 17, §J.4 · p. 17, Figure 7 · p. 17
  • Retrieving m=5 similar patients yields Macro F1 0.8284 and sensitivity 0.6831, balancing analogy with prompt length.The paper associates m=1 with underfitting and m=7 with distraction.Source: §J.5 · p. 17, Figure 7 · p. 17
  • An MMR weight λ=0.7 yields Macro F1 0.8284 and sensitivity 0.6831; λ=0.5 increases redundancy, while λ=0.9 loses diversity.MMR governs the relevance-diversity trade-off during trajectory refinement.Source: Figure 7 · p. 17, §J.6 · p. 18

K Correlation Analysis between Severity Scores and Mortality Labels.

The diagnosis-level severity score has a stronger reported association with mortality than CCI. The case comparison and prompts describe how TRACER uses severity-aware trajectories, clinical notes, similar patients, and inference guidance.

K Correlation Analysis between Severity Scores and Mortality Labels.

  • 0.38 correlation with mortality labels for the patient-level severity score, versus 0.29 for CCI; both correlations are statistically significant (p < 0.05).The severity score averages diagnosis-level scores across visits; the mortality label has 10.83% positive cases.Source: K Correlation Analysis between… · p. 18

Comparison of Case Study Inputs Between TRACER and Other Models

  • TRACER’s case representation includes severity-weighted risk and protective trajectories, note evidence, and demographically aligned similar-patient summaries, unlike the compared models’ static structured inputs.The comparison describes prior models as using community-level KG summaries or structured EHR features, and exemplar patients without trajectory modeling or demographic alignment.Source: Comparison of Case Study Inputs… · p. 21

M.1 Prompt for Severity Score Generation

  • The severity-generation prompt uses medical literature and guidelines to score diagnosis concepts for integration into the severity-weighted knowledge graph.The scores distinguish high-risk diagnoses from less critical ones during downstream reasoning.Source: §M.1 · p. 21, Figure 10 · p. 21

M.2 Prompts for Generating KSTs

  • The KST prompt refines supporting trajectories by balancing relevance, severity, and diversity using natural language inference signals.Source: §M.2 · p. 21, Figure 11 · p. 21

M.3 Prompt for Inference

  • Inference prompts combine patient trajectories, clinical notes, demographics, and similar cases to encourage conservative, clinically grounded decisions and avoid overconfident risk predictions.They also encourage step-by-step reasoning.Source: §M.3 · p. 21, Figure 12 · p. 22, Figure 13 · p. 23
Loading 2607.18270v1…