EHR-R1: A Reasoning-Enhanced Foundational Language Model for Electronic Health Record Analysis

1 INTRODUCTION

• EHR analysis supports clinical needs from decision support to prognosis, but current LLMs remain limited on EHR tasks and struggle with selective filtering and integrating information across records.The paper identifies narrow task coverage and weak EHR-oriented reasoning as the central gaps.Source: §1 · pp. 1–2
• EHR-Ins contains 300K reasoning cases and 3.5 million non-reasoning cases spanning 42 EHR tasks, with reasoning data synthesized using a thinking graph and GPT-4o.The pipeline identifies related entities through co-occurrence analysis, links them with UMLS knowledge, and prompts GPT-4o to produce step-by-step reasoning.Source: §1 · p. 3
• EHR-R1 is a family of models up to 72B parameters, trained through domain adaptation, reasoning enhancement, and reinforcement learning on curated cases.The stages use extensive non-reasoning data, high-quality reasoning cases, and a smaller curated set for GRPO.Source: §1 · p. 3
• Over 30 points: EHR-R1-72B improves on GPT-4o across all 42 EHR-Bench tasks, and its zero-shot AUROC on EHRSHOT is 10% higher than baseline models.EHR-Bench is derived from MIMIC-IV and covers decision-making and risk-prediction; the benchmarks span Stanford Medicine and Beth Israel Deaconess Medical Center.Source: §1 · p. 3

2.1 Training Data Overview

• Across eight decision-making tasks, experts rated most synthetic reasoning data above 4 out of 5, and graph-enhanced chains supplied more adequate EHR evidence than naïve distillation.The manual evaluation covered 100 reasoning chains; score 5 meant reasoning fully supported the predicted labels, while 1 meant it was irrelevant.Source: §2.1 · pp. 3–4

2.2.1 Benchmarks

• EHR-Bench covers 42 MIMIC-IV tasks across 12 subtypes, spanning generative decision-making and binary risk prediction.Its task groups include 24 decision-making tasks and 18 risk-prediction tasks.Source: §2.2.1 · p. 5

2.2.2 Metrics

• Decision-making is evaluated with multi-label F1 using exact entity matches, while binary risk-prediction tasks use AUROC.General-purpose models receive a candidate pool combining 100 randomly sampled labels with the true labels; AUROC estimates use yes/no token scores normalized with softmax.Source: §2.2.2 · p. 6

2.2.3 Baselines

• The baseline set includes general-purpose open models, reasoning models, and specialized medical models, alongside commercial GPT-4o.Examples include Llama3.3-70B, Qwen2.5-72B, DeepSeek R1, and OpenBioLLM-70B.Source: §2.2.3 · pp. 6–7
• EHR-R1 achieves a clear performance advantage on nearly all 24 EHR-Bench decision-making tasks, measured by F1 score against nine baseline LLMs.Figure 4 uses cross-hatched bars for reasoning-enhanced models.Source: Figure 4 · p. 7

2.3.1 Decision-making Tasks

• EHR-R1-72B averages 0.6744 F1 on decision-making tasks, versus 0.3535 for next-best Qwen2.5-72B and 0.3155 for GPT-4o.The paper reports that its advantage is consistent across all decision-making tasks.Source: §2.3.1 · p. 8
• Commercial and open-source models perform inconsistently across decision-making tasks; GPT-4o’s average F1 of 0.3155 trails Qwen2.5-72B’s 0.3535.GPT-4o is often surpassed by Medgemma-27B and Qwen2.5-72B in inpatient diagnostic settings.Source: §2.3.1 · p. 8
• General reasoning models show no clear decision-making advantage: Qwen2.5-72B scores 0.3535 F1 versus 0.3418 for Qwen3-235B.The authors state that effective EHR reasoning requires integrating medical knowledge with case-structured analysis.Source: §2.3.1 · p. 8

2.3.2 Risk-prediction Tasks

• Risk-prediction performance varies across tasks, and Qwen2.5-72B trails Qwen3-235B’s 0.8245 average AUROC despite strength in decision-making.The authors describe the variability as evidence that baseline models struggle to perform uniformly across EHR analysis.Source: §2.3.2 · p. 8
• All models’ AUROC decreases at longer prediction horizons; EHR-R1-72B falls from 0.9898 at 1-day to 0.9610 at 14-day ICU mortality prediction.The paper identifies long-term patient-outcome prediction as a continuing challenge.Source: §2.3.2 · p. 8
• EHR-R1-72B averages 0.9523 AUROC across 18 risk-prediction tasks, versus 0.8245 for Qwen3-235B.Figure 5 shows per-task ROC curves and a Total subplot aggregating performance across all 18 tasks.Source: §2.3.2 · p. 8, Figure 5 · p. 9

2.4 Generalization Evaluation on MIMIC-IV-CDM

• EHR-R1-72B scores 0.8913 on main-disease prediction, ahead of DeepSeek-R1 at 0.8841 and GPT-OSS-120B at 0.8793.The paper characterizes main-disease prediction as a less noisy, more idealized scenario.Source: §2.4 · p. 9
• Other models trade off diagnostic granularity: Med-Gemma-27B scores 0.2860 on ICD code and 0.7939 on main disease, while DeepSeek-R1 and GPT-OSS-120B have lower ICD scores.DeepSeek-R1 and GPT-OSS-120B score 0.2597 and 0.2422, respectively, on ICD code prediction.Source: §2.4 · p. 9
• EHR-R1-72B achieves the highest reported scores on both zero-shot tasks: 0.8913 for main disease and 0.3501 for ICD code.The two diagnosis granularities use the same samples.Source: §2.4 · pp. 9–10, Figure 6 · p. 10

2.5 Generalization Evaluation on EHRSHOT

• The zero-shot protocol evaluates directly on an unseen dataset, while few-shot evaluation measures adaptation using a small number of labeled examples.Few-shot evaluation uses a 1.7B model and at most 128 training samples per task.Source: §2.5 · p. 11, §2.5.2 · p. 11

2.5.1 Zero-shot Setting

• EHR-R1-72B attains the highest AUROC in all three EHRSHOT categories and the overall average, including against similarly sized general and specialized medical models.The categories are operational outcomes, lab test forecasting, and new-diagnosis assignment; EHRSHOT differs from EHR-Bench in data format and task types.Source: §2.5.1 · p. 11

2.5.2 Few-shot Setting

• At k = 128, EHR-R1-1.7B averages 0.7465 AUROC versus 0.5998 for Qwen3-1.7B in few-shot fine-tuning.The evaluation uses at most 128 labeled examples; EHR-R1’s advantage is modest below 16 examples and widens at k ≥16, primarily for operational outcomes and lab forecasting.Source: §2.5.2 · p. 11
• For new-hypertension prediction, the EHR-R1-1.7B versus Qwen3-1.7B AUROC gap peaks at k = 16 (0.6593 vs. 0.5108) and narrows as k increases.The paper suggests general-model pretraining on diagnosis tasks may explain the narrowing.Source: §2.5.2 · p. 11

2.6 Ablation Study

• Reasoning-data training raises direct-answer F1 from 0.5060 to 0.5300 for EHR-R1-1.7B and from 0.6039 to 0.6281 for EHR-R1-72B.The training curriculum also includes domain adaptation and reinforcement learning with GRPO.Source: §2.6 · p. 12
• Reasoning at inference raises EHR-R1 F1 at 1.7B, 8B, and 72B, while general reasoning prompts lower F1 for Qwen3-1.7B and Qwen3-8B.EHR-R1-1.7B rises from 0.5060 to 0.5438 (±0.0035); Qwen3-1.7B falls from 0.1624 to 0.1456.Source: §2.6 · p. 12
• Across model scales, average F1 rises from 0.5438 for Qwen3-1.7B-based EHR-R1 to 0.5894 at 8B and 0.6481 at 72B.The ablation compares five configurations across seven decision-making task categories and their average performance.Source: §2.6 · p. 12, Figure 7 · p. 12

Overview of Our Approach

• The thinking-graph pipeline extracts EHR entities, links temporal and causal relations, and synthesizes stepwise reasoning for three-stage EHR-R1 training.The stages are domain adaptation, reasoning enhancement, and reinforcement learning with Group Relative Policy Optimization.Source: Overview of Our Approach · p. 13

Main Contribution

• EHR-Ins pairs 300K reasoning cases with 3.5M non-reasoning cases across 42 EHR tasks, with reasoning traces generated by the thinking-graph pipeline.Source: Main Contribution · p. 13
• EHR-Bench uses MIMIC-IV to assess LLM performance across 42 diverse EHR tasks and address coverage gaps in existing evaluations.Source: Main Contribution · p. 13

Key Findings

• EHR-R1 shows strong performance across 42 EHR-Bench tasks and zero-shot transfer to unseen EHRSHOT data and tasks.The discussion describes this transfer as evidence that the model captures transferable structure rather than memorizing training data.Source: Key Findings · p. 13, Main Contribution · p. 13
• The authors report that reasoning chains improve EHR analysis by supporting denoising, temporal-relation analysis, and longitudinal clinical narratives.They attribute the gains to an auto-synthesis pipeline that uses relationships among medical entities.Source: Key Findings · p. 13
• Improvements from the training framework and reasoning data are consistent across model sizes from 1.7B to 72B parameters.Source: Key Findings · pp. 13–14

Limitations and Future Work

• Reasoning-data synthesis was limited to decision-making tasks, and graph construction excluded cases lacking related entities or retrieved UMLS relations.These boundaries constrain reasoning coverage and the ability to form longitudinal narratives for every case.Source: Limitations and Future Work · p. 14
• Specializing the model for EHR analysis may attenuate general-purpose capabilities and risks overfitting to clinical settings.Source: Limitations and Future Work · p. 14

• The Methods section begins with problem formulation.Source: §4 · p. 14
• It then presents the reasoning-data curation pipeline and model training.Source: §4 · p. 14
• The section concludes with evaluation.Source: §4 · p. 14

• An EHR is represented as a chronologically ordered sequence of longitudinal records, including laboratory results, medication administrations, and transfers.Source: §4.1 · p. 14
• Each record contains a clinical event category, its medical entities, and its timestamp.Source: §4.1 · p. 14
• The model is trained to generate a free-text answer from the task instruction and the EHR history observed up to prediction time.The instruction specifies the analysis task, and the answer is the ground truth.Source: §4.1 · p. 14

4.2 Data Curation

• The pipeline filters and reorganizes raw MIMIC-IV cases into instruction-tuning format, then augments training data with reasoning while reserving the test set for EHR-Bench.The overview begins with MIMIC-IV and depicts reasoning synthesis as part of creating EHR-Ins.Source: §4.2 · pp. 14–15, Figure 8 · p. 15

4.2.1 MIMIC-IV Processing

• Decision-making samples predict entities in the next event, while risk-prediction samples ask whether a specified critical outcome occurs within a time window.Training samples are represented as S = (I, R≤t, A).Source: §4.2.1 · p. 16
• Label-wise weighted sampling uses reciprocal label frequencies to rebalance training targets and broaden target-label coverage in EHR-Bench; patient-level splitting keeps each patient out of either train or test, not both.Samples with fewer than 10 or more than 100 historical events are filtered, and the observable history is limited to 24 hours.Source: §4.2.1 · p. 17
• 3.5M non-reasoning samples and 300K reasoning samples form EHR-Ins, while 21K test samples form EHR-Bench.The train and test samples are selected with label-wise weighting.Source: §4.2.1 · p. 17

4.2.2 Thinking-Graph-Driven Reasoning Synthesis

• The synthesis pipeline identifies context entities with high Lift associations to labels, maps entities to UMLS, and searches the graph bidirectionally to find medical relations.Pairs are retained when both entity and label counts exceed 5 and Lift exceeds 5; unmappable entities are discarded.Source: §4.2.2 · pp. 18–19
• GPT-4o uses historical events and retrieved relations to produce extraction, reasoning, and final-result stages; samples are retained only when at least 70% of their labels are inferable.The authors note that chains for labels without a valid reasoning path may introduce hallucinations.Source: §4.2.2 · pp. 19–20

• The three-stage training process comprises large-scale domain adaptation, reasoning enhancement, and reinforcement learning for EHR analysis.The stages target task knowledge, inference from historical events, and reasoning capabilities, respectively.Source: §4.3 · p. 20
• Domain adaptation instruction-tunes on non-reasoning question-answer pairs and supervises both the EHR input X and target labels A.The authors report better downstream performance with supervision on both than on A alone, and attribute the improvement to patterns in X.Source: §4.3 · p. 20
• Reasoning enhancement fine-tunes on synthesized samples whose outputs include a tagged reasoning chain followed by the final result.The input remains the patient’s observable events and task instruction.Source: §4.3 · p. 20
• GRPO rewards reasoning trajectories for format and prediction accuracy, with ACC for risk prediction and F1 for decision-making.The format reward checks the thought tags and prescribed stages; updates are restricted to 500 examples per task to limit drift and hallucination.Source: §4.3 · pp. 20–21

4.4.1 Benchmarks

• EHR-Bench evaluates decision-making and risk-prediction, using candidate selection for baseline models on multi-answer tasks while EHR-R1 generates answers directly.Candidate sets contain 100 labels where available, or all labels when the task has fewer than 100.Source: §4.4.1 · p. 22
• MIMIC-IV-CDM covers four diseases and uses each event’s most recent information rather than a chronological history.Its context is described as less noisy and the task simpler than EHR-Bench and EHRSHOT.Source: §4.4.1 · p. 22
• EHRSHOT includes 14 risk-prediction tasks from 7,000 patients, with codes and measurements reformatted or enriched for LLM evaluation.Some CPT4 and ICD10PCS codes required added mappings; frequent measurement and observation items were supplemented with units, normal ranges, and anomaly indicators.Source: §4.4.1 · p. 22

4.4.2 Metrics

• Decision-making uses precision, recall, and F1, while risk-prediction uses AUROC.AUROC measures discrimination between positive and negative outcomes across decision thresholds.Source: §4.4.2 · pp. 22–23

4.4.3 Baselines

• The baseline set includes Qwen, MedGemma, Llama 3, and OpenBioLLM model families.The passages characterize Qwen and Llama 3 as general-purpose, and MedGemma and OpenBioLLM as medically oriented.Source: §4.4.3 · p. 23
• Additional baselines include Baichuan M2, GPT-4o, DeepSeek R1, and GPT-OSS.Source: §4.4.3 · pp. 23–24

• EHR-Ins and EHR-Bench derive from public MIMIC-IV data, and both are under PhysioNet review.Source: §5 · p. 24

8 Author Contributions

• Y.L., C.W., and J.L. contributed equally, while Y.W. and W.X. are corresponding authors.Source: §8 · p. 28
• All listed authors contributed to the work’s conception or design, critically reviewed it, approved the version for publication, and agreed to be accountable for its integrity.Source: §8 · p. 28
• Y.L., C.W., and J.L. additionally acquired, analyzed, or interpreted data and drafted the work.Source: §8 · p. 28

9.1 Details of Experiments

• Supplementary Tables 1–2 report decision-making F1 scores as mean ± std and describe ablations across four Qwen base-model sizes, with average performance across tasks in the final column.The supplied captions identify the evaluations but do not provide the underlying scores.Source: Table 1 · p. 29, Table 2 · p. 29

9.2 Details of EHR-Bench

• EHR-Bench covers 42 tasks grouped into 2 task types and 12 subtypes, with target events, MIMIC-IV item names for decision tasks, manual-mapping columns, and candidate-set sizes documented.Risk-prediction tasks have no item name because their target labels are not specific entities.Source: Table 3 · p. 30

9.3 Prompt Collection

• The prompt collection asks clinicians to score whether extracted EHR information supports a diagnostic conclusion on a five-level scale, without validating the model’s reasoning result for correctness.Its examples distinguish fully supported conclusions from partial, weak, or irrelevant support; the collection also includes a GPT-4o synthesis prompt using patient context, retrieved medical knowledge, ground truth, and task instructions to produce Extraction, Reasoning, and Final Results stages.Source: §9.3 · pp. 31, 35, 36
• The prompt specifies extraction of event names, times, and relevant EHR locations, with analysis of how extracted information relates to the ground truth.It also asks the model to focus on useful information from retrieved medical knowledge.Source: §9.3 · pp. 36–37
• Reasoning instructions require concise, clinically specific contextual inference, prohibit reverse inference, and require medical knowledge and ground-truth references to remain implicit.The prompt also prohibits “maybe,” “possible,” and “though.”Source: §9.3 · p. 37
• Final results must include only ground-truth items correctly inferred during reasoning, while Supplementary Tables 4–5 list task instructions for EHR-Bench and MIMIC-IV-CDM/EHRSHOT.The prescribed output separates Extraction, Reasoning, and Final Results.Source: §9.3 · p. 37, Table 4 · p. 38, Table 5 · p. 39

9.4 Case Collection

• Case examples show single-item and multi-item event formats, plus free-text EHRSHOT and MIMIC-IV-CDM inputs and a chronological EHR-Bench record.The examples include structured measurements, medications, conditions, orders, and other clinical events.Source: §9.4 · pp. 39, 40, 43
• The case study shows the medical graph identifying valid medical entities from the EHR trajectory and using them to enhance reasoning.The figure presents the patient trajectory, connections between context medical entities and target items, and a reasoning chain for inferring a diagnosis.Source: Figure 1 · p. 48