Evolving Interactive Diagnostic Agents in a Virtual Clinical Environment

1 Introduction

• Diagnosis requires clinicians to integrate partial information, choose examinations or commit to a diagnosis, while passive instruction-style training assumes a fixed patient record.The introduction says this static paradigm removes interaction and revision as evidence accumulates, and models often fail to plan test choices, stopping, and final diagnosis.Source: §1 · pp. 1–2
• The framework trains an LLM through multi-turn RL to recommend examinations or finalize diagnoses, using DiagGym to simulate examination results from evolving patient states.The agent explores patient trajectories and receives rewards based on examination informativeness, diagnostic accuracy, and efficiency.Source: §1 · pp. 2–3
• DiagBench contains 2.2K physician-validated cases from four sources and 3.3K physician-authored weighted rubrics for evaluating interactive diagnostic decisions.Cases are linked to physician-verified reference trajectories; the rubrics cover a subset of 399 cases.Source: §1 · p. 3
• DiagGym outperforms the DeepSeek-v3-based simulator in instance-wise consistency (96.90% vs. 88.81%) and examination-wise Wasserstein distance (0.128 vs. 1.336).Its normalized variance is 4.65 versus 5.31 for ground truth, which the passage describes as robust simulation diversity with reduced mode collapse.Source: §1 · p. 3
• In end-to-end evaluation, DiagAgent leads by 11.20% in accuracy and 17.58% in F1 score, and exceeds the runner-up by 7.1% in weighted rubric score.In-domain single-turn results also exceed the nearest competitor by 8.94% in diagnostic accuracy and 43.99% in examination recommendation hit rate.Source: §1 · p. 3
• On the left multi-center out-of-distribution test set, DiagAgent reaches 63.84% accuracy versus Claude-4-Sonnet’s 60.12% and a weighted rubric score of 50.27 versus 39.07.In single-turn evaluation, its hit rate is 65.30% and accuracy is 92.57%, versus GPT-4o’s 52.09% and 87.06%.Source: §1 · p. 3

2 Problem Formulation

• DiagGym is a conditional textual EHR generator that produces synthetic examination results from a patient’s background profile and prior examination records.The patient state includes the complaint, medical history, final diagnosis, and records of examinations already performed.Source: §2 · p. 3
• DiagGym generates plausible results for queried examinations across patient states, enabling safe, repeatable RL training without direct access to real patient records.The passage says the model captures conditional dependencies across diseases, histories, and prior tests.Source: §2 · p. 4
• The agent observes an initial inquiry and performed examination results, then selects among clinical examinations or a final-diagnosis action.Its learnable policy is parameterized by a large language model.Source: §2 · p. 4
• DiagGym returns results for the recommended examination, updating the state; the trajectory continues until the agent chooses a final diagnosis.The environment’s results serve as feedback, and the trained agent manages trajectories by iteratively selecting relevant examinations before diagnosis.Source: §2 · p. 4
• The training objective maximizes expected cumulative reward, combining diagnostic accuracy, relevant examination recommendations, and fewer turns.The three sub-rewards are weighted by hyperparameters.Source: §2 · p. 4

3.1.1 Evaluation Settings

• DiagGym was evaluated on 863 MIMIC-IV cases spanning distinct diseases and 35,548 examination records, using instance-level quality, distribution-level fidelity and diversity, and computational metrics.The simulator reconstructs examination results sequentially from the patient profile, prior results, and the current query, enabling responses to arbitrary examination queries during RL training.Source: §3.1.1 · p. 5

Automated Evaluation

• DiagGym achieved a similarity score of 3.565 versus 2.576 for DeepSeek-v3 and 2.495 for Qwen2.5-72B, and the lowest Wasserstein Distance (0.128) and FID (0.747).Across nearly all automated metrics, it combined high fidelity with greater computational efficiency; baselines tended toward high-diversity distribution gaps or overly deterministic outputs.Source: Automated Evaluation · pp. 6–7, Table 1 · p. 7

Computational Efficiency

• DiagGym generated examination results in about 0.5 seconds on a single A100, compared with more than a minute on a 16×A100 setup for DeepSeek-v3-671B.The reported speedup supports the rapid interactions needed for diagnostic-agent reinforcement learning without compromising generative quality.Source: Computational Efficiency · p. 7

Physician Evaluation

• Physicians rated DiagGym’s average similarity at 4.49 and majority-vote consistency at 95%, while baseline consistency ranged from 44% to 74%.Physicians also noted that DiagGym avoided over-extrapolation and logical inconsistencies while maintaining balanced, case-aligned coverage.Source: Physician Evaluation · p. 7, Table 2 · p. 8

3.2.1 Evaluation Settings

• DiagBench contains 2,257 physician-validated cases—750 in-domain and 1,507 out-of-domain—with 3,318 physician-authored rubrics covering diagnostic interaction logic and necessary examinations.Cases include an initial inquiry, a physician-curated reference trajectory, and a final diagnosis; the out-of-domain subsets draw on PMC-OA, MTSamples, and DDXPlus.Source: Table 3 · p. 8, §3.2.1 · p. 8

Single-turn Evaluation / End-to-End Evaluation

• Single-turn evaluation extends an oracle trajectory by one response, whereas end-to-end evaluation lets agents choose actions and interact with DiagGym until diagnosis.End-to-end assessment uses recommendation precision, recall, and F1, final-diagnosis accuracy, and weighted rubric scores for criteria such as examination order and critical rule-out steps; real EHRs cannot supply results for examinations not performed.Source: Single-turn Evaluation · pp. 8–9, End-to-End Evaluation · p. 9

In-Domain Analysis

• DiagAgent-14B achieved a 68.49% Hit Ratio and 87.87% diagnosis accuracy in single-turn evaluation on MIMIC-IV, exceeding the strongest medical LLM’s recommendation performance by over 40%.It also exceeded general-purpose models such as DeepSeek-v3 by nearly 48% on examination recommendation; basic LLMs and medical-specialized models remained well below the DiagAgent series.Source: In-Domain Analysis · pp. 9–10, Figure 3 · p. 10

Out-of-Domain Analysis

• 65.30% Hit Ratio and 92.57% Diagnosis Accuracy accompany DiagAgent-14B’s OOD leadership; on DiagBench OOD it also leads with 33.63% F1-score and 63.84% Diagnosis Accuracy.Its OOD dialogues average 6.88 turns, and it is the only model described as achieving high precision in both examination retrieval and diagnosis.Source: Out-of-Domain Analysis · pp. 10–11, Figure 4 · p. 12

3.2.3 End-to-end Evaluation Analysis on Automatic Metrics

• 62.63% diagnostic accuracy and 52.14% recall: DiagAgent-14B leads in-domain, sustaining 6.66-turn dialogues versus about 2.5 turns for DeepSeek-v3.It also achieves the highest scores across the reported evaluation metrics; its accuracy exceeds the nearest competitor by over 10%.Source: In-Domain Analysis · p. 11, Figure 4 · p. 12

3.2.4 End-to-end Evaluation Analysis on Rubric-based Metrics

• 32.86% weighted rubric score: DiagAgent-14B surpasses Qwen3-235B by over 8% and MDAgent by over 10% in-domain.General LLMs outscore smaller medical-specialized models, while MDAgent offers only marginal gains over its base model, suggesting discussion alone does not ensure procedural quality.Source: In-Domain Analysis · p. 11, Figure 5 · p. 13

Out-of-Domain Analysis

• 50.27% aggregate weighted score: DiagAgent-14B leads on OOD rubrics by over 10% versus both the strongest proprietary and open-source baselines.It surpasses Claude-4-Sonnet by 10.65% on PMC-OA, GPT-4o by 18.62% on MTSamples, and DeepSeek-V3 by 19.98% on DDXPlus; reported improvements are significant (p < 0.05).Source: Out-of-Domain Analysis · p. 12, Figure 5 · p. 13

Effect of Reward Design

• 32.76% → 46.86% and 35.95% → 47.89% examination-recommendation F1: adding the examination reward improves Qwen2.5-7B and Qwen2.5-14B over diagnosis-reward RL.Diagnosis-only RL already improves accuracy over full SFT, but its examination-recommendation F1 remains at or below 36%; the added reward also yields slight accuracy gains.Source: Effect of Reward Design · p. 14

Model Size and Family

• 61.63% accuracy and 47.89% F1: Qwen2.5-14B achieves the best post-training performance, ahead of Qwen2.5-7B and Llama3.1-8B.Virtual-environment RL improves diagnosis accuracy across all three base models, while base-model quality constrains the attainable performance ceiling.Source: Model Size and Family · p. 14

3.4 Case Study

• Case studies show DiagAgent following reference-aligned test sequences and dynamically adjusting inquiry, but a failure case exposes omitted emergency stabilization and surgical notification.In an appendicitis case, CBC and contrast CT findings support the diagnosis; in an infection case, cultures guide targeted investigation and timely stopping, whereas an unstable ectopic-pregnancy case misses highest-weighted emergency-care rubrics.Source: A Case of Interactive Diagnostic… · p. 14, Cases of Rubric-based Evaluation · pp. 15–16

Supervised Finetuning (SFT) vs. Reinforcement Learning (RL)

• 61.47%, 53.33%, and 62.67% diagnosis accuracy: RL-trained DiagAgent beats SFT-only counterparts on Qwen2.5-7B, Llama3.1-8B, and Qwen2.5-14B, respectively.SFT performance plateaus after epoch 3 even when extended to 10 epochs; the ablation compares zero-shot, full SFT, cold-start, diagnosis-reward RL, and full dual-reward DiagAgent.Source: Experimental Design · p. 13, §3.3 · p. 13, Supervised Finetuning (SFT) vs.… · p. 14

4 Discussion

• Current LLMs often struggle to plan end-to-end diagnostic trajectories, including which examinations to order, when to stop, and when to diagnose.The paper contrasts these limitations with diagnosis as a dynamic process in which decisions adapt to new information.Source: §4 · p. 16

Main Contribution

• DiagGym generates novel examination results in a virtual clinical environment, while DiagAgent uses end-to-end multi-turn reinforcement learning to optimize diagnostic accuracy and examination efficiency.DiagGym is distinguished from simulators constrained to static, pre-collected data.Source: Main Contribution · p. 16

Key Findings

• Reinforcement learning improves interactive trajectory planning over supervised fine-tuning across 7B–14B Qwen2.5 and Llama3.1 models, with reported gains in accuracy, efficiency, and robustness.The paper also reports improvements over evaluated LLMs and agentic systems, without specifying those comparisons numerically in these passages.Source: Main Contribution · p. 16, Key Findings · p. 17

Limitations and Future Work

• DiagGym relies primarily on MIMIC-IV, which may introduce ED- and ICU-specific biases, and is a text-based probabilistic model rather than a verified disease-mechanism simulator.The authors propose cross-center data and clinical knowledge priors as directions for improving fidelity.Source: Limitations and Future Work · p. 17
• Evaluated models reach at most 14B parameters, a scale the authors say may constrain the framework’s full potential.They suggest larger foundation models could improve both simulation fidelity and interactive diagnostic capabilities.Source: Limitations and Future Work · p. 17
• DiagAgent’s rubric scores are modest partly because the rubrics reward immediate treatment and patient-management actions outside the agent’s diagnostic focus.The authors identify integrating treatment planning with diagnosis as future work.Source: Limitations and Future Work · p. 17

5 Methods

• The Methods section covers DiagGym and DiagAgent training processes and the baseline methods used for comparison.Source: §5 · p. 17

5.1 DiagGym Training

• The DiagGym Training subsection introduces the system’s training procedure.Source: §5.1 · p. 17

Data Construction

• The patient-EHR dataset used to train DiagGym was derived from MIMIC-IV.Source: Data Construction · p. 17

a. DiagGym Data Construction

• 118,478 restructured patient EHRs comprise profiles and time-ordered examinations; 114,239 train the world model, while 4,239 are reserved for evaluation and sampled into 863 disease-unique cases.Cases without physical examination records or with discharge diagnoses already present in medical history were excluded; examinations were chronologically ordered, with earliest repeated pre-admission records prioritized.Source: a. DiagGym Data Construction · pp. 18–19

Training Details

• DiagGym is trained as an autoregressive text-generation model that reconstructs examination-result tokens from prior examination records.Numerical and textual results are both treated as free text.Source: Training Details · p. 19

Data Construction

• DiagAgent’s 15,324 training trajectories pair diagnosis-free initial inquiries with LLM-generated preliminary diagnoses and rationales, real-EHR examination sequences and results, and final diagnoses.The three components are generated together, filtered to prevent diagnosis leakage into earlier text, and converted into assistant and user dialogue turns.Source: Data Construction · pp. 20–21

Training Details

• DiagAgent training uses a two-stage process: cold-start instruction tuning on 1,000 manually selected, high-quality cases precedes GRPO reinforcement learning over the full training set.The cold-start loss applies only to assistant-response tokens; RL rewards diagnosis accuracy and examination recommendation alignment while penalizing excessive turns.Source: Training Details · pp. 21–22

5.3 DiagBench Construction

• DiagBench draws on four sources to assess in-domain and out-of-domain diagnostic trajectories, with physician review of case plausibility and, for OOD cases, profile and examination relevance.The four sources are MIMIC-IV, PMC-OA, MTSamples, and DDXPlus; physicians review each case’s inquiry and reference trajectory.Source: §5.3 · pp. 22–23, Figure 8 · p. 24
• The benchmark combines 750 MIMIC-IV cases with 631 PMC-OA, 379 MTSamples, and 497 DDXPlus cases, and its rubric set covers 399 cases with 3,318 physician-authored rubrics.PMC-OA’s complex reports and MTSamples’ incomplete transcripts led to substantial filtering; DDXPlus retained almost all candidates during reference-trajectory extraction.Source: §5.3 · pp. 23, 25
• For rubric evaluation, two physicians independently authored process-oriented rubrics for sampled cases, and a third reviewed coverage and coherence; the final set contains 399 cases.Rubrics prioritize history taking, hypothesis generation, and test ordering, rather than final-diagnosis accuracy alone.Source: §5.3 · pp. 23, 25
• DiagBench assigns each rubric an importance weight from 0 to 10, prioritizing high-impact clinical steps such as test ordering and diagnostic narrowing.Ancillary actions without a clear clinical rationale receive lower weights.Source: §5.3 · p. 25

5.4 Baselines

• Baseline comparisons include general and medical LLMs, plus MedAgents and MDAgents, both implemented with DeepSeek-V3 as their base model.The listed LLM baselines include OpenBioLLM, Baichuan-M1, Baichuan-M3, and GPT-OSS.Source: §5.4 · pp. 25–26
• DiagGym world-model baselines use zero-shot prompts except for one-shot numerical-examination prompting, adopted to ensure consistent formatting and reliable parsing.Closed-source API models are excluded from scalable RL training comparisons because of latency and cost.Source: §5.4 · p. 26
• DiagAgent comparisons use zero-shot prompt 5 as a standardized interface, with end-to-end simulated cases and single-turn real cases prompting for the next action.The real-case prompt explicitly directs the model to request an examination or make a final diagnosis.Source: §5.4 · p. 26

6 Data Availability

• Licensing restrictions prevent direct release of MIMIC-IV, though the authors are discussing possible public availability with relevant parties.Source: §6 · p. 26

• P.Q., C.W., and J.L. contributed equally and served as the authors who drafted the work.Source: §9 · p. 30
• All listed authors contributed to the work’s conception or design, and P.Q., C.W., and J.L. also acquired, analyzed, or interpreted data.Source: §9 · p. 30
• The remaining listed authors reviewed the work critically; all authors approved the published version and accepted accountability for its accuracy and integrity.Source: §9 · p. 30

• The section provides detailed calculation formats for the adopted evaluation metrics.Source: §A.1 · p. 31

A.1.1 DiagGym Evaluation

• Instance-wise evaluation combines two traditional quantitative metrics with two LLM-based semantic assessments of generated examination sequences.Source: §A.1.1 · p. 31
• For numerical results, NMAE normalizes prediction error by each test type’s global data range and reports a macro-average across examination types.Source: §A.1.1 · p. 31
• For free-text radiology reports, BLEU measures n-gram overlap between generated narratives and reference reports.Source: §A.1.1 · p. 31
• Step-wise similarity scores each generated result from 0 to 5 against ground truth, conditioned on the queried examinations, case summary, and historical examination records.Source: §A.1.1 · p. 31
• Full-chain consistency uses a binary yes/no assessment of whether generated examination sequences remain clinically consistent and aligned with the patient profile.Source: §A.1.1 · p. 31
• Examination-wise metrics assess distribution fidelity and diversity: numerical fidelity uses 1-Wasserstein distance, while normalized variance measures diversity.For multiple-value examinations, scores are averaged across value items, with same-item computations standardized to unified units.Source: §A.1.1 · pp. 31–32
• For free-text examinations, BioLORD embeddings support FID-based fidelity assessment and inter-case cosine similarity assessment of diversity.Lower FID indicates better ground-truth alignment; higher Intra-LPIPS indicates greater diversity among generated texts.Source: §A.1.1 · p. 32

A.1.2 DiagAgent Evaluation

• Final diagnosis is evaluated with accuracy, using GPT-4o to compare model diagnoses with reference standards while accounting for disease synonyms.Source: §A.1.2 · p. 32
• Examination recommendations are assessed with precision, recall, and F1 in end-to-end evaluation, and hit ratio in single-turn evaluation, with GPT-4o judging synonymous matches.Source: §A.1.2 · p. 32
• The weighted rubric score evaluates diagnostic trajectories against physician-authored process rubrics, weighting satisfied rubrics by clinical significance and averaging scores across cases.GPT-4o determines rubric satisfaction for each case.Source: §A.1.2 · p. 33

A.1.3 Human Validation of Automated Evaluation Metrics

• Twelve senior physicians, each with over 10 years of clinical experience, evaluated 100 sampled instances per metric, with three independent annotations per instance.Binary-task ground truth used majority vote; examination-count ground truth used the median of three physician counts.Source: §A.1.3 · p. 33
• 100.00% Diagnosis Accuracy and 96.00% Rubric Hit Ratio aligned LLM judging with physician consensus; diagnosis alignment exceeded physicians’ 92.67% average.Source: §A.1.3 · p. 33
• The LLM judge’s examination-count MAE was 0.25, compared with 0.15 for disagreement among physicians.Source: §A.1.3 · p. 33

A.2 Supervised Finetuning

• Each multi-turn SFT sample pairs a dialogue history with its next expected response.Source: §A.2 · p. 33
• The dialogue history comprises user inputs and model responses accumulated through turn t.Source: §A.2 · p. 33
• The response at turn t is the supervised target.Source: §A.2 · p. 33

A.3 Ablation on Training Compute (FLOPs)

• At approximately epoch 3, SFT reaches peak diagnostic accuracy and examination-recommendation F1, with negligible gains through epoch 10.The Qwen2.5-7B ablation attributes the plateau to limits of supervised imitation rather than insufficient training steps; the figure shows the end-to-end evaluation across 10 epochs.Source: Figure 1 · p. 34, §A.3 · p. 34

A.4 Selected Examinations Collection

• The distributional analysis retained 11 numerical examinations with over 500 occurrences and five radiology examination types with over 100 occurrences.The 11 numerical examinations comprised 24 subevents; the five textual examination types were radiology exams.Source: §A.4 · p. 35, Table 2 · p. 35

A.5.1 Results on MIMIC-IV; A.5.2 Results on PMC-OA

• For MIMIC-IV and PMC-OA, tables report single-turn Hit Ratio and Accuracy, plus end-to-end examination-recommendation Precision, Recall, F1-score, and final diagnostic accuracy.The MIMIC-IV end-to-end table also reports average conversation turns; the supplied captions do not provide metric values or model comparisons.Source: Table 3 · p. 36, Table 4 · p. 37, Table 5 · p. 38

A.5.3 Results on MTSamples; A.5.4 Results on DDXPlus; A.5.5 Results on combined out-of-domain datasets

• For MTSamples, DDXPlus, and combined OOD data, tables report single-turn Hit Ratio and Accuracy and end-to-end Precision, Recall, F1-score, and final diagnostic accuracy.The combined OOD tables cover PMC-OA, MTSamples, and DDXPlus (N = 1,507); the supplied captions do not provide metric values or model comparisons.Source: Table 7 · p. 40, Table 8 · p. 41, Table 9 · p. 42

A.6.1 Case study for DiagGym

• DiagGym’s simulated findings closely match the case context; its predicted total bilirubin was 6.7 mg/dL versus 4.3 mg/dL recorded, both above range.In a representative jaundice case, the findings supported the confirmed diagnosis of CBD obstruction from a common hepatic duct mass, while minor numerical variations did not change the interpretation.Source: §A.6.1 · p. 46, Figure 2 · p. 47

A.6.2 Case study for DiagAgent

• DiagAgent case studies show rubric-validated procedural success in one case and a management deficit despite correct diagnosis in another.The success case met most high-weighted process criteria; in the ruptured ectopic pregnancy case, the agent omitted emergency-management steps including fluid resuscitation and surgical transfer.Source: §A.6.2 · p. 48, Figure 4 · p. 50, Figure 5 · p. 51

A.7 Prompt Collection; A.7.1 Prompt Design Principles; A.7.2 Full Prompt Content

• Prompting used one-shot examples for schema-constrained data construction and numerical lab formatting, and generally zero-shot prompts for inference and evaluation.The full diagnostic-generation prompt structures pre-arrival case summaries, pertinent results, and a chronological test-ordering and reasoning timeline, with at most 12 turns.Source: §A.7.1 · p. 52, §A.7.2 · pp. 54–56

A.7.2 Full Prompt Content

• The data-filtering prompt rejects case summaries that directly reveal the diagnosis or use a procedure name or non-subjective discomfort as the chief complaint.It assesses information leakage and chief-complaint reasonableness separately.Source: §A.7.2 · pp. 56–57
• The diagnostic prompt directs the model to recommend one additional test when more information is needed, or provide a final diagnosis and rationale when sufficient information is available.The prompt also asks the model to state its current diagnosis and explain the reason for any recommended investigation.Source: §A.7.2 · p. 57
• The diagnosis evaluator accepts aliases and equivalent wording, considers a correct diagnosis valid even with extra complications, and excludes causes, symptoms, and treatment from its judgment.Its output is restricted to “Correct” or “Wrong.”Source: §A.7.2 · p. 60
• Prompt 17 extracts physical exams and laboratory, microbiology, and radiology events, allowing up to 30 related events and combining similar events to avoid duplication.It seeks a complete inventory of tests, including normal, abnormal, and incidental results, while excluding procedures and unrelated tests.Source: §A.7.2 · p. 63
• Prompt 18 asks a medical quality controller to determine whether the final diagnosis in the diagnostic trajectory is clinically equivalent to the profile diagnosis.The prompt requires a Yes-or-No answer and treats synonymous terms such as “CML” and “Chronic Myelogenous Leukemia” as equivalent.Source: §A.7.2 · pp. 63–64