PaperlayerView exact revision on arXiv

Source-linked AI summary

MedCaseAgent: From Multimodal Clinical Evidence to Rare Disease Case Reports

Hongkang Chu, Yifan Shen, Tianjiao Yu, Zhuoqing Zhong, Jianguo Cao, Meihuan Huang, Xu Cao

arXiv:2610.32258v1cs.AI

TL;DR

Rare-disease case-report generation requires rigorous reasoning across fragmented clinical notes, images, and other evidence, a task not well served by existing general clinical documentation approaches. MedCaseAgent combines tool-integrated planning and report writing with new case-report resources. The authors report that it outperforms general-purpose and specialized medical systems, with expert reviews also favoring its reports.

  • Problem

    Generating rigorous rare-disease case reports requires multi-hop reasoning across fragmented multimodal evidence while avoiding clinical hallucinations.

    Source: §1 · p. 2

  • Method

    MedCaseAgent uses an interactive planner-writer-editor workflow with literature retrieval, citation tracking, image analysis, and genetic-search tools, alongside MedCase-150K and MedCase-Bench.

    Source: §1 · p. 2, §3 · p. 3

  • Results

    MedCaseAgent outperforms existing general-purpose and specialized medical LLMs and agents; it retrieves the exact source article for 84.6% of cases versus 46.9% for the strongest direct baseline.

    Source: §1 · p. 2, §5 · p. 10

  • Takeaways & Limitations

    Blinded clinician reviews and user-study findings support the reported clinical relevance, reliability, and usefulness of MedCaseAgent reports.

    Source: §1 · p. 2, §5 · p. 10

Abstract

from arXiv · show

Although Large Language Models (LLMs) have demonstrated potential in clinical documentation, generating the rigorous, evidence-based reasoning required for unprecedented, atypical, and rare disease case reports remains a significant challenge. We present MedCaseAgent, an end-to-end agentic reasoning paradigm that bridges this gap by synthesizing publication-quality reports from unstructured clinical notes. MedCaseAgent leverages a hierarchical structure, integrating patient-level data mining with iterative, tool-integrated reasoning. By dynamically querying medical literature and knowledge bases, the agent maintains high clinical fidelity and assesses case novelty. The framework is inherently modular, supporting various backbone LLMs in a plug-and-play fashion. To facilitate this research, we release MedCase-150K, a large-scale multimodal clinical case-report corpus, and MedCase-Bench, a robust temporally held-out evaluation suite. We also introduce an open-source model post-trained from Qwen3.5-9B and optimized specifically for this task. Under both LLM-judged and rule-based evaluation, MedCaseAgent outperforms existing general-purpose and specialized medical LLMs and agents. Blinded expert reviews and ablations further support the system's clinical relevance and evidence grounding.

1 Introduction

Rare-disease case reports require rigorous synthesis of fragmented clinical evidence and literature, yet drafting them remains a burdensome manual task. MedCaseAgent combines tool-grounded report generation with new multimodal resources, and the paper reports that it surpasses mainstream MLLMs and medical agents.

1 Introduction

  • Rare-disease report writing demands multi-hop reasoning over fragmented, multimodal evidence while avoiding clinical hallucinations such as fabricated citations and spurious gene-literature matches.Physicians must also assess novelty, synthesize histories and treatment plans, and conduct extensive literature reviews.Source: §1 · p. 2
  • MedCaseAgent uses an interactive planner-writer-editor workflow with tools for literature retrieval, citation tracking, image analysis, and genetic search.The framework is presented as an end-to-end system for generating case reports from unstructured clinical notes.Source: §1 · p. 2
  • MedCase-150K and MedCase-Bench provide a large-scale case-report collection and a temporally held-out, rarity-stratified evaluation resource for multimodal report generation.The resources are designed to test interpretation of fragmented multimodal evidence and production of complete reports.Source: §1 · p. 2
  • MedCaseAgent surpasses mainstream MLLMs and medical agents on MedCase-Bench, with clinician evaluations and a user study also supporting its reliability and usefulness.The supplied passages do not specify the evaluation metrics or margins for these comparisons.Source: §1 · p. 2

2 Related Work

Prior clinical agents address imaging, records, diagnosis, and structured tasks, while scientific-writing systems target other domains. MedCaseAgent instead focuses on end-to-end synthesis of patient-grounded, multimodal evidence into cited case reports.

Agentic Systems in Clinical Practice

  • Most clinical agents address classification, segmentation, question answering, planning, diagnosis, recommendation, or short-form writing rather than full case-report synthesis.Related systems improve traceability through retrieval, verification, and iterative refinement, but typically stop short of manuscript-level writing.Source: §2 · pp. 2–3

Automated Scientific and Case-Report Writing

  • General-purpose research-writing agents do not address patient-grounded clinical evidence, rarity assessment, or medical citation grounding.Prior medical case-report prompting studies reported substantial expert revision needs and potentially fabricated references; existing corpora have been used for diagnosis or information extraction.Source: §2 · p. 3

3 Methodology

MedCaseAgent turns fragmented clinical notes and images into structured case reports through planning, drafting, and verification. The methodology also describes the construction of its case-report dataset and temporally held-out benchmark, and a two-stage training recipe for its planner.

3.1 Task Formulation

  • Clinical atoms organize patient-specific notes and diagnostic images into evidence for generating a complete report with source-driven sections.The inputs are partial observations rather than a polished academic narrative; atoms also provide a unit for planner supervision and coverage checks.Source: §3.1 · p. 4, Figure 2 · p. 4

3.2 MedCase-150K and MedCase-Bench Construction

  • MedCase-150K contains 154,872 rare-disease or atypical-presentation reports; MedCase-Bench retains 584 peer-reviewed cases published after January 2026 as a temporal holdout.The benchmark candidates were reviewed for atom fidelity and rarity-label validity.Source: §3.2 · pp. 4–5

3.3 MedCaseAgent: Agentic Medical Case Report Synthesis Pipeline

  • The three-stage workflow plans over clinical atoms and images, drafts from the plan and retrieved evidence, then checks factual support, citations, figures, and report structure.Tool use is concentrated in the Planner, which can retrieve literature, inspect images, search genes, and find similar prior cases.Source: §3.3 · pp. 5–6
  • The Planner maps clinical atoms to manuscript sections, creates an evidence plan from literature and prior cases, and assigns images to report callouts and captions.This intermediate plan connects raw notes to the Writer's manuscript structure.Source: §3.3 · p. 5
  • The Writer separates patient facts drawn from clinical atoms from background and diagnostic claims supported by retrieved literature.The Editor then checks atom support, citation order and density, figure callouts, section order, and unsupported statements.Source: §3.3 · pp. 5–6

3.4 Training Objective for the Planner

  • Planner training combines supervised fine-tuning on teacher-generated trajectories with GRPO, which optimizes tool-use decisions and plan quality using a weighted reward.Reward components check schema, DOI retrieval, evidence support, atom and section coverage, image placement, and quality; DOI retrieval is weighted 0.20, while schema and structure/image placement are each weighted 0.10.Source: §3.4 · p. 6

4 Experiments

Across benchmark and clinical evaluations, MedCaseAgent configurations show stronger report quality and structural grounding, while raw-record and atom-based generation receive comparable expert scores.

4.1 Experimental Setup

  • Baselines receive the same tools; direct comparisons include general and specialist medical VLMs and adapted medical-agent systems, while Base, Tuned, and Max use different centered LLM configurations.Base and Tuned use Qwen3.5-9B, with the latter's planner trained using the Section 3.4 objective; Max uses GPT-5.4.Source: §4.1 · p. 7

4.2 Report Quality and Disease-Stratified Evaluation

  • MedCaseAgent-Tuned raises average score from 64.43 to 67.79, while MedCaseAgent-Max leads every disease subset and stronger backbones form the top report-quality group.Figure 5 compares configurations using five rubric metrics and their average; Table 1 evaluates selected disease subsets.Source: Figure 5 · p. 7, §4.2 · pp. 7–8, Table 1 · p. 8

4.1 Experimental Setup

  • Evaluation combines four-family VLM judging of five report dimensions with disease-subset, rule-based structural, and clinician A/B assessments.The four-model judge ensemble spans four model families; rubric dimensions are scaled to 0–100 for display.Source: §4.1 · p. 7

4.2 Report Quality and Disease-Stratified Evaluation

  • In blinded review, 26 of 32 clinicians identified generated reports in at most three of five pairs, and reviewers more often preferred MedCaseAgent-Max.MedCaseAgent-Base also stays above prior medical-agent baselines across disease-level evaluation.Source: §4.2 · p. 8

4.3 Rule-Based Structural Metrics and Ablation

  • 84.59% Exact DOI versus 46.90% for the strongest direct baseline; MedCaseAgent-Max also leads novelty recognition, figure sequencing, and P0/P1 citation recall.Exact DOI measures per-case retrieval of the PubMed article describing the same patient; the ground-truth article is masked during generation.Source: §4.3 · pp. 8–9, Table 2 · p. 8
  • Removing literature retrieval causes the largest drops in average score and citation depth, while image inspection and disease-index retrieval contribute most to image integration and novelty recognition.Table 3 examines planner training and tool removal using report-quality metrics and novelty recognition.Source: §4.3 · p. 9, Table 3 · p. 9
  • 1.85 points: removing source-DOI reward lowers the four-metric average (71.68 to 69.83), novelty recognition (59.08% to 55.78%), and report quality (67.23 to 65.37).The default reward-weight profile achieves the highest four-metric average among the tested profiles.Source: §4.3 · p. 9

4.4 Clinical Evaluation and User Study

  • Raw-record and atom-based generation score 7.4 versus 7.6 overall, while LLM judgments agree with clinicians on 23–26 of 30 dimension assessments and all hallucination checks.More than 100 users gave largely positive feedback on the system’s usefulness for case-report preparation.Source: §4.4 · pp. 9–10, Table 4 · p. 9

5 Conclusion

MedCaseAgent converts fragmented clinical notes and images into publication-style rare-disease case reports, with MedCase-150K and a 584-case temporal holdout, MedCase-Bench. The conclusion reports gains in grounding and expert preference, while noting that discovery of diseases without retrieval precedent remains the clinician’s role.

5 Conclusion

  • MedCaseAgent-Max retrieves the exact source article for 84.6% of cases versus 46.9% for the strongest direct baseline, recognizes novelty in 88.4%, and leads on must-cite citation recall.It is the best-performing system across disease subsets and scores above the human-publication reference under a four-judge ensemble; clinicians more often preferred its reports, and most did not consistently identify generated reports.Source: §5 · p. 10
  • For diseases without precedent in retrieval sources, MedCaseAgent can organize available evidence and uncertainty, but discovery remains the clinician’s task.Source: §5 · p. 10

AI Use Statement

  • Generative AI was used to polish language and readability; the authors reviewed the edits and take responsibility for the final manuscript.LLMs used for clinical-atom construction, training-trajectory generation, report generation, and automated evaluation are part of the research methodology.Source: AI Use Statement · p. 10

Ethics Statement

  • Hospital-based cases were evaluated under existing IRB approval after de-identification, and generated reports require clinical review before dissemination.Review should cover patient facts, image interpretations, and citation support.Source: Ethics Statement · p. 10

Reproducibility Statement

  • The authors plan to release the datasets, code, construction scripts, evaluation prompts, metadata, and data splits, while hospital cases used for expert evaluation remain non-public.Appendices describe the method, evaluation, workflow, validation, ablations, compute, and human- and LLM-evaluation protocols.Source: Reproducibility Statement · p. 10, Appendix · p. 17

A Extended Related Work: Relevant Benchmarks

Existing clinical benchmarks largely assess narrower tasks, while case-report generation requires multimodal evidence interpretation, patient-course reconstruction, literature linkage, and scientific writing. The appendix displays the agent workflow and reports staged expert verification of the benchmark’s clinical atoms and rarity labels.

A Extended Related Work: Relevant Benchmarks

  • Existing clinical-agent benchmarks cover reasoning, multimodal decision-making, workflow, safety, and extraction, but case-report generation combines fragmented evidence interpretation, patient-course reconstruction, literature linkage, and scientific writing.Source: A Extended Related Work: Relevant… · p. 18

B Prompt and Workflow Displays

C Compression To Clinical Atoms

  • Source reports are compressed into fragmented clinical atoms that preserve patient history, findings, tests, interventions, outcomes, and image details while removing background prose and final diagnostic exposition.Source: Figure B1 · p. 20

Prompt Stack Used by MedCaseAgent

C.1 Human Verification of Clinical Atoms and Rarity Labels

  • 584 of 632 candidate cases passed sequential atom-fidelity and rarity-label review, for 92.41% overall retention; pre-adjudication agreement was 94% and 87%, respectively.Cases failing either stage were excluded rather than edited or relabeled, and each case received at least two reviews.Source: §C.1 · p. 23, Table C1 · p. 23
  • Disease-specific subsets use 100 cases per category except Erdheim–Chester disease, which uses all 53 available corpus cases.Source: §C.1 · p. 23

C.2 Corpus Statistics and Data Construction Views

The corpus views describe how cases are classified by rarity or novelty and visualize temporal coverage, modality breadth, and source-journal distribution. The classification distinguishes epidemiological rarity from novel or atypical presentations.

Rarity and novelty classification

  • The four labels distinguish epidemiologically rare disease, unprecedented presentation, atypical presentation, and standard cases; the first three form the rare-or-novel group for stratified views.Source: §C.2 · p. 23

Corpus Statistics and Data Construction Views

  • Annual case-report counts and cumulative coverage are plotted to show the corpus’s temporal coverage.Source: Figure C2 · p. 24
  • Benchmark examples span radiology, clinical photography, ophthalmology, dentistry, pathology, and structured graphical evidence.Source: Figure C3 · p. 25
  • The journal-source view pairs leading venues with a long-tail panel to show whether corpus coverage is concentrated or broadly distributed.Source: Figure C4 · p. 26

D GRPO Reward-Weight Sensitivity

Across the evaluated reward-weight profiles, the default performed best on the four-metric average, atom coverage, and report quality, while other profiles led on novelty recognition or image placement. Removing the source-DOI reward lowered the average, novelty recognition, and report quality.

Reward design and experimental setup

  • The default assigns 0.20 each to DOI retrieval, evidence support, coverage, and quality, and 0.10 each to schema compliance and image-placement constraints; profiles were compared from the same SFT checkpoint with other settings fixed.The alternatives were Uniform, Retrieval-heavy, Quality/image-heavy, and No-DOI.Source: D GRPO Reward-Weight Sensitivity · p. 27

Results

  • 71.68 was the highest four-metric average under Default, which also led atom coverage and report quality; Retrieval-heavy led novelty recognition and Quality/image-heavy led image placement.Across the four profiles retaining source-DOI reward, averages ranged from 70.65 to 71.68, a 1.03-point spread.Source: Table D1 · p. 27, D GRPO Reward-Weight Sensitivity · p. 27

Removing the source-DOI reward

  • 69.83 was the No-DOI four-metric average, versus 71.68 for Default; novelty recognition fell from 59.08% to 55.78%, and report quality from 67.23 to 65.37.The authors interpret these results as supporting a source-DOI reward contribution under the evaluated training configuration.Source: D GRPO Reward-Weight Sensitivity · p. 27

E Training and Compute Resources

The tuned planner uses a two-stage SFT-to-GRPO adaptation of Qwen3.5-9B, trained on teacher-generated planning trajectories with weighted rewards. The reported compute estimate covers the adaptation run only, not teacher inference or development work.

E Training and Compute Resources

  • The planner is initialized from Qwen3.5-9B and trained with a two-stage SFT-to-GRPO recipe on 3,000 teacher-generated planning trajectories.The Qwen3.5-397B-A17B teacher builds plans from compressed clinical atoms and mapped images, using image-analysis and literature-search actions.Source: E Training and Compute Resources · p. 28
  • Training used LoRA adapters targeting attention and MLP projections, with rank 16, alpha 32, dropout 0.05, 4-bit quantization, and bfloat16 compute.Gradient checkpointing was also used.Source: E Training and Compute Resources · p. 28
  • Approximately 192 H100 GPU-hours were estimated for the planner adaptation, which took about one day on eight 80GB NVIDIA H100 GPUs.The estimate excludes teacher-model inference, preliminary prompt or tool development, and inference-only evaluation.Source: E Training and Compute Resources · p. 28
  • SFT used two epochs, while GRPO used four candidate trajectories per prompt and a 500-update cap, with weighted schema, DOI, evidence, coverage, image, and quality rewards.The default reward path is deterministic and does not call a live judge.Source: E Training and Compute Resources · pp. 28–29

F.1 Hospital-Based Expert Evaluation

The hospital evaluation compares reports generated from raw records and report-derived atoms with clinician-written reports, and separately calibrates LLM-judge agreement and conducts blinded reviewer studies. The expanded baseline comparison reports MedCaseAgent-Max as least often identified as generated and most often preferred over the reference.

F.1 Hospital-Based Expert Evaluation

  • Report-derived atoms came from clinician-written manuscripts, retaining core history and care while compressing some source context; raw records required filtering, deduplication, reconciliation, and timeline reconstruction.Images were supplied separately without captions or explicit atom-level links.Source: §F.1 · p. 30, Table F1 · p. 31
  • On 30 hospital cases reviewed by three experts each, atom-based generation scored 7.6/7.6/7.6 and raw-record generation 7.4/7.5/7.8 for overall quality, timeline, and omission control.Clinician-written reports scored 7.5/7.8/7.9; atom-based generation scored slightly higher on quality and timeline, while raw-record generation scored higher on omission control.Source: §F.1 · p. 30

F.2 LLM–Clinician Agreement

  • In 30 paired LLM–clinician assessments, agreement was 23/30 for factuality, 24/30 for coherence, 26/30 for omission, and 24/30 for citation accuracy; hallucination ratings matched in all 30.Agreement on scored dimensions allowed an absolute difference of at most two points on a 10-point scale.Source: §F.2 · pp. 31–32, Table F2 · p. 31

F.3 Expert-Reviewer Blind A/B Test

  • In a blinded A/B study, 26 of 32 experienced clinicians did not consistently identify generated reports across five assigned pairs, and reviewers more often preferred MedCaseAgent reports.The study yielded 160 pairwise judgments; independent random guessing meets the same source-identification threshold with probability 81.25%.Source: §F.3 · p. 32

F.4 Web-Based User Studies and Feedback

  • More than 100 users participated in web-based studies and feedback collection, and feedback was largely positive about the system’s usefulness for case-report preparation.The platform collected study participation and system-usefulness feedback.Source: §F.4 · p. 32

G LLM-Judged Blind A/B Comparison Across Baselines

G.1 Judge Robustness and Self-Preference

The judge-configuration comparison reports general-quality scores from individual judges and their four-judge average; three judges also serve as model backbones in the configuration comparison.

G.1 Judge Robustness and Self-Preference

  • General-quality scores are reported under individual judges and the four-judge average, with three judges also used as backbones in the configuration comparison.The supplied passage identifies GPT-5.4, Gemini 3.1 Pro, and Claude Opus 4.6 as those judges.Source: §G.1 · p. 35, Table G2 · p. 35

H Example Case Reports Generated by MedCaseAgent

The appendix presents three complete generated case reports as qualitative examples of the planner–writer–editor pipeline. The broader-impact discussion frames MedCaseAgent as clinician-reviewed support for organizing evidence and reducing manuscript-preparation burden.

H Example Case Reports Generated by MedCaseAgent

I Broader Impacts

  • MedCaseAgent is intended to reduce documentation and literature-review burden by organizing fragmented notes, images, prior-case evidence, and citation candidates into a structured draft for expert review.The discussion describes the task as evidence-based and grounded in an observed patient case.Source: I Broader Impacts · p. 40
  • Planning-stage retrieval may surface diagnostic possibilities or comparative context for specialists evaluating difficult cases, but the outputs are not intended to replace expert diagnosis.The discussion also proposes faster dissemination and more consistent report structure as possible broader effects.Source: I Broader Impacts · p. 40
  • Before submission or dissemination, clinicians should verify patient facts, image interpretations, and citation support; real-record deployment should follow local governance, consent, de-identification, and access-control requirements.Source: I Broader Impacts · p. 40

J Release and LLM Usage Disclosure

The authors plan to release the datasets and materials for MedCaseAgent. They disclose that LLM-based tools assisted only with manuscript writing and editing, not with creating study records or results.

J Release and LLM Usage Disclosure

  • The release will include MedCase-150K, MedCase-Bench, source code, data-construction scripts, evaluation prompts, metadata, and data splits under CC-BY-NC 4.0.Source: J Release and LLM Usage Disclosure · p. 41
  • LLM-based tools were used for manuscript writing and editing assistance, not to fabricate patient records, measurements, baselines, or outcomes.Reported numbers must come from the evaluation pipeline and be traceable to released scripts, prompts, and logs.Source: J Release and LLM Usage Disclosure · p. 41

Related papers

Loading 2610.32258v1…