Source-linked AI summary
CaseWeaver: A Multi-Agent Framework for Multimodal Virtual Clinical Case Generation
Jierui Qu, Jiachuan Peng, Lin Li, Kyle Lam, Jianing Qiu
TL;DR
Longitudinal multimodal clinical data are costly to acquire and often fragmented or incomplete. CaseWeaver uses a timeline-anchored patient graph and protocol-guided agents to generate linked evidence across modalities. It achieved the highest reported clinical-inferability and diversity results among the evaluated methods, though its diversity metric does not assess medical correctness.
Problem
Longitudinal multimodal patient data are difficult to collect at scale because collection requires substantial resources and real-world cohorts often have fragmented records and missing modalities.
Source: §1 · p. 1Method
CaseWeaver organizes patient context, disease progression, clinical events, and expected observations in a timeline-anchored LCCG, then routes scoped observation subgraphs to protocol-constrained modality agents.
Source: §2 · p. 3, §2.1 · p. 4Results
CaseWeaver ranked highest among four baselines for diagnostic success (S = 0.930) and CII (0.860), and highest for VCD across three disease tasks (macroaverage 0.618).
Source: §3.2 · p. 7, §3.3 · p. 8Takeaways & Limitations
The generated cases provide evidence that can be progressively acquired and integrated to support diagnostic reasoning, with potential use in medical AI training and clinical education.
Source: §4 · p. 8The VCD nearest-neighbor term penalizes duplicate or near-duplicate cases but does not assess their medical correctness.
Source: Virtual Case Diversity · p. 6
Abstract
from arXiv · showhide
Clinical diagnosis relies on consistent multimodal data collected from the same patient throughout the disease course, yet such data are difficult to acquire at scale because of collection costs, missing modalities, fragmented systems, and longitudinal follow-ups. Existing synthetic-data approaches largely focus on individual modalities or vision-language dual modalities at report-level generation. Little work has been done to construct synthetic data with consistent patient backgrounds, coherent disease trajectories, and interrelated modality-specific evidence at a complete clinical case level. We introduce CaseWeaver, a multi-agent framework built around a timeline-anchored Latent Clinical Case Graph (LCCG). The LCCG organizes patient context, latent disease states, clinical events, and expected observations in a shared patient-level representation. Modality-agents use scoped observation subgraphs and clinical protocols to generate evidence including clinical records, laboratory results, physiological signals, and medical images. We evaluate clinical inferability using a calibrated AgentClinic protocol and case diversity using Virtual Case Diversity (VCD) score. CaseWeaver outperformed general-model and agentic-workflow baselines on both metrics, producing more diverse and coherent multimodal virtual clinical cases.
1 Introduction
Longitudinal multimodal clinical data are difficult to collect at scale, while existing synthetic-data methods lack a unified patient-level representation of heterogeneous evidence. CaseWeaver addresses this gap with a timeline-anchored graph and modality-specific agents that generate interrelated evidence for a shared virtual patient.
1 Introduction
- Collecting longitudinal multimodal data at scale is resource-intensive, and real-world cohorts often have fragmented records and missing modalities.Sharing patient data raises privacy concerns, and cohorts may underrepresent populations.Source: §1 · pp. 1–2
- Most synthetic-data methods target a single data type or narrow objective, without a unified patient-level representation of heterogeneous clinical evidence.Prior work spans generated medical text, structured data, physiological time series, and medical images.Source: §1 · p. 2
- Cross-modal generation either depends on source-dataset distributions or lacks modality-specific evidence for the same patient.MedM2G conditions one modality on others but remains constrained by source data; Patient-Zero generates clinically grounded patient agents but centers on reports and natural-language descriptions.Source: §1 · p. 2
- CaseWeaver uses a timeline-anchored LCCG and protocol-guided modality agents to generate interrelated evidence for consistent patient backgrounds and disease trajectories.The agents generate clinical records, laboratory results, physiological signals, medical images, and pathology images linked to the same patient state.Source: §1 · p. 2
2 Method
CaseWeaver builds virtual clinical cases around a timeline-anchored Latent Clinical Case Graph (LCCG), then routes scoped observations to protocol-constrained modality agents. Its methodology evaluates diagnostic inferability and case diversity.
2.1 Latent Clinical Case Graph Construction
- Guideline-derived disease programs and a random seed shape each case plan, with a similar de-identified MIMIC-IV case informing examination planning.The plan specifies presentation, acuity, severity, demographics, risk factors, comorbidities, and history; disease mechanisms remain governed by guideline-derived knowledge.Source: §2.1 · pp. 3–4
- The timeline-anchored LCCG provides a shared patient-level representation linking clinical time points, latent disease states, observable findings, and their relations.Each observable finding traces to an underlying disease or pathophysiology node, while cross-modal edges connect findings from the same latent patient state.Source: §2.1 · p. 4
2.2 Protocol-Driven Multimodal Evidence Rendering
- The router sends each modality agent a temporally anchored observation subgraph containing only its scoped findings, patient context, and assigned tools.Agents can process their respective subgraphs in parallel, while latent disease nodes and observations from other modalities remain excluded.Source: §2.2 · p. 4
- Protocol-driven tools transform observation requirements into modality representations by applying constrained finding patches to protocol-specific baselines before rendering evidence.The angiography example selects coronary anatomy and applies acquisition protocols; the shared update confines modifications to each protocol’s representation space.Source: §2.2 · p. 5, Figure 2 · p. 5
2.3 Evaluation Methodology
- Clinical inferability combines diagnostic success and information coverage, with calibrated thresholds and an imbalance range determining whether a case qualifies.Success and coverage are evaluated across independent runs; criteria are calibrated from real-data reference cases and assessed on a validation set.Source: AgentClinic-Based Clinical… · pp. 5–6
- Virtual Case Diversity combines pairwise dispersion with nearest-neighbor uniqueness across case features, but does not measure medical correctness.Its feature groups are core phenotype, patient background, clinical course, and evidence plan; the nearest-neighbor term penalizes duplicate or near-duplicate cases.Source: Virtual Case Diversity · p. 6
3 Evaluation
The evaluation presents a temporally aligned myocardial-infarction case and assesses clinical inferability and case diversity. CaseWeaver scores highest on diagnostic success, CII, and VCD across the reported comparisons.
3.1 Generated Virtual Case
- A generated 68-year-old woman’s temporally aligned ECG, troponin, echocardiography, and angiography findings formed a coherent phenotype of inferior STEMI with right ventricular involvement.Angiography 90 minutes after symptom onset showed complete proximal right coronary artery occlusion.Source: §3.1 · p. 7
3 Evaluation
- AgentClinic criteria were calibrated on 130 reference cases and validated on 54; 87.0% of validation cases met the combined criterion, while CaseWeaver was compared with four baselines spanning two paradigms and two backbone models.For all methods, image evidence was rendered from case-specific prompts using gemini-3-pro-image.Source: §3 · p. 6, §3.2 · p. 7
3.2 Clinical Inferability Evaluation
- Figure 3 relates calibration and validation distributions for CII, coverage, and the success–coverage gap to a joint view of diagnostic success and coverage.CaseWeaver maintained high diagnostic success and information coverage, indicating accessible and diagnostically informative evidence.Source: §3.2 · p. 7, Figure 3 · p. 7
- CaseWeaver achieved the highest diagnostic success rate (S = 0.930) and CII (0.860) among evaluated methods.Methods were compared on cases for the same fixed list of 50 diseases.Source: §3.2 · p. 7, Table 1 · p. 8
3.3 VCD Results
- CaseWeaver achieved the highest VCD across myocardial infarction, diabetes, and colorectal cancer, with a macroaverage of 0.618 versus 0.265–0.352 for the four baselines.Its task scores were 0.547, 0.620, and 0.686, respectively.Source: Table 1 · p. 8, §3.3 · p. 8
3.2 Clinical Inferability Evaluation
- The OpenAI Agents SDK baseline had the highest information coverage (C = 0.821) but a diagnostic success rate of only 0.752.The authors suggest accessible evidence may lack sufficient organization or diagnostic direction; CaseWeaver’s evidence supported interactive reasoning from information acquisition to diagnosis.Source: §3.2 · p. 8
4 Conclusion
CaseWeaver links longitudinal disease progression with multimodal examination findings in patient-level virtual cases. Its evaluations indicate diverse profiles and evidence that can be progressively acquired and integrated for diagnostic reasoning.
4 Conclusion
- CaseWeaver links longitudinal disease progression with multimodal examination findings to produce patient-level cases with an inferable clinical structure.The cases use a timeline-anchored LCCG and protocol-driven modality agents to generate interrelated evidence for the same patient.Source: §4 · p. 8
- AgentClinic and VCD evaluations indicate that generated cases encompass diverse patient profiles and provide evidence that can be progressively acquired and integrated for diagnostic reasoning.Source: §4 · p. 8
- The framework may serve as a virtual case generator for medical AI training and clinical education.Source: §4 · p. 8