PaperlayerView exact revision on arXiv

Source-linked AI summary

KMGen: A Skill-based Approach for Synthetic Individual Patient Data Generation

Jalen Jiang, Chufan Gao, Ethan Rasmussen, Stephen Z. Xie, Jimeng Sun

arXiv:2608.22618v1cs.LG

TL;DR

Clinical-trial IPD is rarely released, leaving prior work focused mainly on reconstructing survival curves without generating adverse-event streams. KMGen combines automated KM extraction with mechanistic synthetic IPD generation, achieving close survival and demographic fidelity and recovering at least 71% of top-15 adverse events across held-out oncology trials under one fixed parameter set.

  • Problem

    Restricted access to clinical-trial IPD leaves downstream analyses dependent on aggregated statistics and prior reconstruction methods without a mechanism for generating trial-specific AE streams.

  • Method

    KMGen decouples LLM-based trial-record and patient-archetype extraction from deterministic sampling that couples survival to the empirical KM curve and schedules adverse events mechanistically.

  • Results

    Across three held-out oncology trials and 30 regenerations per trial, KMGen achieved mean integrated KM absolute difference Δ_KM ≤0.051 and top-15 AE overlap ≥71% under a single fixed parameter set.

  • Takeaways & Limitations

    KMGen provides synthetic IPD for methodological development when real patient-level data are restricted, with interpretable generation and marginal survival preservation by construction.

  • Takeaways & Limitations

    The shared scalar frailty model does not represent term-specific AE co-occurrence because registry records report marginal incidences rather than joint AE statistics.

Abstract

from arXiv · show

Individual patient data (IPD) from clinical trials is the substrate for survival modeling, meta-analysis, and safety research, yet IPD is rarely released. Prior work has addressed only half of this gap: reconstructing Kaplan-Meier (KM) curves from published plots -- typically requiring manual digitization or human-in-the-loop correction -- while offering no mechanism for generating the adverse-event (AE) streams that constitute the other half of a patient record. We introduce KMGen, the first end-to-end framework that (i) fully automates KM curve extraction at accuracy competitive with human-guided tools, and (ii) generates synthetic per-patient AE trajectories from public trial registry records. The extraction stage is a fully automated agentic pipeline -- an agent generates code to extract each step in the KM curve -- achieving a mean Integrated Absolute Error (IAE) of 0.0151 on a 32-plot benchmark spanning clean, edge-case, and adversarial conditions. The IPD generation stage decouples patient archetype extraction from statistical sampling: an LLM distills the trial record into arm-specific statistics, adverse events, patient demographics, and risk multipliers. A mechanistic sampler generates patient events via clinical archetypes, bootstrap rank-correlation coupling to the empirical KM curve (preserving the marginal survival distribution exactly), and cycle-based AE scheduling with an induction/maintenance split. Across three held-out oncology trials spanning an order of magnitude in cohort size and 30 independent regenerations per trial, KMGen achieves mean integrated KM absolute difference $Δ_{\text{KM}}\,{\leq}\,0.051$, sex/ECOG JSD ${\leq}\,0.013$ on 5 of 6 demographic slots, and recovers ${\geq}\,71\%$ of the top-15 AEs by exact MedDRA term under a single fixed parameter set. The pipeline is released as open source at https://github.com/chufangao/kmgen.

1. Introduction

KMGen addresses restricted access to clinical-trial IPD by unifying automated KM-curve extraction with synthetic per-patient data generation, including adverse-event trajectories. Its design separates LLM-based interpretation from deterministic computation to support auditable, reproducible outputs.

  • Motivation: Restricted IPD access forces downstream analyses to rely on aggregated statistics that obscure granular event timing and censoring patterns.These limitations affect applications including time-to-event modeling, machine learning, and meta-analysis.
  • Framework: KMGen unifies adaptive extraction of patient-level survival data from published KM plots with synthetic IPD generation from clinical trial records.The framework is designed for settings where reconstruction is infeasible or trial data must be simulated.
  • Framework: The generation pipeline uses structured trial information, patient archetypes, risk multipliers, bootstrap rank-correlation coupling, and cycle-based AE scheduling with induction–maintenance phases.Clinical priors are explicit, citable parameters rather than latent weights, supporting auditability.
  • Evaluation: KMGen evaluates extraction on 32 plots spanning clean, edge-case, and adversarial conditions, and evaluates generation on three oncology trials with held-out IPD.The evaluations cover both components of the unified framework.
  • Design principle: The paper’s central design principle assigns interpretive tasks to the multimodal LLM and computational tasks to deterministic code.This separation confines approximation to strategy selection and structured-field population while preserving exactness and reproducibility for computation.

2. Methods

KMGen combines adaptive Kaplan–Meier extraction with registry-to-IPD generation, using agent-guided plot analysis, mechanistic sampling, and empirical survival coupling. Its design preserves the marginal KM distribution exactly while imposing explicit assumptions and cohort-size boundaries on conditional structure.

  • KM extraction: The extraction pipeline splits plot interpretation between a multimodal LLM analyst and deterministic Python code tailored to each figure.The analyst identifies plot challenges and selects toolbox techniques; the extractor executes pixel-precise code, with localized corrections for residual artifacts.
  • KM extraction: Verification checks initial survival, patients-at-risk consistency, monotonicity, median annotations, and visual overlays before accepting an extraction.The agent self-corrects and re-extracts when overlays reveal drift or anomalies.
  • Evaluation: The extraction benchmark contains 32 plots spanning standard synthetic figures, published-figure edge cases, and single- and combination-degradation stress tests.Axis ranges span 12–144 months, and errors are normalized to [0, 1] before integration.
  • Evaluation: IAE compares extracted and truth arms after left-step interpolation onto merged x-grids, with trapezoidal integration and a range of [0, 1].Secondary metrics include Score, median pointwise absolute error, and median overall-survival error when the curve crosses 0.5.
  • IPD generation: The IPD pipeline extracts per-arm trial configurations, samples patient event streams, and evaluates both configurations and cohorts against held-out real IPD.Configurations include enrollment, treatment timing, demographics, empirical KM data, adverse events, and adjustable risk multipliers.
  • IPD generation: The sampling design couples bootstrap draws by permutation, preserving the marginal KM distribution exactly, while shared frailty induces positive within-patient AE correlation.The model does not represent term-specific AE co-occurrence beyond the shared scalar, and conditional survival structure becomes less estimable in small cohorts; a few dozen patients per arm are recommended.

3. Results

KMGen evaluates automated KM extraction and end-to-end synthetic IPD generation across adversarial plot benchmarks and three held-out oncology trials. It achieves low survival-curve error, strong demographic and AE fidelity, while revealing taxonomy, timing, and AE-burden limitations.

  • KM Extraction Results: 0.0151 mean IAE across 32 plots, with 22 plots below 0.015 and combination stress tests averaging 0.0283.The heavy-tailed error distribution is driven mainly by combination stress tests and a few outliers rather than broad under-performance.
  • KM Extraction Results: 0.0262 mean IAE across three real-trial arms, including 0.0133 for the n=430 Gemcitabine arm.The two smaller cohorts yielded higher errors: 0.0309 for Placebo n=53 and 0.0345 for E/P n=37.
  • End-to-End Fidelity: Top-15 AE overlap is ≥71% on every trial, and AE rank ordering agrees for dominant hematologic and gastrointestinal terms.Cosine similarity reaches 0.940±0.003 on the largest trial, while mean per-AE incidence difference decreases from 0.106 at nr=53 to 0.027 at nr=430.
  • Limitations: Race and region JSD equals 1.0 on all trials because ClinicalTrials.gov and sponsor IPD use non-overlapping label taxonomies.Synthetic onset timing also shows cycle-length striping and tighter concentration than real ascertainment, although mean onset error remains below 0.8 months.
  • Extraction Quality: Sex JSD is ≤0.006 on every trial, while ECOG JSD is ≤0.013 on two trials with complete {0, 1, 2} support.ECOG JSD reaches 0.272±0.048 on NCT03041311 because the real IPD excludes ECOG 0 while the extracted configuration permits it.
  • End-to-End Fidelity: Mean ∆KM is 0.049, 0.051, and 0.040 across the three trials, all below 0.06.The largest-trial Mann–Whitney rejection was associated with a 1.3-month median shift and greater statistical power.

4. Discussion

KMGen unifies automated KM-curve extraction with mechanistic synthetic IPD generation, producing interpretable cohorts while preserving survival marginals by construction. Evaluation supports methodological use across oncology trials, but validated non-oncology performance remains future work.

  • Framework: KMGen combines adaptive hybrid KM extraction with deterministic synthetic generation to address IPD accessibility.The framework separates interpretive LLM tasks from computational sampling and extraction tasks.
  • Mechanistic generation: Bootstrap rank-correlation coupling preserves the marginal survival distribution exactly, independent of the rank-noise parameter σ.This property was confirmed in ablation studies.
  • Mechanistic generation: Explicit clinical priors, including induction–maintenance scheduling and archetype risk multipliers, make the generation pipeline interpretable end-to-end.These priors are parameterized components rather than learned weights.
  • Scope: The framework claims architectural generality but has not validated performance outside oncology.Only the archetype risk multipliers and induction fraction are tuned to cytotoxic oncology; non-oncology validation is future work.
  • Evaluation: ΔKM ≤0.051 and top-15 AE overlap ≥71% across three oncology trials spanning 37 to 430 patients and 30 regenerations per trial.The evaluation used one fixed parameter set across the trials.
  • Algorithm: The algorithm takes a trial configuration as input and outputs a synthetic cohort containing demographics, overall survival, and adverse-event streams.Its stages instantiate patients, couple survival to empirical KM pairs, and schedule adverse events by cycle and phase.

Appendix B. Risk-Multiplier Evidence Base

Appendix B documents the clinical evidence and assumptions behind KMGen’s default adverse-event and survival risk multipliers. The multipliers are calibrated estimates because cited studies generally report overall effects rather than per-organ-system risks.

  • Multiplier construction: Per-SOC AE multipliers are clinical estimates informed by overall effect sizes and are relative to a baseline of 1.0.The cited evidence does not directly provide per-organ relative risks.
  • Patient factors: Age evidence includes associations with severe toxicity, cardiac risk, neuropathy risk, and renal insufficiency, but lacks consistent organ-system stratification.The cited studies include Hurria et al., Swain et al., Argyriou et al., and Launay-Vacher et al.
  • Patient factors: Sex evidence reports higher grade 3+ AE odds in women versus men, alongside reviewed sex-based pharmacokinetic and toxicity differences.Unger et al. report OR 1.34 for grade 3+ AEs in women versus men.
  • Patient factors: ECOG evidence reports higher severe nausea rates for performance status 2 than for PS 0–1, while other studies find ECOG nonsignificant or exclude it from final models.The evidence is therefore mixed across toxicity outcomes and analyses.
  • Patient factors: Comorbidity evidence covers chemotherapy receipt, cancer survival, and febrile-neutropenia mortality, but does not directly establish organ-specific relative risks.The appendix uses these findings to inform calibrated multipliers.
  • Overall survival: Age and comorbidity OS multipliers are inferred through toxicity–survival associations rather than reported survival ratios.The cited studies predict chemotherapy toxicity rather than overall survival directly.

Appendix C. Implementation Details

The implementation uses fixed LLM backbones for interpretation while deterministic routines handle numeric extraction, sampling, and data processing. Trial-specific loaders, term normalization, and a non-exercised Weibull fallback complete the implementation details.

  • Agent backbone: Both agents run on Claude Opus 4.6 with default decoding settings, and no controlled cross-model comparison is reported.The agents select techniques or populate typed configurations, but deterministic routines produce numeric outputs.
  • Data handling: The KM-extraction agent cannot access the IPD directory, enforcing separation between extraction and evaluation data.The discovery glob explicitly filters paths containing the IPD segment.
  • Real IPD ingestion: Trial-specific loaders ingest CDISC SDTM-AD domains, with NCT00844649 requiring custom demographic, ECOG, and survival assembly.ECOG is derived from Karnofsky performance status for that trial.
  • AE processing: AE term normalization lowercases text, collapses whitespace, strips punctuation, and converts British to American MedDRA spellings.Normalization is applied during both extraction and evaluation.
  • Survival input: When empirical KM data are unavailable, the sampler falls back to a parametric Weibull model, but this branch is not used in the main experiments.The fallback is documented as an implementation path rather than an evaluated experimental condition.

Appendix D. Ablation Studies

The ablations test sample size, induction timing, patient frailty, and archetype risk multipliers, showing that AE realism depends especially on calibrated timing and heterogeneity while survival fidelity remains stable.

  • Induction fraction α: 0.151 onset KS at α=0.85, versus 0.372 at α=0.50 and 0.192 at α=0.95, supports the induction–maintenance split.The U-shaped sweep supports α=0.80–0.85 as the default for cytotoxic regimens.
  • Patient propensity ση: 0.127 burden KS at ση=1.0 versus 0.308 without frailty, confirming that lognormal frailty is load-bearing for AE-burden dispersion.The default ση=0.4 is conservative, while higher dispersion narrows the burden gap.
  • Patient propensity ση: At ση=0.8, burden improves while ∆KM remains unchanged to four decimal places and AE cosine similarity moves by less than 0.01.This supports recommending ση=0.8 as the operating default.
  • Archetype risk multipliers: Removing all archetype multipliers preserves marginal KM fidelity but eliminates within-cohort heterogeneity in AE onset and burden.The full model produces an AE-burden distribution closer to real IPD.
  • Sample size sensitivity: AE frequency metrics improve as synthetic cohort size increases, while KM fidelity and onset timing show bootstrap-level variance rather than a monotone trend.AE JSD decreases from 0.192 to 0.173 and cosine similarity increases from 0.84 to 0.86 across 0.5×–5× enrollment.

E.1. Extraction-Error Propagation

The propagation analysis isolates plot-extraction error by comparing extracted-curve and oracle-curve regenerations while holding the remaining sampler fixed.

  • Extraction-error propagation: Thirty extracted and thirty oracle regenerations per matched arm quantify propagated error as the difference in mean ∆KM.Extracted curves come from published figures, whereas oracle curves are reconstructed from held-out real IPD.
  • Extraction-error propagation: 0.006–0.015 propagated ∆KM represents 12–32% of total ∆KM, and the sampler does not amplify upstream extraction error.The largest propagated value occurs for the smallest cohort; oracle inputs still have a finite-sample error floor.

E.2. AE-Burden Recalibration via ση

The AE-burden channel can be recalibrated independently of survival, and ση=0.8 is selected as a balanced setting across the three trials.

  • Recalibration result: At ση=0.8, synthetic mean burden approaches real burden from below on all three trials while burden KS improves by 31%, 25%, and 66% relative to default.The reported synthetic-versus-real means are 13.0 vs. 15.1, 9.5 vs. 12.8, and 14.8 vs. 15.4.
  • Recalibration result: ση=1.0 minimizes burden KS on two trials but overshoots mean burden, so the authors recommend ση=0.8 instead.The improvement over ση=0.8 is within one standard deviation on both trials.
  • Channel separation: Across all four dispersion settings, ∆KM is invariant and AE-frequency metrics move by less than 0.01, demonstrating isolated burden-channel tuning.Survival assignment does not depend on ση.
  • Additional fidelity checks: The truncated-Gaussian sampler reproduces real anthropometric distribution shapes, with KS statistics ≤0.24 for weight, height, and BMI across all three trials.The recalibration tables report demographic and survival rows within resampling noise of the ση=0.4 values at ση=0.8.

Appendix G. Full Extraction Gallery

The extraction gallery documents paired visual and numeric diagnostics across standard, edge-case, and adversarial KM plots, including successes and characteristic failure modes.

  • Gallery diagnostics: Each gallery figure overlays extracted step coordinates on the source plot and compares ground-truth and extracted step functions on a common axis.Captions report IAE, median AE, median OS error when defined, and arm count.
  • Plot categories: The benchmark includes clean standard plots, seven edge cases, and adversarial stress tests targeting blur, compression, overlap, dense steps, and other extraction challenges.The edge cases include four-arm comparisons, cumulative incidence, near-flat survival, and multi-panel layouts.
  • Successful cases: The gallery shows clean tracking for standard two-arm plots, four-arm comparisons, high-survival follow-up, and near-flat curves.Markers remain aligned through closely overlapping segments near 96–120 months in the high-survival example.

G.3. Single-Degradation Stress Tests (12)

The single-degradation tests examine how individual visual distortions affect KM extraction, including color ambiguity, image quality, plot layout, and resolution. Most conditions remain manageable, but dense multi-arm overlap makes color attribution the main bottleneck.

  • Three similarly hued arms are separated by HSL masking with only a narrow margin.
  • JPEG compression smears curve edges, but extraction remains within 1–2 px; overlapping legends are handled as rectangular exclusion zones.
  • Black-and-white plots retain solid-versus-dashed separation through grayscale intensity, while dashed gaps can be bridged.
  • Low pixel density increases absolute survival error from small mask errors, whereas blur widens edges symmetrically and is absorbed by centroid tracking.
  • Per-arm IAE ranges 0.042–0.072 on an 8-arm cumulative-incidence plot, where shared hue bands cause continuity-tracker arm switches in dense regions.Color attribution, rather than curve tracing alone, limits extraction when several arms occupy similar hue bands.

G.4. Combination Stress Tests (8)

Combination tests stack multiple visual stressors to probe failure interactions. Flat overlapping curves can make difficult conditions deceptively easy, while blur, tiny resolution, and compounded masking failures increase extraction difficulty.

  • Flat overlapping curves remain between 0.95–1.00 across 60 months, making pixel-assignment errors nearly indistinguishable in survival space.
  • Black-and-white overlapping solid and dashed curves are resolved using grayscale separation and gap bridging.
  • Four-arm tiny-resolution plots preserve color separation, but low per-column sample counts limit evidence for tracing.
  • Tiny blurry plots are dominated by edge widening and produce the highest IAE among the single-style combinations.
  • JPEG artifacts plus blur keep spatial extraction within a few pixels but make step timing noisier, while gridlines, annotations, and B&W stack three failure modes.

G.5. Best vs. Worst Case Analysis

Best- and worst-case analyses show that extraction error depends on how visual distortions interact with curve geometry and masking. The pipeline combines adaptive image handling, tracing constraints, and verification checks to manage these cases.

  • Best case: IAE 0.00118 ranks first of 32 for flat overlapping curves, because flatness and overlap make pixel assignment nearly irrelevant in survival space.
  • Worst case: IAE 0.07442 ranks last of 32 for gridline, annotation, and B&W stress, where three masking and attribution failures compound.The annotation leader line pulls the red mask off the curve near t = 14, producing a median OS error of 4.89 months.
  • Intermediate cases: IAE 0.054 on an 8-arm cumulative-incidence plot reflects hue-band collisions that cause continuity tracking to switch arms in dense overlap.
  • Intermediate cases: IAE 0.041 for JPEG-plus-blur preserves small spatial error but introduces noisy step-onset timing as the curve edge widens to about 4 px.
  • Pipeline response: The extraction workflow combines visual assessment, adaptive technique selection, image-specific code, monotonicity enforcement, and verification against anchors and patients-at-risk data.
  • Pipeline response: The OS CSV is treated as ground truth, so parametric survival fields are ignored when the empirical KM curve is supplied.

Appendix J. Limitations and Future Directions

The current implementation has limitations across adverse-event, demographic, and survival fidelity, plus missing configuration safeguards. The proposed fixes target burden calibration, label normalization, upstream curve reconstruction, and validation.

  • Limitations: Synthetic cohorts systematically generate 4–6 fewer adverse events per patient than real IPD.Increasing propensity variance from ση = 0.4 to 0.8 recovers most of the burden gap without changing survival or frequency channels.
  • Limitations: LLM-populated configurations are not range-checked before sampling, so invalid fractions or non-normalized categorical distributions can reach the sampler.
  • Limitations: Race and region distributions reach JSD = 1.0 because ClinicalTrials.gov and sponsor IPD use non-overlapping labels such as “White” and “Caucasian.”The passage attributes this to taxonomy mismatch rather than sampling error.
  • Limitations: Median overall survival error ranges from 7% to 19% across trials and is inherited from upstream KM reconstruction.Human-in-the-loop methods can reduce reconstruction error by using additional number-at-risk constraints.
  • Future directions: Future work prioritizes raising ση to 0.8, adding prevalence-to-rate correction, implementing configuration validation, and learning dispersion parameters.
Loading 2608.22618v1…