PaperlayerView exact revision on arXiv

Source-linked AI summary

StochIKM: Automated High-Fidelity Reconstruction of Individual Patient Data from Published Kaplan-Meier Curves

Liangze Ke, Daniel Zelterman, Wei Wei, Guannan Gong

arXiv:2610.29122v2cs.AIstat.AP

TL;DR

Published Kaplan–Meier curves offer a route to time-to-event IPD, but existing reconstruction methods use inconsistent fidelity criteria and return only one dataset despite ambiguity. StochIKM imposes explicit constraints and returns admissible datasets; in published-arm comparisons, it reproduced risk tables and medians more closely than LLM-improved IPDfromKM.

  • Problem

    Reconstruction methods use inconsistent fidelity criteria, while Kaplan–Meier figures do not uniquely identify patient records and IPD are rarely accessible.

    Source: §1 · p. 1

  • Method

    StochIKM uses a vision-capable LLM to digitize the figure and generates a set of IPD constrained by the risk table, curve, and reported summaries.

    Source: §1 · p. 2

  • Results

    StochIKM reproduced the risk table in every arm versus 44 for LLM-improved IPDfromKM, and median error in the median survival was 0.00 versus 0.10 months.

    Source: §3 · p. 5

  • Takeaways & Limitations

    The spread across admissible datasets measures what the publication left undetermined and can be propagated into downstream analysis.

    Source: §1 · p. 2, §5 · p. 6

    StochIKM cannot reliably use censoring marks, which can contain information about patient-level censoring times.

    Source: §5 · p. 6

Abstract

from arXiv · show

Individual patient data are the gold standard for secondary analysis of time-to-event evidence, but they are rarely released, and reconstruction from a published Kaplan-Meier figure could help. Existing methods all claim high fidelity, each judged by whichever agreements it happens to achieve, and each returns a single dataset although the map from figure to data is not bijective. We define fidelity explicitly: a dataset is admissible when it reproduces the risk table exactly, follows the digitized curve to within the image resolution, and respects the printed median, interval and event count. We propose StochIKM, a stochastic algorithm that digitizes the figure with a vision-capable language model and uses the high-fidelity definition as constraints, returning an admissible set of individual patient datasets. A solution is guaranteed to exist whenever the figure has been read correctly, so a failure indicts either the published figure or its digitization. In simulation and in 79 arms from published oncology trials, StochIKM reproduces the published risk tables and medians more closely than LLM-improved IPDfromKM, a high-impact alternative.

1 Introduction

Published time-to-event IPD supports a range of secondary analyses but is rarely accessible. StochIKM addresses inconsistent fidelity criteria, non-unique reconstructions, and digitization challenges by defining constraints and returning a set of admissible datasets.

1 Introduction

  • IPD enable meta-analysis, assessment of modeling assumptions, sensitivity analysis, extrapolation, hypothesis generation, and emulation of unstudied treatment comparisons, but confidentiality and proprietary concerns limit access.Even where data-sharing platforms exist, requests are fulfilled for only a small number of applicants.Source: §1 · p. 1
  • Improved reporting standards and vision-capable LLMs ease figure extraction, but do not resolve the statistical ambiguity of reconstructing patient records from aggregate survival data.Earlier curve-reconstruction methods include Parmar et al. (1998), Tierney et al. (2007), Guyot et al. (2012), and Liu et al. (2021).Source: §1 · pp. 1–2
  • Existing reconstruction studies assess fidelity using inconsistent criteria, and some reconstructed datasets fail to match the published risk table.Prior checks include the curve and hazard ratio, reported events and medians, or synthetic ground truth.Source: §1 · p. 2
  • A published Kaplan–Meier curve does not uniquely identify patient records: sparse tables, overlapping or missing censoring marks, and finite image resolution permit multiple event and censoring configurations.Prior methods also do not use printed median confidence intervals as constraints or return more than one dataset.Source: §1 · p. 2
  • StochIKM enforces exact risk-table agreement, curve deviation within a prespecified bound, and agreement with reported events, median, and interval when available, returning a set of admissible datasets.Under internally consistent reporting, an error-free figure, and exact extraction, the authors establish that at least one such dataset exists.Source: §1 · p. 2

Step 1: Digitization

StochIKM uses a vision-capable LLM to digitize a Kaplan–Meier figure and separates directly checkable printed values from image-read curve information. The latter is bounded by the figure’s pixel resolution.

Step 1: Digitization

  • A vision-capable LLM automatically digitizes the Kaplan–Meier figure and accompanying risk table for StochIKM’s reconstruction.Printed values are treated separately from elements that require image interpretation.Source: Step 1: Digitization · p. 2
  • Asterisked inputs are directly checkable printed values: risk-table counts, median survival and its interval, and the event count.The paper treats these values as containing no digitization error.Source: Step 1: Digitization · p. 2
  • Image-read inputs include plot coordinates, the traced curve and its drops; their precision is limited by pixel resolution.Drop timing is localized to about one pixel column and curve height to about half the drawn-line width; censoring marks are not used.Source: Step 1: Digitization · p. 2

Step 2: Reconstruction of admissible IPD

StochIKM constructs patient records interval by interval and retains only datasets that satisfy its fidelity constraints. Under perfect digitization and correct transcription, the original data satisfy those constraints, so the admissible set is nonempty.

Step 2: Reconstruction of admissible IPD

  • An admissible reconstruction matches the reported risk table exactly, follows the digitized curve within its tolerance, and respects printed summaries when available.The summary constraint covers event count, median survival, and its interval; the curve constraint also requires each event to lie within the pixel-scale window of its observed drop.Source: Step 2: Reconstruction of admissible IPD · p. 2
  • The algorithm advances through risk-table intervals, carrying forward the at-risk count and survival while choosing event and censoring counts that sum to interval attrition.The interval attrition is obtained from the reported risk table.Source: Step 2: Reconstruction of admissible IPD · p. 3
  • Censoring times are sampled uniformly within an interval, while events at a visible curve drop are placed within a time window bounded by image resolution.A reference time determines which censorings precede the events in a fine interval.Source: Step 2: Reconstruction of admissible IPD · p. 3
  • Given censoring placements, the algorithm inverts the Kaplan–Meier product one fine interval at a time to determine event counts.The paper describes this step as deterministic once censorings are given.Source: Step 2: Reconstruction of admissible IPD · p. 4
  • The algorithm checks the constraints as they become decidable, retries a failed interval, and continues runs until it has collected the requested number of admissible datasets.After repeated failures, it discards and restarts the run from the first risk-table interval.Source: Step 2: Reconstruction of admissible IPD · p. 4
  • With correctly transcribed risk tables and summaries and curve readings within the stated image-resolution bound, the original trial data satisfy the constraints and the admissible set is nonempty.The proposition assumes perfect digitization and absorbs rounding of printed summaries within the stated tolerance.Source: Step 2: Reconstruction of admissible IPD · p. 4

Step 3: Relaxation

Step 3 relaxes selected fidelity constraints when digitization errors or a difficult search leave no admissible dataset, while the risk-table constraint is never relaxed. Relaxation follows fixed levels and can still end with the figure declared irreconcilable.

Step 3: Relaxation

  • The number-of-events constraint most commonly causes failure, so its tolerance is relaxed separately from the other second-order conditions.The relative event-count tolerance is 0.035; it was set to the largest discrepancy observed in the case study, 9 of 262 events.Source: Step 3: Relaxation · p. 4
  • When no admissible dataset is found after R runs, the algorithm identifies the constraint responsible for most failures, relaxes it by one preset level, and restarts the search.The second-order constraint’s parts are relaxed separately, and the same order is used for every figure.Source: Step 3: Relaxation · p. 4
  • Median and interval conditions are progressively loosened, while curve tolerance increases in preset steps up to a ceiling.The median and interval sequence eventually drops the interval condition and then the median condition; curve tolerance is multiplied by 1.5 at each level.Source: Step 3: Relaxation · p. 4
  • The risk-table constraint is never relaxed; if every relaxation level is exhausted without an admissible dataset, the figure is declared irreconcilable and no IPD is returned.Reconstructions obtained after relaxation are reported with the levels used.Source: Step 3: Relaxation · p. 4

3 Case Study

Across the 79 published arms with comparable medians, StochIKM reproduced every risk table and had lower median and maximum median-survival errors than LLM-improved IPDfromKM. Results also show that admissible datasets could be returned for nearly all traceable arms, though figure ambiguity and stricter constraints limited coverage.

3 Case Study

  • In the illustrated trial, StochIKM’s 50 curves form a narrow band following the published curve, while LLM-improved IPDfromKM’s single curve takes fewer, larger steps.StochIKM reproduced both arms’ risk tables in all 50 datasets; in one arm, every dataset also carried the 247 printed deaths.Source: §3 · pp. 4–5
  • StochIKM returned at least one admissible dataset for all 104 traceable arms, and the full set of 50 for 97; 15 of 101 traceable arms instead had an empty admissible set.The 15 empty sets were attributed to figure quality, LLM tracing, or both.Source: §3 · p. 5
  • Four arms could not be traced because two pairs in one figure shared colors and differed only by line style.The corresponding figure was NCT02545504.Source: §3 · p. 5
  • Among 101 arms whose curves could be traced, StochIKM returned 50 admissible IPD for 86 arms; 55 arms met the full criterion, while 42 needed relaxation.The accepted dataset was the one whose median was closest to the published value (50 datasets per arm; fewer in 7 arms).Source: §3 · pp. 5–6
  • Every risk table was reproduced by StochIKM across 79 arms, versus 33 by LLM-improved IPDfromKM; median-survival errors were 0.01 versus 0.13 month, and maximum errors were 0.91 versus 5.26 months.Across 61 arms reporting confidence-interval bounds, median interval overlap was 0.93 for StochIKM and 0.92 for LLM-improved IPDfromKM.Source: §3 · pp. 5–6, Table L · p. 6

4 Simulation

In simulations with perfectly read figures, StochIKM recovered the generating data exactly across every scenario, while IPDfromKM was close but not exact. The experiments varied sample size, censoring, and risk-table intervals across 1,000 replicates per scenario.

4 Simulation

  • StochIKM returned the data exactly in every replicate of every scenario, with zero curve bias and empirical SE and exact recovery of the risk table, event count, median, and confidence interval.Each scenario had 1,000 replicates.Source: §4 · p. 6
  • IPDfromKM reproduced the risk table in every replicate, but exact event-count recovery ranged from 4% to 97%, and median error was 0.01 to 0.10 month.Its reconstructed confidence intervals overlapped the original intervals by 0.98 to 0.99.Source: §4 · p. 6
  • The simulation varied sample size between 50 and 1,000, censoring between 10% and 50%, and the number of risk-table intervals between 4 and 12, with 1,000 replicates per scenario.The baseline used 350 patients, 30% censoring, and 8 intervals.Source: §4 · p. 6

5 Discussion

StochIKM defines high fidelity through joint constraints and returns a portfolio of admissible datasets, reflecting what a published figure leaves undetermined. Across 97 published arms, it reproduced every risk table and matched published medians within one month; its limitations lie in reading censoring marks.

5 Discussion

  • Admissible reconstructions reproduce risk tables exactly, follow the digitized curve within image resolution, and respect the printed summaries; their spread reflects what the publication leaves undetermined.StochIKM returns a portfolio rather than a single dataset.Source: §5 · p. 6
  • Across 97 published arms, StochIKM reproduced every risk table and matched published medians within one month.Proposition 1 shows that an admissible set exists when the figure is traced exactly.Source: §5 · p. 6
  • StochIKM cannot use censoring marks, which can determine censoring times that the K–M curve and risk table do not specify.The available vision-language models cannot yet read these marks reliably.Source: §5 · p. 6
  • A figure whose reading cannot be reconciled with its printed summaries can flag a publication that no patient data could have produced.The paper connects this possibility to systematic screening of trials for fabricated data.Source: §5 · p. 6
  • Across every dataset a figure permits, the question becomes how much of a conclusion survives.The paper presents a portfolio of admissible datasets as a natural input to Bayesian borrowing from historical controls.Source: §5 · pp. 6–7
Loading 2610.29122v2…