PaperlayerView exact revision on arXiv

Source-linked AI summary

SurvICL: a Foundation Model for in-context Survival Analysis

Julie Alberge, Linus Bleistein, Judith Abécassis, Marine Le Morvan, Gaël Varoquaux, David Holzmüller

arXiv:2610.22242v1stat.MLcs.LG

TL;DR

Survival prediction must learn from censored data, and existing survival TFMs have not surpassed Cox on small datasets. SurvICL uses in-context learning with synthetic survival pretraining and latent event-time quantile supervision. On 63 held-out datasets, it leads in IBS comparisons and ranks above Cox on small datasets.

  • Problem

    Existing survival foundation models do not beat Cox, despite the promise of tabular foundation models and the prevalence of small survival datasets.

    Source: §1 · pp. 1–2

  • Method

    SurvICL is pretrained on synthetic survival data and learns continuous event-time quantiles from uncensored latent targets while observing censored context in-context.

    Source: §1 · pp. 2–3, §4 · p. 5

  • Results

    SurvICL wins 82.9% of IBS comparisons against 71.3% for Cox on 63 held-out datasets and is the first model to rank above Cox on small datasets.

    Source: §1 · p. 2

  • Takeaways & Limitations

    On small clinical cohorts, SurvICL ranks above the long-standing Cox default without training or tuning and remains among the best-calibrated methods.

    Source: §7 · p. 9

    SurvICL addresses right-censored, single-event data; competing risks and time-varying covariates remain extensions.

    Source: §7 · p. 10

Abstract

from arXiv · show

Tabular foundation models (TFMs) now outperform gradient-boosted trees in classification and regression. Yet in survival analysis, which predicts when an event occurs from censored data, often on small clinical cohorts, we find that no existing TFM beats the half-century-old Cox model. We address this issue by introducing SurvICL, a TFM that predicts survival curves in-context. SurvICL is pretrained on purely synthetic datasets generated from a novel flexible proportional-odds prior that outperforms other prior types in our ablation study. Unlike most other survival models, SurvICL is trained with a plain quantile regression loss, which is possible because its synthetic pre-training data generator can provide uncensored event times. Additionally, we introduce the largest survival benchmark to date spanning 103 datasets. On the 63 datasets held out from all design decisions, SurvICL outperforms 20 tree-based, deep, and foundation-model baselines without any tuning: it wins 82.9\% of pairwise comparisons in integrated Brier score, against 71.3\% for the runner-up, the Cox model, and is the first model to rank above Cox on small datasets.

1 INTRODUCTION

Survival analysis must predict calibrated event-time probabilities from censored observations, often in small clinical cohorts where existing flexible models and survival foundation models have not surpassed Cox. SurvICL combines in-context prediction with latent event-time supervision, and leads a large held-out benchmark.

Motivation

  • Survival predictions must account for right-censoring and provide calibrated probabilities, not merely rankings, because they can inform decisions such as treatment choice.Discarding censored observations biases predictions, while treating them as observed events is incorrect.Source: §1 · p. 1
  • Survival cohorts are often small, and none of the evaluated gradient-boosted trees, deep survival networks, or existing survival foundation models outperforms Cox on the benchmark’s small datasets.More than 70% of the collected datasets have fewer than 1,000 samples.Source: §1 · p. 1

Scaling Evaluation of Survival Models

  • SurvICL wins 82.9% of held-out IBS comparisons against 71.3% for Cox across 63 datasets, while existing survival TFMs trail Cox.The benchmark also reports SurvICL as best in prediction accuracy and discrimination.Source: Figure 1 · p. 2, §1 · p. 2

Building a High-Performing TFM for Survival Analysis

  • SurvICL learns from censored context using latent uncensored event times as supervision, allowing a plain quantile-regression loss and continuous survival-curve predictions.Its predictions are continuous quantile functions from which a monotone survival function can be inferred.Source: §1 · p. 2

Scaling Evaluation of Survival Models

  • The benchmark covers 103 real-world datasets and 20 baselines, with design choices made on 40 development datasets and results reported on 63 held-out datasets.The held-out results show SurvICL above Cox on small datasets, while gradient boosting retains a slight edge on the largest datasets.Source: §1 · p. 2

Contributions

  • The contributions include a proportional-odds prior with a flexible multimodal baseline hazard that outperforms Cox, accelerated-failure-time, and existing TFM priors, plus event-only rather than censoring-distribution targets.The paper also contributes a benchmark of 103 datasets, including high-dimensional genomics cohorts and large registries.Source: §1 · pp. 2–3

2 RELATED WORK

Related work spans deep survival architectures and tabular foundation models, while survival TFMs adapt generic models, add survival heads, or pretrain on synthetic survival data. SurvICL builds on tabular-model advances while targeting calibrated survival prediction through in-context inference.

Deep Learning for Survival Analysis

  • Deep survival architectures accommodate varied data and event types, while this work tailors deep transformers and training procedures to tabular survival data.The cited examples include time series, competing events, images, and text.Source: §2 · p. 3

Deep Learning for Tabular Data

  • Tabular foundation models have challenged gradient-boosted trees, and SurvICL adapts their architectural advances for calibrated survival prediction without fine-tuning or weight updates at inference.The passage describes gradient-based decision-tree models as long dominant in tabular benchmarks.Source: §2 · p. 3

Tabular Foundation Models for Survival Analysis

  • Survival TFMs either reformulate survival for generic TFMs, train survival heads on their representations, or pretrain on synthetic survival data; existing synthetic-data models predict on discretized time grids.These approaches respectively inherit limits such as time discretization or fitted heads, while SurvICL outputs a continuous event-time quantile function.Source: §2 · p. 3

3 SURVIVAL ANALYSIS SETTINGS: CENSORED TARGETS, SMALL SAMPLES

Survival data contains potentially censored event times, so models must use partial observations while estimating survival probabilities across time. The design goals emphasize small samples and calibrated predictions without hyperparameter tuning.

3.1 CENSORED DATA AND SURVIVAL MODELS

  • A survival dataset contains i.i.d. feature, observed-time, and censoring-indicator triples, where a censored observation reveals only that the true event occurs later.The observed proxy time is the minimum of the true event time and censoring time.Source: §3.1 · pp. 3–4
  • A survival model estimates the survival function over time, from which an individual event-time prediction can be taken where the function first crosses 0.5.Hazard-based models learn the time-wise derivative of event probability and use numerical integration to obtain survival functions; discrete-time models instead bin time and frame prediction as classification.Source: §3.1 · p. 4

3.2 DESIGN GOALS FOR A SURVIVAL TFM

  • Small samples and calibrated probabilities are central design goals because survival datasets are often small and probability estimates inform decisions, while C-index alone does not assess calibration.The passage states that SurvICL is trained to provide calibrated predictions without hyperparameter tuning.Source: §3.2 · p. 4

4 SURVICL ARCHITECTURE: QUANTILES SUPERVISED BY LATENT TIMES

SurvICL predicts event-time quantiles from censored contexts while training against latent, uncensored event times. This allows a plain quantile-regression loss without a censored likelihood, censoring weights, or time discretization.

The TabICLv2 Backbone

  • SurvICL adapts the TabICLv2 architecture to take features, censoring indicators, and observed times as triplet inputs.It retains repeated feature grouping and target-aware embedding.Source: §4 · p. 4

Continuous Quantile Head

  • The model outputs 999 event-time quantiles at levels 1/1000 through 999/1000 and uses a multiquantile pinball loss.Predicted quantiles are sorted to ensure monotonicity except during pre-training; the paper identifies this loss as a major design choice distinguishing it from classification and hybrid losses.Source: §4 · pp. 4–5

Latent Time Supervision

  • SurvICL learns event-time distributions from censored contexts by predicting query points’ latent uncensored event times during pre-training.The survival-specific knowledge is supplied by the prior, allowing the regression loss to work without a censored likelihood, inverse-probability-of-censoring weights, or time discretization.Source: §4 · p. 5

Mean-Duration Scaling

  • Survival times are divided by the mean observed duration to accommodate datasets with different time ranges..Source: §4 · p. 5

Event-Only Targets

  • SurvICL predicts event-time quantiles rather than also predicting the censoring distribution, unlike the mixed-target approach ablated in the paper.The paper reports that the mixed prediction task degrades performance.Source: §4 · p. 5

5 PRETRAINING: SYNTHETIC DATA PRIORS WITH PROPORTIONAL-ODDS

SurvICL is pretrained on synthetic survival datasets built from a flexible proportional-odds prior. The prior generates diverse event and censoring times, while filtering removes highly censored or non-predictable datasets.

Proportional-Odds Prior

  • The synthetic prior uses proportional odds, allowing time-varying hazard ratios that may vanish over time, a pattern the paper says Cox priors cannot generate.The baseline hazard is combined with an individual effect on the odds scale.Source: §5 · p. 5
  • The prior adds a random number of positive Gaussian bumps at random times, each with its own width and height, to the smooth baseline trend.The number of bumps is sampled from 1 to 5.Source: §5 · p. 5
  • Event times are generated by applying a random feature function to each individual and inverting the resulting cumulative distribution function.Setting η(xi) to zero recovers the baseline hazard, while positive values increase event odds at any time.Source: §5 · p. 6

Generating Censoring Times

  • Censoring combines a shared end-of-study cutoff in half of datasets with a separately generated censoring-time distribution; observed times are the minimum of event and censoring times.Some individual times are moved to a common grid with low probability to introduce ties.Source: §5 · p. 6

Filtering

  • The pipeline removes datasets with more than 95% censored samples and those where predicting mean survival time beats an ExtraTrees-based survival model on IBS.The filtering rule is adapted from Qu et al. (2026), and retained pretraining datasets carry exploitable signal.Source: §5 · p. 6

Training

  • Training splits each generated dataset into context and query sets, computes loss on query points only, and varies context size during pretraining.The setup uses stage 1 for 200K steps, taking around 72 hours on two H100 GPUs.Source: §5 · p. 6

6 LARGE SCALE EVALUATION ON 63 DATASETS

Across 63 held-out datasets, SurvICL leads overall and ranks above Cox on small datasets, while SurvivalBoost leads IBS on larger datasets. The evaluation compares survival models across four metrics under a held-out protocol.

6.1 SURVIVAL METRICS

  • The evaluation uses IBS, C-index at the median training-follow-up horizon, MAE-Hinge, and KM-calibration to assess probability accuracy, discrimination, time error, and marginal calibration.IBS assesses predicted probabilities, while the truncated C-index uses comparable pairs with observed events before the median horizon.Source: §6.1 · pp. 6–7

6.2 BASELINES: FROM COX TO FOUNDATION MODELS

  • Baselines span Cox and Kaplan–Meier, five survival deep networks, tree-based survival methods, and four open-weight survival foundation models.The Kaplan–Meier estimator does not use covariates.Source: §6.2 · p. 7

6.3 DATASETS AND HOLD-OUT PROTOCOL

  • The benchmark partitions 103 datasets into 40 development and 63 held-out datasets; each method is evaluated on ten event-stratified 80/20 splits.SurvICL uses training data as context, capped at 4,096 observations, without fine-tuning or dataset-specific hyperparameter tuning.Source: §6.3 · pp. 7–8

6.4 RESULTS: SURVICL IS BEST OVERALL

  • SurvICL wins 82.9% of pairwise IBS comparisons on 63 held-out datasets, versus 71.3% for Cox, and leads C-index win rate at 77.7% versus 71.6% for random survival forests.The IBS difference is statistically significant; the C-index win-rate comparison is also reported, with the cited passage not specifying significance.Source: §6.4 · p. 8
  • On datasets with fewer than 1,000 samples, SurvICL wins IBS comparisons against Cox 84.9% to 77.2% and matches Cox in C-index, without statistically significant differences.These datasets constitute 69.8% of the held-out benchmark.Source: §6.4 · p. 8
  • Above 1,000 samples, SurvivalBoost leads IBS with an 86.9% win rate, while SurvICL ranks second in average IBS and first in C-index at the 50th quantile.SurvICL wins fewer pairwise comparisons in this size regime than on small datasets.Source: §6.4 · p. 8
  • SurvICL lies on the performance-runtime Pareto frontier across dataset-size regimes, and its average KM-calibration rank of 6.0 trails four methods.For MAE, it is close to Survival-IC and slightly behind SurvivalPFN, without a statistically significant difference.Source: §6.4 · p. 9
  • The development-set ablation favors the proportional-odds prior over Cox, mixed, and other TFM priors; adding censoring-time prediction reduces performance.Administrative censoring and time discretization have limited effects at this scale; the final model uses 999 quantiles despite a small ablation advantage for 99.Source: §6.4 · p. 9

7 DISCUSSION AND CONCLUSION

SurvICL uses a continuous quantile-function output and a flexible proportional-odds prior, and is reported to rank above Cox on small clinical cohorts without training or tuning. Its scope is right-censored, single-event data; gradient boosting remains slightly stronger on the largest datasets.

Designing Efficient Survival TFMs

  • SurvICL is the first survival TFM to output a continuous quantile function, using latent-time supervision and a proportional-odds prior with a flexible, multi-modal baseline hazard.The paper reports that its prior outperforms SurvPFN, Cox, and AFT priors in ablations, and that SurvICL is the first comparison of all four survival TFMs on 63 datasets.Source: §7 · p. 9

Implications

  • On small clinical cohorts, SurvICL is the first model reported to rank above Cox without training or tuning, while remaining among the best-calibrated methods.The discussion frames this as a potential credible default for survival analysis.Source: §7 · p. 9

Limitations and outlook

  • On the largest datasets, gradient boosting remains slightly stronger; SurvICL covers right-censored, single-event data, with competing risks and time-varying covariates identified as extensions.The authors also note that longer contexts or light fine-tuning are called for on the largest datasets.Source: §7 · p. 10

AI USE STATEMENT

The authors report using generative AI for synthetic data, methodology and experiments, implementation, preliminary plots, and information sourcing, while retaining responsibility for the final work. They state that code, preprocessing files, and checkpoints would be released in a few weeks.

AI USE STATEMENT

  • Generative AI was used to generate synthetic datasets, contribute to methodology and experiments, implement methods, draft preliminary plots, and source information.The authors say it was not used to develop theoretical models or conceptual frameworks, formulate mathematical claims, translate, or interpret results.Source: AI USE STATEMENT · p. 11
  • The authors reviewed all AI-assisted work, proofread the paper, and take responsibility for its final content and artifacts.Source: AI USE STATEMENT · p. 11

REPRODUCIBILITY STATEMENT

A.1 ARCHITECTURE

SurvICL adapts TabICLv2 for in-context survival prediction, representing censoring and observed times in context and predicting positive event-time quantiles. The accompanying appendices describe pretraining, generated-data examples, baseline families, and tuning procedures.

A.1 ARCHITECTURE

  • TabICLv2 processes context rows through target-aware and column-wise embeddings, row-wise interactions, and an in-context transformer; its regression head predicts 999 quantiles.SurvICL adapts the backbone to survival inputs and preserves its stated runtime complexity of O(n^2 + nm^2).Source: §A.1 · p. 17
  • SurvICL normalizes times using context-set mean duration, embeds observed time with the event indicator, and predicts positive quantiles that are sorted before estimating the survival function.Survival outcomes are supplied only for context rows.Source: §A.1 · pp. 17–18

A.2 PRETRAINING

  • Pretraining omits later scaling stages, runs for 200K steps, uses 1,024-sample synthetic datasets, and lowers the context-fraction bound to 0.03 to cover smaller benchmark datasets.Training is conducted in 32-bit floats to avoid potential instabilities.Source: §A.2 · p. 18

A.3 EXAMPLE OF GENERATED DATASETS

  • The generated examples use the final proportional-odds prior with administrative censoring and time discretization, showing observed-event and censoring histograms alongside Kaplan-Meier curves.Source: §A.3 · p. 18, Figure 5 · p. 19

B.1.1 USUAL METHODS

  • The usual-method baselines include Kaplan-Meier, a covariate-independent marginal survival estimator, and Cox, a semiparametric model with a proportional-hazards assumption.The passage identifies Kaplan-Meier and Cox as the two most-used methods in medical settings.Source: §B.1.1 · p. 19

B.1.2 TREE-BASED MODELS

  • Tree-based baselines adapt survival trees and boosting to right-censored data, including random survival forests, two gradient-boosting variants, and SurvivalBoost.Source: §B.1.2 · pp. 19–20

B.1.3 DEEP NEURAL NETWORKS

  • Deep baselines include neural survival models such as DeepHit, DeepSurv, DSM, SuMo-Net, and several TabSurv variants.The passage describes deep models as capturing nonlinear covariate-survival relationships, often dropping the proportional-hazards assumption.Source: §B.1.3 · p. 20

B.1.4 TABULAR FOUNDATION MODELS

  • The benchmark includes synthetic-data tabular foundation models, including SurvPFN, SurvivalPFN, Survival-IC, and TFM4Survival; baseline search spaces are tuned with 50 randomized iterations.TFMs are described as generalizing to new datasets in a zero-shot or in-context manner.Source: §B.1.4 · p. 20, §B.2 · p. 21

B.3 ISSUES FOR SOME METHODS

Some baselines did not complete tuning or fitting on a small subset of datasets, and the benchmark assigns metric-specific scores to those cases. The appendix also provides overviews of the development and held-out datasets.

B.3 ISSUES FOR SOME METHODS

  • Some baselines failed to complete tuning or fitting on a small subset of datasets because of method-intrinsic issues; those cases receive IBS 1 or C-index 0.5 for full 63-dataset comparisons.Affected methods are marked in critical-difference diagrams and win-rate figures, and Table 5 lists affected method-dataset pairs and causes.Source: §B.3 · p. 23

B.4 DETAILS ON DATASETS

C ABLATIONS

The ablations vary one design choice at a time against a proportional-odds control, then rank arms by mean normalized IBS on frozen, held-out validation datasets. The appendix presents metric visualizations across all, small, and large held-out datasets, but these passages do not state comparative outcomes.

C ABLATIONS

  • Each ablation shares a nanotabiclexsurvival backbone trained for 10,000 steps, with one variable changed against a proportional-odds control; one arm trains in 2 hours on 2 A100s.The backbone is shared across every arm.Source: C ABLATIONS · p. 26, Table 8 · p. 26

C.0.1 ABLATION DESIGN

  • The design tests survival-family and recipe axes; seven recipe arms flip one flag while holding generation type fixed, and two control-off flags are instead turned on.The family-axis table lists each tested family’s generative equation; the recipe table lists the one-at-a-time changes.Source: §C.0.1 · p. 26, Table 9 · p. 26, Table 10 · p. 27

C.0.2 EVALUATION PROTOCOL

  • Arms are evaluated at their final checkpoint on frozen held-out validation datasets, using per-dataset min–max-normalized metrics and mean IBS as the primary ranking metric.Normalization maps the best arm to 100 and the worst to 0 on each dataset.Source: §C.0.2 · p. 27

D.1.1 IBS

  • The all-dataset IBS figures show individual dataset scores relative to Kaplan–Meier, rank distributions, pairwise win rates, and normalized IBS versus fit-plus-predict time.The captions identify the plots but do not report which method performs best.Source: Figure 6 · p. 27, §D.1.1 · p. 27, Figure 7 · p. 28

D.1.2 C-INDEX50

  • All-dataset C-index-at-the-50th-quantile figures compare scores with Kaplan–Meier and show rank distributions, pairwise win rates, and mean C-index versus fit-plus-predict time.The passages specify the plot contents, not comparative outcomes.Source: Figure 10 · p. 30, §D.1.2 · p. 30, Figure 11 · p. 30

D.1.3 KM-CALIBRATION

  • All-dataset KM-calibration and MAE figures show dataset-level scores, rank distributions, pairwise win rates, and metric-versus-time Pareto fronts.The captions define lower KM-calibration as better; they do not state method winners.Source: §D.1.3 · p. 31, Figure 14 · p. 32, Figure 15 · p. 32

D.2.1 IBS

  • For small held-out datasets, the figures cover IBS and C-index through dataset-level scores, rank distributions, pairwise wins, and time comparisons, plus KM-calibration and MAE plots.The captions describe plotted measures and encodings, without giving comparative outcomes.Source: §D.2.1 · p. 35, Figure 21 · p. 36, Figure 22 · p. 36

D.3.1 IBS

  • For large held-out datasets, the figures likewise cover IBS and C-index alongside KM-calibration and MAE, with score distributions, ranks, pairwise wins, and metric-versus-time plots.The captions specify lower-is-better for KM-calibration and MAE, but do not identify winners.Source: §D.3.1 · p. 43, Figure 36 · p. 44, Figure 37 · p. 44
Loading 2610.22242v1…