Source-linked AI summary
Adaptation Interfaces for In-Context Tabular Foundation Models in Time-to-Event Prediction
Minh-Khoi Pham, Luca Cotugno, Dan Cernei, Alina Sirbu, Stefano Masi, Giuseppe Prencipe, Alessandro Pingitore, Patrizia Landi, Working Group on Uric Acid, Cardiovascular Risk of the Italian Society of Hypertension, Tai Tan Mai, Martin Crane, Marija Bezbradica
TL;DR
Generic TabFMs are not designed for censored time-to-event prediction, where targets depend on censoring, event time, and competing risks. This paper evaluates zero-shot, classification fine-tuning, and survival-head interfaces over single- and competing-risk benchmarks using frozen backbones. Zero-shot works best in smaller single-risk settings, while supervised adaptation gains value with scale and Cox is the most consistently strong single-risk interface, especially for IBS.
Problem
Generic TabFM interfaces assume standard classification or regression targets, leaving adaptation to censored, time-dependent, and competing-risk prediction insufficiently established.
Method
The study compares temporal zero-shot reformulation, censoring-aware classification fine-tuning, and CoxPH, DeepHit, and MTLR survival-head adaptation with frozen TabFM backbones.
Results
Zero-shot inference is most competitive on smaller data sets, supervised adaptation becomes increasingly useful as data sets grow, and Cox is the most consistently strong single-risk interface, particularly for IBS.
Takeaways & Limitations
Effective TabFM transfer depends on the data regime and on the statistical structure represented by the chosen adaptation interface.
Takeaways & Limitations
The competing-risk benchmark is much smaller and limited to survival-head comparisons, while data-size comparisons do not isolate sample size from other dataset characteristics.
Abstract
from arXiv · showhide
Tabular foundation models (TabFMs) achieve strong performance on structured data, particularly for standard classification and regression problems. Yet, extending them to censored time-to-event prediction is challenging because it requires properly handling censoring and event-time dynamics. Building on our prior work, we further link TabFMs with CoxPH and DeepHit and revise the context-resampled training procedure. We evaluate temporal zero-shot reformulation, classification-based fine-tuning, and survival-head adaptation using frozen TabFM backbones on 74 single-risk data sets, and we additionally study 4 competing-risk data sets. Zero-shot inference is effective on smaller single-risk data sets, whereas supervised adaptation becomes increasingly advantageous as data sets scale. Cox provides the most reliably strong interface, especially for Integrated Brier Score (IBS) on larger data sets. DeepHit is relatively stronger for the time-dependent Concordance Index than for IBS, while cause-specific MTLR ranks highest among the TabFM survival heads in the four-data-set competing-risk analysis. Classification fine-tuning becomes more competitive with zero-shot inference as data sets grow but remains weaker for probabilistic prediction. Overall, our results indicate that effective TabFM transfer depends on the data regime and on the statistical structure represented by the chosen adaptation interface. The implementation scripts used for this work are available at https://github.com/kaylode/survival-fm.
1 Introduction
The paper frames censored time-to-event prediction as an adaptation-interface problem for generic TabFMs, whose standard classification and regression interfaces do not directly accommodate censoring, time dependence, or competing risks. It compares zero-shot reformulation, classification fine-tuning, and survival-head adaptation to identify effective transfer strategies.
- Motivation: Generic TabFM interfaces do not directly accommodate censored outcomes, time-dependent targets, or multiple mutually exclusive event types.The resulting challenge concerns how pretrained priors should be adapted to downstream tasks with different statistical structure.
- Approach: The study systematically compares zero-shot reformulation, censoring-aware classification fine-tuning, and survival-head adaptation across a broad multi-domain benchmark.The supervised approaches retain frozen TabFM backbones while adapting task-specific heads.
- Approach: Survival-head adaptation extends the previously studied MTLR interface to CoxPH and DeepHit under context-resampled, context-conditioned training.The heads are compared for discrimination and probabilistic prediction across data regimes, with MTLR retained as a reference.
- Competing risks: The competing-risk analysis compares survival-head interfaces and distinguishes cause-specific adaptation from jointly normalized cause–time modeling.This tests whether conclusions from single-risk transfer persist when multiple event types are present.
2 Related Work
Related work spans classical and deep survival methods, tabular foundation models with different pretraining and inference designs, and routes for adapting generic TabFMs to censored prediction. This paper studies downstream interface choice while retaining frozen generic backbones rather than redesigning pretraining or requiring survival-native foundation models.
- Survival analysis: Classical survival methods include Kaplan–Meier estimation, CoxPH, and tree-based models, while deep methods broaden modeling of censored event-time distributions.Examples include DeepSurv, MTLR, DeepHit, and continuous- and discrete-time extensions.
- Tabular foundation models: TabFMs replace separate per-dataset fitting with broad pretraining followed by in-context prediction.The benchmark uses TabPFN, TabDPT, and TabICL, which differ in synthetic pretraining, real-data retrieval, and scalable row- and column-processing designs.
- Tabular foundation models: TabFM advantages are strongest on tiny- to medium-sized IID problems, while tree ensembles and supervised deep models can regain an advantage on non-IID, large, or high-dimensional data.This motivates examining transfer across data regimes rather than assuming universal superiority of foundation models.
- Adaptation interfaces: One prior route reformulates survival prediction as censoring-aware binary classification so existing TabFMs can use their pretrained classification interface.This route inherits backbone-specific restrictions and depends on horizon-wise formulation choices.
- Adaptation interfaces: The present work instead studies downstream transfer through different interfaces, extending earlier frozen-backbone MTLR adaptation with additional survival heads.It retains generic pretrained TabFMs rather than redesigning pretraining distributions or requiring survival-native foundation models.
3 Methods
The paper compares three transfer regimes for generic pretrained TabFMs: zero-shot temporal reformulation, classification fine-tuning, and survival-head adaptation with frozen backbones. It uses context-conditioned training and adaptive temporal discretization to transfer TabFMs to censored and competing-risk prediction.
- Transfer regimes: The benchmark compares zero-shot in-context reformulation, temporal classification fine-tuning, and CoxPH, DeepHit, or MTLR survival-head adaptation.In supervised regimes, only the task-specific head is optimized while the pretrained TabFM backbone remains frozen.
- Evaluation and context separation: Nested cross-validation separates training, validation, and test folds while preventing validation or test observations from entering support contexts.Support contexts are sampled only from the training set during fine-tuning and inference.
- Context-conditioned training: Context-resampled training pairs each query minibatch with a sampled support context, making downstream heads robust to variation in in-context demonstrations.The query minibatch is excluded from its paired context, and representations can vary across resampled supports even with frozen backbone parameters.
- Interface design: For survival-head contexts, the pretrained backbone receives only the binary observed-event indicator; event time, censoring, and cause identity enter through the downstream survival objective.In competing-risk settings, all observed causes map to an observed-event context label, and the survival head introduces cause identity.
- Interface design: Survival-head adaptation uses subject representations, whereas classification interfaces use temporally expanded subject–horizon queries with temporal features.Classification targets indicate whether an observed event has occurred by a queried horizon, while survival-head contexts use subject-level observed-event indicators.
- Competing-risk survival head: Competing-risk DeepHit uses a joint softmax for cause–time probabilities and a joint likelihood without the single-risk pairwise ranking term.The implementation therefore differs from single-risk DeepHit in both its joint output structure and its loss composition.
4 Experimental Setup
The benchmark spans diverse public tabular survival data sets and evaluates frozen TabFM backbones with zero-shot, classification, and survival-head interfaces against neural and classical baselines. Performance is assessed with time-dependent Ctd and IBS under standardized preprocessing and cross-validation.
- Data sets: The benchmark covers public tabular survival data sets spanning diverse domains, sample sizes, censoring rates, and event structures.The single-risk benchmark uses SurvSet data sets, while the competing-risk benchmark includes SUPPORT2-CR, FRAMINGHAM, PBC2, and SYNTHETIC.
- Preprocessing: PCA is applied before backbone inference only when input dimensionality exceeds a TabFM interface’s native feature capacity.PCA is fitted within each outer training fold, with retained components also bounded by available training observations.
- Models: The evaluation compares TabPFN, TabDPT, and TabICL with non-pretrained neural and classical survival baselines using frozen supervised TabFM variants.Classification fine-tuning is denoted -CE, while survival-head variants are denoted -Cox, -DH, and -MTLR.
- Evaluation protocol: All methods use 5-fold cross-validation and fixed seeds, while non-pretrained baselines receive model-specific 20-trial Optuna searches within each training fold.TabFM variants use fixed settings without hyperparameter search.
- Metrics: Ctd measures discrimination, whereas IBS measures probabilistic prediction accuracy through time-integrated, censoring-weighted squared error.IBS is sensitive to both calibration and discrimination and should not be interpreted as calibration-only.
5 Results
Results vary with adaptation interface, metric, and data-set size. Cox is especially strong for IBS and large single-risk data sets, whereas cause-specific MTLR ranks highest among TabFM survival heads in the limited competing-risk benchmark.
- 5.1 Adaptation and Backbone Effects: Changing the survival head or replacing a non-pretrained backbone produces adaptation and backbone effects that vary across data sets and model families.For Ctd, both effects are generally near or above zero; IBS shows clearer interface differences.
- 5.1 Adaptation and Backbone Effects: Cox shows the strongest and most consistent improvement over CE for IBS, while DeepHit gains are more pronounced for Ctd than for IBS.The results indicate that pretrained-representation value depends on the downstream survival head and evaluation metric.
- 5.2 Performance Across Data Regimes: On small data sets, RSF ranks best overall and zero-shot TabPFN remains competitive for Ctd.The size-stratified comparison aggregates results across 40 small data sets.
- 5.2 Performance Across Data Regimes: On medium data sets, supervised survival heads move toward the top, with MTLR and DeepHit competitive for Ctd and Cox adaptations strong for IBS.The size-stratified comparison includes 25 medium data sets.
- 5.2 Performance Across Data Regimes: On large data sets, Cox-adapted TabFMs lead for both Ctd and IBS, while zero-shot and CE variants fall substantially behind.Across data sizes, classification fine-tuning becomes more competitive with zero-shot inference but remains weaker than the strongest survival heads for IBS.
- 5.3 Transfer to Competing Risks: Cause-specific MTLR is the strongest TabFM interface on both IBS and Ctd across the four competing-risk data sets.Joint DeepHit ranks in the middle, cause-specific Cox is generally weaker, and the authors caution that four data sets are insufficient for a stable global ordering.
6 Discussion
The discussion frames survival transfer as an adaptation-interface problem whose effective strategy depends on data scale, evaluation target, and how directly the interface represents survival structure. Zero-shot inference is strongest on smaller data sets, while supervised adaptation becomes more useful as data grow; Cox is consistently strong for single-risk prediction, whereas cause-specific MTLR leads the limited competing-risk comparison.
- Data regime: Native zero-shot inference is most competitive on smaller data sets, whereas supervised adaptation becomes increasingly useful as data sets grow.Classification fine-tuning also becomes more competitive with zero-shot inference as data sets grow, but survival heads are stronger for probabilistic prediction, particularly IBS.
- Single-risk interfaces: Cox is the most consistently strong single-risk interface, particularly for IBS and concordance on larger data sets.DeepHit is relatively stronger for Ctd than for IBS, while MTLR remains competitive mainly in medium and large regimes without a consistent advantage across settings.
- Competing risks: Cause-specific MTLR ranks highest among TabFM survival heads on both metrics across four competing-risk data sets, while cause-specific Cox ranks lower.Joint DeepHit ranks in the middle of the field, and the ordering differs from the single-risk benchmark where Cox is strongest overall.
- Interpretation: The competing-risk ordering suggests that survival objectives interact with event structure and modeling formulation rather than identifying one universally preferable head.The stronger MTLR result is not explained simply by joint probability normalization because MTLR is fitted independently for each cause while DeepHit jointly normalizes cause–time outcomes.
- Limitations: Interpretation is constrained by mean ranks, the small competing-risk benchmark, and comparisons that do not isolate sample size or the survival head alone.The data-size analysis compares different data sets, while Cox and MTLR use cause-specific formulations and DeepHit uses a jointly normalized cause–time distribution.
- Future work: Future work should use controlled experiments and larger competing-risk benchmarks to separate target construction, censoring treatment, ranking supervision, and sample-size effects.The paper also proposes combining context adaptation with survival-specific heads and comparing generic TabFM adaptation with survival-aware pretraining.
7 Conclusion
This paper studies how generic tabular foundation models can transfer to censored time-to-event prediction through different adaptation interfaces. Across single- and competing-risk evaluations, performance depends on data scale, evaluation target, and how directly the interface represents survival structure.
- The paper transfers generic tabular foundation models to censored time-to-event prediction through different adaptation interfaces.
- Across 74 single-risk data sets, zero-shot inference remains competitive in small-data regimes, while supervised adaptation becomes increasingly useful on medium and large data sets.
- Cox is the most consistently strong single-risk interface, while cause-specific MTLR performs best among TabFM survival heads in the four-data-set competing-risk analysis.
- Successful TabFM transfer depends on data amount, evaluation target, and how directly the downstream interface represents survival structure.
Appendix A. Data Set Summary
The appendices describe the benchmark data through subject counts, covariates, event rates, and time-bin settings, including separate conventions for competing-risk data.
- Single-risk data: Table 1 records each single-risk data set’s subject count, feature count, event rate, and adaptive time-bin count.The table also identifies the domain and size regime; SurvSet data sets come from Drysdale (2022).
- Time discretization: The data-set summary tables distinguish single-risk adaptive bins from competing-risk interval conventions used by discrete-time models.This distinction is relevant when comparing formulations across the two benchmark settings.
- Competing-risk data: Table 2 summarizes competing-risk data sets using subject counts, event rates, and discrete time-interval counts.Its interval construction differs by formulation: cause-specific MTLR uses adaptive single-risk discretization, whereas joint DeepHit uses equal-width intervals over observed event times.
Appendix B. Model Details
The model appendix defines the principal classical and deep survival baselines, including Cox PH, DeepSurv, and DeepHit, with distinct hazard and event-time formulations.
- Cox PH: Cox PH models the hazard as h(t | x) = h0(t) exp(β⊤x), combining an unspecified baseline hazard with a covariate-based relative-risk term.It is reported as the strongest non-pretrained competitor in the competing-risk benchmark.
- DeepSurv: DeepSurv generalizes Cox PH by replacing the linear predictor with a neural network fθ(x) and optimizing the negative Cox partial log-likelihood.Its hazard is written h(t | x) = h0(t) exp(fθ(x)).
- DeepHit: DeepHit models a discrete event-time distribution using likelihood and ranking supervision in single-risk experiments.The competing-risk TabFM implementation reports a normalized joint DeepHit objective.
Appendix C. Training and Hyperparameter Details
TabFM interfaces use fixed strategy-level settings without hyperparameter search, while non-pretrained baselines are tuned separately within training folds.
- TabFM settings: No hyperparameter search is performed for TabFM variants; zero-shot inference, classification fine-tuning, and survival-head adaptation use fixed settings across data sets.Table 3 summarizes the strategy-level settings, with dashes indicating settings that do not apply.
- Baseline tuning: Non-pretrained baselines are tuned independently within each training fold using model-specific search spaces and 20 Optuna trials.Their search spaces are reported separately in Table 4.
- Training design: The appendix contrasts fixed TabFM configurations with fold-specific tuning for non-pretrained baselines.This separates pretrained-interface settings from baseline hyperparameter optimization.
C.1 Hyperparameter Search for Non-Pretrained Baselines
Non-pretrained baselines receive model-specific, fold-specific hyperparameter searches using 20 Optuna trials and documented parameter distributions.
- Search procedure: Each non-pretrained baseline is tuned independently within every training fold using 20 Optuna trials.The exact model-specific search spaces are reported in Table 4 and the experimental configuration files.
- Parameter sampling: Continuous log-uniform parameters are sampled on a logarithmic scale, while integer ranges use unit steps unless otherwise specified.These rules define how numerical search spaces are sampled.
- Parameter sampling: Categorical hyperparameters are sampled from the values listed in each model’s search space.The search configurations are documented in Table 4.
Appendix D. Ablation Studies
The appendix examines when supervision helps, how temporal resolution affects discrete-time interfaces, and how adaptation regimes differ computationally. It also reports the search procedure used for tuned non-pretrained baselines.
- Appendix D. Ablation Studies: The supplementary analyses probe supervision, temporal resolution, and computational differences among adaptation regimes.These analyses complement the main benchmark rather than introducing separate performance claims.
- Appendix D. Ablation Studies: Each tuned non-pretrained baseline is optimized using 20 Optuna trials within each training fold.
D.1 Label Efficiency Under Increasing Supervision
Label-efficiency analysis shows that zero-shot inference is useful with scarce supervision, while DeepHit continues improving as more labeled data become available. Classification fine-tuning remains lower throughout the experiment.
- D.1 Label Efficiency Under Increasing Supervision: Zero-shot inference improves quickly and then saturates as the labeled training fraction increases, whereas DeepHit continues gaining through the full-label setting.At the extreme low-label regime, the two methods begin at essentially the same discrimination.
- D.1 Label Efficiency Under Increasing Supervision: Classification fine-tuning remains lower than zero-shot inference and DeepHit throughout the increasing-supervision experiment.
- D.1 Label Efficiency Under Increasing Supervision: The figure tracks time-dependent Ctd as the available labeled training fraction increases from 1% to 100%.
D.2 Sensitivity to Temporal Discretization
Temporal resolution matters most when the discretization grid is very coarse, while increasing the number of bins beyond a moderate level yields diminishing returns. DeepHit is more sensitive to bin count than CE in discrimination.
- D.2 Sensitivity to Temporal Discretization: The analysis tests sensitivity of classification and discrete-time survival interfaces to the number of temporal bins.
- D.2 Sensitivity to Temporal Discretization: DeepHit changes most when moving from a very coarse grid to a moderate number of temporal bins, then largely plateaus.
- D.2 Sensitivity to Temporal Discretization: CE changes less in discrimination as the number of temporal bins increases.
- D.2 Sensitivity to Temporal Discretization: Beyond a moderate number of bins, increasing temporal resolution produces diminishing returns.The result supports the adaptive event-balanced discretization used in the benchmark.
D.3 Efficiency–Performance Trade-Off
The three interfaces trade fitting and inference costs differently. Zero-shot avoids optimization but shifts cost to repeated horizon-wise inference, while supervised adaptation adds fitting time and survival heads avoid full subject–horizon expansion.
- D.3 Efficiency–Performance Trade-Off: Zero-shot inference avoids task-specific optimization but performs native in-context prediction separately across time horizons and context samples.
- D.3 Efficiency–Performance Trade-Off: Classification adaptation optimizes temporally expanded data, whereas survival-head adaptation optimizes directly on subject-level representations.
- D.3 Efficiency–Performance Trade-Off: Survival-head adaptation avoids the full subject–horizon expansion used by classification adaptation, while supervised methods trade additional fitting time for benchmark performance gains.
- D.3 Efficiency–Performance Trade-Off: The efficiency figure compares full reported pipelines because tuned baselines include hyperparameter-search cost while TabFM variants use fixed settings.
D.4 Risk Stratification and Survival Curve Separation
Risk-stratified Kaplan–Meier curves on COLON and DIABETES illustrate how adaptation strategies translate predictions into risk-group separation. Survival-head variants generally yield clearer separation, while zero-shot and classification-based predictions can show more overlap.
- Survival-head variants generally produce clearer separation between risk tiers than zero-shot and classification-based predictions.The figures provide a qualitative view of how predicted risk groups translate into ordered Kaplan–Meier curves.
- COLON and DIABETES illustrate this behavior in medium- and small-data regimes, respectively.COLON has N = 911 and is categorized as medium; DIABETES has N = 394 and is categorized as small.
- The examples are intended to illustrate benchmark-wide discrimination results, not establish a separate interface ranking.
TabPFN TabDPT TabICL
The cited material identifies the figure panels and method labels for the TabPFN, TabDPT, and TabICL comparisons, including classification fine-tuning and other-method baselines.
- The figures include Tabular Foundation Models and a classification fine-tuning panel.
- The cited captions identify risk-stratified Kaplan–Meier curves for COLON and DIABETES.
- Other-method labels include DeepHit, DeepSurv, DySurv, and CoxPH.
Appendix E. Full Results
The full-results passages list detailed performance values across classical, neural, and TabFM survival methods. They cover multiple interfaces and data sets using Ctd, IBS, and AUC values reported as mean ± standard deviation.
- The detailed results compare Cox PH, RSF, GBSA, DeepSurv, MLP-MTLR, MLP-DH, and TabFM survival variants.
- The reported method values vary substantially across the listed data sets and model interfaces.
- TabFM survival-head results include DeepHit and MTLR variants across TabPFN, TabDPT, and TabICL.