Adaptation Interfaces for In-Context Tabular Foundation Models in Time-to-Event Prediction

Problem setting

• Censoring, time-dependent targets, and multiple event types create a statistical-interface mismatch that standard TabFM classification and regression objectives cannot directly represent.The challenge is adapting pretrained priors to downstream survival tasks with fundamentally different statistical structure.Source: §1 · p. 2

Prior work

• Prior approaches either reformulate survival as censoring-aware classification or build survival structure into pretraining, while this work broadens prior MTLR-head adaptation beyond clinically focused single-risk benchmarks.The present study treats MTLR as a baseline and evaluates alternative interfaces across broader benchmarks, additional objectives, probabilistic evaluation, and competing risks.Source: §1 · p. 2

Adaptation interfaces

• The study compares zero-shot horizon-wise reformulation, censoring-aware classification fine-tuning, and frozen-backbone survival-head adaptation using CoxPH, DeepHit, and MTLR.Supervised adaptation uses task-specific classification or survival heads while keeping the pretrained TabFM backbone frozen.Source: §1 · pp. 2–3

Evaluation scope

• The evaluation spans a broad multi-domain benchmark and competing-risk data, comparing discrimination and probabilistic prediction while distinguishing cause-specific adaptation from jointly normalized cause–time modeling.The competing-risk analysis tests whether conclusions from single-risk transfer persist with multiple event types.Source: §1 · p. 3

2.1 Modern Survival Modeling

• Survival modeling has progressed from classical estimators to deep, transformer-based, and competing-risk methods that explicitly represent event-time structure, but these models are generally trained separately for each task.The surveyed methods include CoxPH, tree-based models, DeepSurv, MTLR, DeepHit, transformers, attention mechanisms, latent-variable formulations, and longitudinal encoders.Source: §2.1 · p. 3

2.2 Foundation Models for Tabular Learning

• TabPFN, TabDPT, and TabICL represent pretrained TabFMs that use prior fitting, retrieval, or scalable in-context architectures to transfer across unseen tabular data sets.These are the exact backbone generations used in the experiments, referred to without version numbers.Source: §2.2 · p. 3
• Generic TabFMs are strongest in small- and medium-data IID regimes, while adaptation and retrieval become important for large, non-IID, or high-dimensional settings, and their standard objectives do not model censoring directly.Native interfaces can also impose output constraints, motivating separate downstream adaptation strategies.Source: §2.2 · p. 4

2.3 Foundation Models for Censored Time-to-Event Prediction

• Survival-specific TabFM approaches either reformulate prediction as censoring-aware binary classification or incorporate survival structure during pretraining, but both differ from reusing a generic tabular foundation model unchanged.Kim et al. (2026) uses the native classifier, whereas Survival In-Context (Seletkov et al., 2026) and SurvivalPFN (Qi et al., 2026) require survival-aware prior-fitted pretraining.Source: §2.3 · p. 4
• The present study transfers generic pretrained TabFMs through downstream interfaces, extending prior MTLR adaptation with CoxPH and DeepHit heads, classification fine-tuning, IBS evaluation, context-resampled training, broader data regimes, and competing-risk analyses.The benchmark covers 74 heterogeneous single-risk data sets and retains the earlier MTLR interface as a reference baseline.Source: §2.3 · p. 4

3 Methods

• Three transfer regimes use zero-shot in-context reformulation, temporally expanded classification fine-tuning, or CoxPH, DeepHit, and MTLR survival-head adaptation.Supervised regimes freeze the pretrained TabFM backbone and optimize only a task-specific head.Source: §3 · pp. 4–5

3.1 Problem Setup

• Nested cross-validation separates training, validation, and testing while targeting survival functions or competing-risk cumulative incidence functions under censoring.The pretrained representation remains context dependent because it is conditioned on sampled labeled support examples rather than used as a fixed feature extractor.Source: §3.1 · p. 5

3.2 Context-Conditioned Training and Inference

• Supervised training resamples support contexts across optimization steps, excludes each query from its context, and keeps validation or test observations out of support contexts.Survival-head contexts encode only the observed-event indicator; event time, censoring, and cause-specific information enter through downstream survival objectives.Source: §3.2 · pp. 5–6

3.3 Temporal Classification Reformulation

• Classification reformulation expands subjects across horizons with censoring-aware validity masks, retaining known labels and excluding post-censoring unknowns.Adaptive event-balanced quantile discretization selects finer grids with abundant events and coarser grids with sparse event information; classification expansion can reach O(NK) examples.Source: §3.3 · pp. 7–8

3.4 Survival Head Adaptation

• Survival-head adaptation uses a 64-unit hidden layer and frozen TabFM parameters, with scalar log-risk for CoxPH and K⋆-dimensional logits for MTLR or DeepHit.CoxPH operates continuously, whereas DeepHit uses discrete-time adaptive event-balanced quantization; MTLR is retained as the reference baseline.Source: §3.4 · pp. 8–9

3.4.1 Cox Proportional Hazards (CoxPH)

• CoxPH models continuous-time relative risk through event-indexed risk sets, while single-risk DeepHit combines discrete event-time likelihood with pairwise ranking supervision.DeepHit assigns probability mass across time intervals and uses a tail-probability contribution for censored observations.Source: §3.4.1 · p. 9, §3.4.2 · p. 9

3.5 Competing-Risk Adaptation

• Competing-risk adaptation distinguishes multiclass horizon reformulation, cause-specific Cox or MTLR models, and joint cause–time DeepHit, while keeping the TabFM backbone frozen.Cause-specific models treat other causes as censoring; the multiclass horizon formulation lacks a jointly normalized, mutually consistent cause–time distribution.Source: §3.5 · p. 9, §3.5.1 · p. 10, §3.5.2 · p. 10

3.5.3 Joint Competing-Risk DeepHit

• Joint competing-risk DeepHit uses a joint softmax over cause and time with likelihood-only training, directly enforcing probability conservation across competing causes and event times.Its grid uses equal-width intervals over the observed event-time range, unlike the adaptive quantile cuts used by single-risk discrete heads.Source: §3.5.3 · pp. 10–11

4.1 Data Set Taxonomy

• The benchmark spans diverse public tabular survival data sets, including SurvSet single-risk data and four competing-risk data sets: SUPPORT2-CR, FRAMINGHAM, PBC2, and SYNTHETIC.Missing values are median-imputed, covariates are clipped and standardized using training data only, and PCA is applied within folds when feature dimensionality exceeds a backbone’s capacity.Source: §4.1 · pp. 11–12

4.2 Pretrained Backbones and Baselines

• TabPFN, TabDPT, and TabICL are evaluated with frozen supervised variants alongside CoxPH, RSF, GBSA, DeepSurv, SurvTRACE, DySurv, matched MLP baselines, and SurvBoost for competing risks.Classification and survival-head variants use the -CE, -Cox, -DH, and -MTLR labels, with -MTLR treated as a reference baseline.Source: §4.2 · p. 12

4.3 Training Details

• All methods use fixed-seed five-fold cross-validation, while TabFM variants use fixed hyperparameters and non-pretrained baselines receive 20-trial Optuna searches within each training fold.Training uses batch size 128, bounded support contexts, event-balanced minibatches, and learning rates of 10^-5; Cox, DeepHit, and MTLR heads train for up to 50 epochs.Source: §4.3 · pp. 12–13

4.4 Evaluation Protocol

• Evaluation centers on Antolini time-dependent concordance and Integrated Brier Score for discrimination and probabilistic prediction, with dynamic AUC reported only as a supplementary metric.IBS uses inverse-probability-of-censoring weighting over 100 follow-up points and reflects both calibration and discrimination.Source: §4.4 · p. 13

5.1 Adaptation and Backbone Effects

• Adaptation and backbone effects vary across data sets and model families: Cox shows the most consistent IBS improvement over CE, whereas DeepHit gains are stronger for Ctd than IBS.These results indicate that pretrained representations do not provide a universal advantage independent of the survival head and evaluation metric.Source: §5.1 · p. 13

5.2 Performance Across Data Regimes

• On 74 single-risk data sets, Cox-adapted TabFMs lead on both Ctd and IBS for large data sets, while zero-shot and CE variants fall substantially behind.RSF ranks best on small data sets; supervised survival heads rise on medium data sets, with MTLR and DeepHit competitive for Ctd and Cox strong for IBS.Source: §5.2 · pp. 13–14
• Classification fine-tuning becomes more competitive with zero-shot inference as data sets grow but remains weaker than the strongest survival heads for IBS.Cox is consistently strong for IBS and becomes especially competitive for Ctd on large data sets, while MTLR remains competitive in medium and large regimes.Source: §5.2 · p. 14

5.3 Transfer to Competing Risks

• Cause-specific MTLR is the strongest TabFM interface for both IBS and Ctd across four competing-risk data sets, differing from the single-risk ordering where Cox leads overall.It performs comparably to the strongest tree-based baseline, while joint DeepHit ranks mid-field and cause-specific Cox is generally weaker; the limited benchmark prevents a definitive global ordering.Source: §5.3 · pp. 14–15

Adaptation regime

• Zero-shot TabFM inference is most competitive on smaller data sets, whereas supervised adaptation becomes increasingly useful as data sets grow.Classification fine-tuning also becomes more competitive with zero-shot inference, but survival heads—particularly for IBS—remain stronger, indicating that the learned survival structure matters beyond adding a new head.Source: §6 · pp. 15–16

Single-risk interfaces

• Cox is the most consistently strong single-risk interface, especially for IBS and concordance on larger data sets, while DeepHit favors Ctd over IBS and MTLR remains regime-dependent.These profiles suggest that different heads emphasize relative-risk ordering and survival-probability accuracy differently rather than following one fixed ranking.Source: §6 · p. 16

Competing risks

• Across four competing-risk data sets, cause-specific MTLR ranks highest among TabFM survival heads on both metrics, while cause-specific Cox ranks lower.This ordering is preliminary and suggests that survival objectives interact with event structure and modeling formulation rather than identifying one universally preferable head.Source: §6 · pp. 16–17

Limitations

• The results have important interpretive limits because mean ranks can mask data-set differences, the competing-risk benchmark is small, and comparisons confound target construction, objectives, and data-set characteristics.Competing-risk conclusions cover survival-head comparisons only, and observed head explanations should be treated as hypotheses rather than causal conclusions.Source: §6 · p. 17

Future work

• Future work should use controlled experiments, larger competing-risk benchmarks, combined context adaptation and survival heads, and direct comparisons with survival-aware pretraining.These directions target separation of target construction, censoring treatment, ranking supervision, and sample-size effects.Source: §6 · p. 17

• Across 74 single-risk data sets, native zero-shot inference remains competitive in small-data regimes, whereas supervised adaptation becomes increasingly useful for medium and large data sets.Source: §7 · p. 17
• Frozen TabFM backbones with task-specific heads support several of the strongest methods in larger data regimes, so the pattern is not simply explained by degraded pretrained representations.Source: §7 · p. 17
• Cox is the most consistently strong single-risk interface, particularly for IBS and concordance on larger data sets, while DeepHit is relatively stronger for discrimination than probabilistic prediction.Source: §7 · p. 17
• In four competing-risk data sets, cause-specific MTLR performs best among TabFM survival heads, although this ordering should be interpreted cautiously because the benchmark is limited.Source: §7 · p. 17
• Overall, successful transfer depends on available data, the evaluation target, and how directly the downstream interface represents survival structure.Source: §7 · p. 18

Single-risk data sets

• Table 1 records single-risk data-set characteristics, including subjects, features, event rates, adaptive time bins, domains, and regimes.The data sets are collected from Drysdale (2022).Source: Table 1 · p. 19
• The single-risk data sets have interquartile ranges of [146, 1,331] subjects, [6, 36] features, [0.24, 0.57] event rates, and [10, 30] adaptive time bins.Source: Table 1 · p. 19, Appendix A · p. 21

Competing-risk data sets

• For competing-risk data sets, Table 2 reports subjects, event rates, and discrete-time bins, with intervals defined differently for cause-specific MTLR and joint DeepHit.Cause-specific MTLR uses the corresponding adaptive single-risk discretization, whereas joint DeepHit uses equal-width intervals over the observed event-time range.Source: Table 2 · p. 21

Non-TabFM baselines

• The benchmark includes Cox PH, RSF, GBSA, DeepSurv, DeepHit, SurvTRACE, and DySurv, implemented through scikit-survival, pycox, or adapted original codebases.Cox PH uses fixed regularization rather than Optuna search, while SurvBoost is included only in the competing-risk benchmark as the strongest non-pretrained competitor.Source: Appendix B · pp. 21–22

Cox-based models

• Cox PH models proportional hazards with an unspecified baseline hazard and linear covariate risk, whereas DeepSurv replaces the linear predictor with a neural network.Both are trained using Cox partial-likelihood objectives, with Cox PH estimated by regularized partial likelihood and DeepSurv minimizing negative Cox partial log-likelihood.Source: Appendix B · pp. 21–22

Representation controls

• MLP-MTLR and MLP-DH isolate pretrained representation effects by matching TabFM survival objectives and adaptive event-balanced time grids while training the trunk from scratch.Comparisons with TabPFN-MTLR, TabPFN-DH, and DeepSurv vary representation source while holding the relevant survival objective and discretization fixed.Source: Appendix B · p. 22

DeepHit

• DeepHit models a discrete event-time distribution using likelihood and ranking supervision for single risk and a normalized cause–time likelihood for competing risks.The competing-risk TabFM implementation uses the joint DeepHit objective described in Section 3.5.Source: Appendix B · p. 22

Training and Hyperparameter Details

• TabFM variants use fixed settings across data sets, with no hyperparameter search performed.All three interfaces follow the same fixed-setting approach across data sets.Source: Appendix C · p. 23
• Non-pretrained baselines are tuned independently within each training fold using 20 Optuna trials.Their model-specific search spaces are reported in Table 4.Source: Appendix C · p. 23
• Table 3 summarizes the strategy-level settings used by the TabFM interfaces.A dash indicates that a setting is not applicable.Source: Table 3 · p. 23

C.1 Hyperparameter Search for Non-Pretrained Baselines

• Each non-pretrained baseline is tuned independently within every training fold using 20 Optuna trials.Table 4 reports the exact model-specific search spaces used in the experimental configuration files.Source: §C.1 · p. 23
• Log-uniform continuous parameters are sampled logarithmically, integer ranges use unit steps unless noted, and categorical parameters draw from listed values.Source: §C.1 · p. 23

Appendix D. Ablation Studies

• The supplementary analyses probe complementary aspects of the main benchmark rather than presenting separate performance claims.Source: Appendix D · p. 23
• Tuned non-pretrained baselines are optimized using 20 Optuna trials within each training fold.Source: Table 4 · p. 24
• The ablations examine when additional supervision becomes useful and how sensitive discrete-time interfaces are to temporal resolution.Source: Appendix D · p. 24
• They also compare the three adaptation regimes in terms of computational and qualitative behavior.Source: Appendix D · p. 24

D.1 Label Efficiency Under Increasing Supervision

• In the extreme low-label regime, zero-shot inference and DeepHit adaptation begin with essentially the same time-dependent Ctd.Source: §D.1 · p. 24
• Classification fine-tuning remains lower than the other approaches throughout the increasing-supervision experiment.Source: §D.1 · p. 24
• As labeled training increases from 1% to 100%, zero-shot inference improves quickly before saturating, whereas DeepHit continues gaining through the full-label setting.Source: §D.1 · pp. 24–25, Figure 4 · p. 25

D.2 Sensitivity to Temporal Discretization

• DeepHit is most sensitive to increasing temporal bins from a very coarse grid, then largely plateaus, while CE changes less in discrimination.The results support adaptive event-balanced discretization to avoid bins with too few events, rather than explaining any interface’s relative advantage.Source: Figure 5 · p. 25, §D.2 · p. 26

• Zero-shot avoids task-specific optimization but performs in-context prediction separately across time horizons and context samples.Its computational burden shifts toward inference rather than fitting.Source: §D.3 · p. 26
• Supervised adaptation trades additional fitting time for performance gains, with survival-head adaptation avoiding classification adaptation’s full subject–horizon expansion.Survival-head models optimize on subject-level representations and average predictions over five training-context samples at inference.Source: §D.3 · p. 26
• Figure 6 compares full reported pipelines, because tuned baselines include hyperparameter-search cost while TabFM variants use fixed settings.Therefore, the efficiency results do not establish that one model family is intrinsically faster to optimize.Source: §D.3 · p. 26

D.4 Risk Stratification and Survival Curve Separation

• Survival-head variants generally produce clearer Kaplan–Meier curve separation across the COLON (N = 911, medium) and DIABETES (N = 394, small) examples, whereas zero-shot and classification-based predictions show more overlap in some panels.These examples illustrate the practical manifestation of benchmark-wide discrimination differences rather than establishing a separate interface ranking or universal separation pattern.Source: §D.4 · p. 27

TabPFN TabDPT TabICL

• The section presents classification fine-tuning for TabPFN, TabDPT, and TabICL.Source: TabPFN TabDPT TabICL · pp. 28–29
• Risk-stratified Kaplan–Meier curves are shown for the COLON and DIABETES data sets.Source: Figure 7 · p. 28, Figure 8 · p. 29
• It identifies CoxPH, Gradient Boosting, and DeepSurv as other methods considered.Source: TabPFN TabDPT TabICL · p. 29

Detailed single-risk results

• Table 5 reports Ctd, IBS, and AUC for classical, neural, zero-shot, classification, Cox, DeepHit, and MTLR models across the detailed single-risk results.Metrics are reported as mean ± standard deviation across five folds; higher is preferred for Ctd and AUC, while lower is preferred for IBS.Source: Table 5 · p. 30
• For one reported dataset block, TabDPT-DH reaches Ctd 0.740±0.079, IBS 0.255±0.082, and AUC 0.768±0.075, while Cox PH records 0.690±0.056, 0.236±0.092, and 0.756±0.027.These entries illustrate the metric variation among adapted TabFMs and the Cox PH baseline within the same block.Source: Appendix E · p. 31

Cox adaptation

• In another block, TabPFN-Cox records Ctd 0.730±0.013, IBS 0.144±0.002, and AUC 0.759±0.010, compared with Cox PH at 0.726±0.011, 0.176±0.002, and 0.741±0.011.The paired values show improved IBS and AUC for TabPFN-Cox in this reported block, while Ctd is also slightly higher.Source: Appendix E · p. 31

Zero-shot and adapted interfaces

• The appendix also includes blocks where zero-shot and adapted interfaces differ substantially, such as TabICL-ZS at Ctd 0.806±0.032, IBS 0.161±0.070, and AUC 0.849±0.031.The same block reports TabICL-Cox at 0.781±0.051, 0.125±0.024, and 0.825±0.029, demonstrating metric-specific variation across interfaces.Source: Appendix E · pp. 36–37

Full benchmark results

• TabFM adaptation results vary markedly across data sets and metrics rather than yielding a single uniformly dominant interface.For example, TabPFN-Cox reaches Ctd 0.658±0.014, IBS 0.132±0.050, and AUC 0.707±0.018 in one table, whereas TabICL-ZS reaches Ctd 0.756±0.034, IBS 0.248±0.269, and AUC 0.837±0.070 in another.Source: Appendix E · pp. 38, 42
• Cox adaptations include TabPFN-Cox with IBS 0.132±0.050 and TabDPT-Cox with IBS 0.144±0.057 in the cited results tables.Source: Appendix E · p. 38

Zero-shot results

• Zero-shot models are competitive on discrimination in several tables, with TabPFN-ZS achieving Ctd 0.658±0.022 and AUC 0.700±0.014, while TabICL-ZS reaches Ctd 0.756±0.034 and AUC 0.837±0.070 elsewhere.The associated IBS values are 0.145±0.062 and 0.248±0.269, respectively, illustrating that discrimination and probabilistic accuracy can differ.Source: Appendix E · pp. 38, 42

Classification fine-tuning

• Classification fine-tuning can produce poor IBS despite reasonable discrimination, as TabDPT-CE records Ctd 0.694±0.030, IBS 0.663±0.042, and AUC 0.718±0.047 in one table.In another table, TabPFN-CE records Ctd 0.638±0.029, IBS 0.163±0.065, and AUC 0.692±0.038, confirming variation across data sets.Source: Appendix E · pp. 38, 42

Survival-head adaptation

• DeepHit and MTLR survival heads also show heterogeneous outcomes: TabDPT-DH reaches Ctd 0.747±0.019, while TabICL-MTLR reaches AUC 0.749±0.058 in separate tables.Their corresponding IBS values are 0.056±0.004 and 0.155±0.014, respectively.Source: Appendix E · pp. 42–43

Single-risk full results

• The full single-risk tables compare all baselines and TabFM interfaces across Ctd, IBS, and AUC, including zero-shot, CE, Cox, DH, and MTLR variants.Reported rows include Cox PH, RSF, GBSA, DeepSurv, MLP survival heads, SurvTRACE, DySurv, and three TabFMs with each adaptation.Source: Appendix E · pp. 45–47
• Across the reported data sets, adaptation outcomes differ by backbone and interface rather than yielding a uniform ranking across Ctd, IBS, and AUC.For example, one table reports TabDPT-Cox at 0.636±0.043 Ctd, 0.143±0.014 IBS, and 0.630±0.056 AUC, while another reports TabICL-MTLR at 0.701±0.055 Ctd, 0.107±0.011 IBS, and 0.682±0.061 AUC.Source: Appendix E · pp. 45–46
• The appendix also shows substantial variation in larger-valued result sets, including Cox PH at 0.783±0.019 Ctd, 0.153±0.013 IBS, and 0.894±0.022 AUC, alongside competing TabFM interfaces.The same table reports TabDPT-DH at 0.790±0.025 Ctd, 0.233±0.016 IBS, and 0.884±0.008 AUC.Source: Appendix E · p. 46

Competing-risk results

• Competing-risk results are reported separately for each cause, with DeepSurv and MTLR-based TabFM heads providing distinct Ctd, IBS, and AUC profiles.DeepSurv reports 0.747±0.007 Ctd, 0.150±0.004 IBS, and 0.807±0.009 AUC for one cause, while TabPFN-MTLR reports 0.714±0.007, 0.166±0.004, and 0.796±0.007.Source: Table 6 · p. 48, Appendix E · p. 49
• For the second competing cause, corresponding values remain method-specific, with TabPFN-MTLR at 0.717±0.006 Ctd, 0.167±0.004 IBS, and 0.798±0.007 AUC.The table also reports DeepSurv at 0.749±0.005 Ctd, 0.152±0.003 IBS, and 0.805±0.006 AUC for that cause.Source: Table 6 · p. 48, Appendix E · p. 49