Source-linked AI summary
Continuous and Discrete-Time Survival Prediction with Neural Networks
Håvard Kvamme, Ørnulf Borgan
TL;DR
The paper asks how discrete-time neural survival methods can predict continuous-time outcomes while handling discretization and interpolation choices. It evaluates PMF and hazard parameterizations, proposes interpolation schemes and PC-Hazard, and finds hazard-based methods slightly stronger with PC-Hazard highly competitive.
Problem
Applying discrete-time survival methods to continuous-time data requires choices about discretization and interpolation that have received little attention.
Method
The paper evaluates neural-network PMF and discrete-hazard likelihoods, two interpolation schemes, quantile-based discretization, and a piecewise-constant continuous-time hazard method called PC-Hazard.
Results
The hazard-rate parametrization performed slightly better than PMF, interpolation typically improved smaller-data performance, and PC-Hazard was highly competitive with existing methods.
Takeaways & Limitations
Kaplan–Meier quantile grids and interpolation can improve smaller-grid or smaller-data performance, while PC-Hazard remains effective with coarser grids.
Takeaways & Limitations
The discrete-time likelihood is restricted to event times within the censoring-time grid because events beyond τm cannot be observed.
Abstract
from arXiv · showhide
Application of discrete-time survival methods for continuous-time survival prediction is considered. For this purpose, a scheme for discretization of continuous-time data is proposed by considering the quantiles of the estimated event-time distribution, and, for smaller data sets, it is found to be preferable over the commonly used equidistant scheme. Furthermore, two interpolation schemes for continuous-time survival estimates are explored, both of which are shown to yield improved performance compared to the discrete-time estimates. The survival methods considered are based on the likelihood for right-censored survival data, and parameterize either the probability mass function (PMF) or the discrete-time hazard rate, both with neural networks. Through simulations and study of real-world data, the hazard rate parametrization is found to perform slightly better than the parametrization of the PMF. Inspired by these investigations, a continuous-time method is proposed by assuming that the continuous-time hazard rate is piecewise constant. The method, named PC-Hazard, is found to be highly competitive with the aforementioned methods in addition to other methods for survival prediction found in the literature.
1. Introduction
The paper studies how neural-network survival methods handle continuous-time data through discretization and interpolation. It compares PMF and discrete-hazard parameterizations and introduces a piecewise-constant continuous-time hazard approach.
- The paper applies neural networks to likelihood-based discrete-time survival methods for right-censored time-to-event prediction.
- Continuous-time data require discretizing the time scale before discrete-time survival methods can be applied.
- The study compares equidistant-time and equidistant-survival-probability grids while varying the number of time points.
- Interpolation is investigated as a way to use coarser grids without increasing approximation error.The two schemes assume either constant density or constant hazard between grid points.
- A continuous-time method is proposed by assuming a piecewise-constant continuous-time hazard rate.
2. Related Works
Related work spans neural-network extensions of discrete-time likelihoods, Cox-based continuous-time models, and nonparametric methods for right-censored survival prediction. The paper positions its methods among these alternatives.
- Neural-network discrete-time methods parameterize either the PMF or the discrete hazard rate.
- DeepHit parameterizes the event-time PMF and combines the right-censored log-likelihood with a ranking loss.It has also been extended to competing-risks data.
- Cox-based neural methods extend a semi-parametric continuous-time model, but proportional hazards restrict survival curves from crossing.
- Random Survival Forest is a fully non-parametric continuous-time method that estimates cumulative hazard with the Nelson–Aalen estimator.
3. Discrete-Time Models
The paper formulates right-censored discrete-time survival likelihoods through either event-time PMFs or discrete hazards, then parameterizes them with neural networks. It also relates multi-task logistic regression to PMF modeling.
- Discrete-time survival models represent event times on an ordered grid and model their PMF or survival function.
- The discrete hazard is the conditional probability of an event at τj given survival beyond τj−1.
- Right-censored likelihood construction assumes independent event and censoring times and separates their contributions when they share no parameters.
- The hazard likelihood becomes a Bernoulli log-likelihood, or binary cross-entropy, for event indicators across time points.
- Neural networks parameterize hazards with sigmoid outputs and PMFs with softmax-based probabilities.
- Multi-task logistic regression is equivalent to the PMF parametrization after applying a reverse cumulative sum to network outputs.The paper reports no benefit from this extra cumulative sum and notes possible numerical instability for large m.
4. Continuous-Time Models
The paper applies discrete-time survival methods to continuous-time data through time discretization and interpolation, then proposes PC-Hazard with a piecewise constant continuous-time hazard. The discretization and interpolation choices are designed to balance neural-network parameter count against approximation error.
- 4.1 Discretization of Durations: Continuous-time survival data are converted for discrete-time PMF and Logistic-Hazard methods by selecting a grid of time points.The paper considers equidistant grids and grids based on quantiles of the estimated event-time distribution.
- 4.1 Discretization of Durations: Quantile-based discretization creates denser grids where more events occur and coarser grids where the survival curve is flatter.The grid points are obtained by solving for durations corresponding to evenly spaced survival probabilities.
- 4.1 Discretization of Durations: Discretization moves events to the end of their intervals and censorings to the end of the previous interval before fitting the discrete-time likelihood.For τj−1 < Ti ≤ τj, an event time becomes τj, whereas a censored time becomes τj−1.
- 4.2 Interpolation for Continuous-Time Predictions: Two interpolation schemes replace stepwise discrete survival estimates with monotone continuous-time estimates: CDI uses constant density, whereas CHI uses constant hazard.These correspond to piecewise linear and piecewise exponential survival estimates, respectively.
- 4.2 Interpolation for Continuous-Time Predictions: CDI implies increasing hazard within each interval, while CHI implies constant hazard and a decreasing density.The schemes therefore encode different within-interval event distributions.
- 4.3 A Piecewise Constant Continuous-Time Hazard Parametrization: PC-Hazard parameterizes a continuous-time hazard that is constant within each interval and uses neural networks to estimate non-negative interval hazards.Its survival estimates are piecewise exponential, and its interval boundaries still require discretization.
5. Simulations
The simulations examine how training-set size, discretization, interpolation, and model parametrization affect neural-network survival prediction. Kaplan–Meier-quantile grids and interpolation generally improve robustness, while PC-Hazard remains competitive with interpolated Logistic-Hazard.
- Kaplan–Meier-quantile discretization generally scores slightly better than equidistant grids, especially for smaller grids and training sets.
- PMF performs slightly worse than Logistic-Hazard, while interpolated estimates provide more stable performance despite nearly identical best scores.
- CHI and CDI have nearly indistinguishable performance, differing only slightly in MSE and concordance.
- PC-Hazard is competitive with interpolated Logistic-Hazard, performing best in some small-grid or small-training-set settings, but is more sensitive to discretization.
- Grid size strongly affects performance: finer grids reduce bias but require more neural-network parameters and can increase variance.
- Interpolation makes performance less sensitive to grid size and generally improves smaller-grid estimates, whereas discrete estimates can perform poorly at grid size 5.
6. Experiments with Real Data
The real-data experiments compare the paper’s methods with established neural, Cox, and forest survival methods using five-fold cross-validation. PC-Hazard and interpolated Logistic-Hazard achieve broadly strong performance across discrimination and calibration measures.
- The comparison covers DeepHit, DeepSurv, Cox-Time, CoxCC, Random Survival Forests, regular Cox regression, and the paper’s survival methods on five real-world data sets.
- Integrated Brier score is used to tune the paper’s methods because it assesses both discrimination and calibration when the true survival function is unavailable.
- Experiments use five-fold cross-validation, repeated fitting, and averaged median concordance and integrated Brier scores across folds.
- DeepHit and PC-Hazard perform very well in concordance, with Logistic-Hazard variants and Cox-Time close to PC-Hazard.
- PC-Hazard performs very well on integrated Brier score, while PMF, Random Survival Forests, and proportional Cox methods are generally slightly worse.
- Across the comparisons, interpolated Logistic-Hazard and PC-Hazard appear to provide the most stable high performance for discrimination and calibration.
7. Discussion
The paper compares neural-network survival parameterizations, interpolation schemes, and discretization grids for continuous-time prediction. Hazard-based methods, interpolation, PC-Hazard, and quantile-based grids each show practical advantages in the reported studies.
- Logistic-Hazard performed slightly better than the PMF parametrization across simulated and real data studies.
- Interpolation typically improved performance for smaller data sets by allowing coarser time grids without increasing approximation error.Coarser grids reduce neural-network parameters, while interpolation addresses discretization error at test time.
- CDI performed slightly better than CHI among the two interpolation schemes investigated.
- PC-Hazard performed very well in discrimination and calibration and remained effective with coarser grids, especially for the smallest simulated data set.
- Quantile-based grids typically outperformed equidistant grids for coarser grids.The grids use quantiles of the event-time distribution estimated by Kaplan-Meier.
Appendix A. More on the Simulations
The appendix provides additional information about the simulation study and points readers to detailed data-generation procedures in Section 5 and Appendix A.2.
- The appendix introduces additional simulation-study information and directs readers to detailed dataset construction and further results.
A.1 Discrete-Time Survival Simulations from Logit Hazards
The simulations generate discrete survival data from covariate-dependent logit hazards built from several time-dependent functions. Covariates are constructed to preserve specified linear combinations while allowing changes in covariate dimensionality.
- Simulated survival data are generated by drawing from discrete hazards over times 0.1, 0.2, …, 100.
- The logit hazard is a weighted sum of sinusoidal, continuous, and accelerated functions.
- Each transformed covariate is a linear combination of a non-overlapping, equal-sized subset of covariates.For subsets of size m, the construction uses x ∈ R9m.
- The simulation uses covariate-dependent γ_i(x) functions chosen to produce reasonable survival functions.One function ensures that the number of periods is a multiple of 2.
- Covariates are drawn conditionally while preserving the target linear combinations defined by β_j.
- The construction permits changing the number of covariates without affecting the hazards.Simulation-generation code is available in the pycox repository.
A.2 Additional Simulation Results
Additional simulation results compare Logistic-Hazard, its CHI interpolation, PC-Hazard, and PMF across many fitted grid and repetition configurations. PC-Hazard and CHI generally outperform discrete estimates, while differences shrink for the best grids.
- Each method was fitted 80 times across 4 grids, 2 discretization schemes, and 10 repetitions.
- PC-Hazard and Logistic-Hazard (CHI) performed better than the discrete estimates of Logistic-Hazard and PMF.
- Logistic-Hazard generally performed better than the PMF method.
- For the best grid configurations, performance differences between all models were very small.Stable performance across varied hyperparameter configurations remains practically desirable.
Appendix B. PC-Hazard and Poisson Regression
PC-Hazard is presented as a neural-network version of the piecewise exponential model, with a likelihood proportional to a Poisson likelihood. The appendix defines interval-based variables and exposure terms used to express this relationship.
- PC-Hazard is essentially a neural-network version of the piecewise exponential model.
- Defining interval indicators y_ij and interval-specific exposure terms allows each individual likelihood contribution to be rewritten in Poisson form.
- Figure A.1 plots simulation-study MSE and concordance scores from best to worst, with training-set size shown above each plot.The plots are not on the same scale.
Appendix C. Implementation details
The implementations differ slightly from the mathematical notation to improve numerical stability. The appendix describes stability choices for PMF, discrete-hazard, and continuous-hazard parameterizations.
- The implementations of the survival methods differ slightly from the mathematical notation because numerical stability must be considered.
- PMF parameterization: The PMF parameterization uses the log-sum-exp trick to stabilize exponential calculations.The reformulation ensures exponentials are taken only of non-positive numbers.
- Discrete hazard parametrization: The discrete hazard parameterization uses Bernoulli negative log-likelihood, or binary cross-entropy, with numerically stable implementations.These implementations apply the log-sum-exp trick to the logits.
- Continuous hazard parametrization: The continuous hazard implementation uses softplus approximations for large positive and sufficiently negative inputs.For large z, log(1 + exp[z]) is approximated by z; for φ_κ(t_i)(x_i) much less than zero, a separate approximation is used.