PaperlayerView exact revision on arXiv

Source-linked AI summary

Survival Regression with Proper Scoring Rules and Monotonic Neural Networks

David Rindt, Robert Hu, David Steinsaltz, Dino Sejdinovic

arXiv:2103.14755v2stat.MLcs.LG

TL;DR

Because existing survival-regression methods lack tractable right-censored likelihoods, they commonly rely on alternative scores that may rank inaccurate distributions above the truth. SuMo-net is a scalable, flexible survival-regression model that directly optimizes right-censored log-likelihood. SuMo-net achieves state-of-the-art results on a range of datasets, together with 20–100 times speedups.

  • Problem

    Because existing survival-regression methods lack tractable right-censored likelihoods, they commonly rely on alternative scores that may rank inaccurate distributions above the truth.

    Source: §1 · p. 2

  • Method

    SuMo-net is a scalable, flexible survival-regression model that directly optimizes right-censored log-likelihood.

    Source: §1 · p. 2

  • Results

    SuMo-net achieves state-of-the-art results on a range of datasets, together with 20–100 times speedups.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The authors conclude that right-censored log-likelihood is attractive for scoring and fit assessment, and SuMo-net makes it tractable by construction.

    Source: §6 · p. 9

    Main-text comparisons use a derivative approximation with k = 2, so comparing approximated and exact likelihood scores is not ideal.

    Source: §5 · p. 8

Abstract

from arXiv · show

We consider frequently used scoring rules for right-censored survival regression models such as time-dependent concordance, survival-CRPS, integrated Brier score and integrated binomial log-likelihood, and prove that neither of them is a proper scoring rule. This means that the true survival distribution may be scored worse than incorrect distributions, leading to inaccurate estimation. We prove that, in contrast to these scores, the right-censored log-likelihood is a proper scoring rule, i.e., the highest expected score is achieved by the true distribution. Despite this, modern feed-forward neural-network-based survival regression models are unable to train and validate directly on the right-censored log-likelihood, due to its intractability, and resort to the aforementioned alternatives, i.e., non-proper scoring rules. We therefore propose a simple novel survival regression method capable of directly optimizing log-likelihood using a monotonic restriction on the time-dependent weights, coined SurvivalMonotonic-net (SuMo-net). SuMo-net achieves state-of-the-art log-likelihood scores across several datasets with 20--100$\times$ computational speedup on inference over existing state-of-the-art neural methods, and is readily applicable to datasets with several million observations.

1 INTRODUCTION AND RELATED WORKS

The introduction frames right-censored survival regression as modeling event times from covariates, then highlights the gap between proper likelihood scoring and the scores existing methods can use tractably. It proposes SuMo-net to optimize likelihood directly and reports state-of-the-art results with substantial inference speedups.

1 INTRODUCTION AND RELATED WORKS

  • Prior approaches include neural extensions of Cox, random forests, discrete-time models, and latent-representation methods.Classical alternatives also include accelerated failure-time and parametric models.Source: §1 · p. 1
  • Right-censored log-likelihood is proved proper, with the true distribution attaining the highest expected score.Source: §1 · p. 2
  • Existing survival-regression approaches lack tractable likelihood, impeding its direct use in estimation; right-censored settings therefore commonly use alternative scores.Source: §1 · p. 2
  • NeuralODE likelihood models are flexible but have scalability pathologies because they scale polynomially with the number of parameters.Source: §1 · p. 2
  • Time-dependent concordance, (integrated) Brier score, binomial log-likelihood, and survival-CRPS are shown theoretically and experimentally not proper, allowing inaccurate distributions to score better.Source: §1 · p. 2
  • SuMo-net directly optimizes right-censored log-likelihood and is compared with existing methods on performance and scalability, achieving state-of-the-art results on a range of datasets with 20–100 times speedups.It is described as scalable and flexible.Source: §1 · p. 2

2 BACKGROUND AND NOTATION

The section establishes the observed-data notation and target for right-censored survival regression, along with an independent-censoring assumption. It also notes the integration burden of density-based likelihoods and the use of partial likelihood instead.

2 BACKGROUND AND NOTATION

  • Each observation pairs covariates with a time capped by censoring and an indicator of whether the event time was observed.The observed time is Z = min{T, C}, with D indicating whether Z = T.Source: §2 · pp. 2–3
  • The conditional survival function is S(t|x) = 1 − F(t|x), and, when T has a density, f(t|x) is the derivative of F(t|x).Source: §2 · p. 2
  • The goal is to estimate the survival function S(t|x) from sampled observations and assess whether the estimate is accurate relative to the true distribution.Source: §2 · p. 2
  • The analysis assumes censoring time and event time are conditionally independent given the covariate.Source: §2 · p. 2
  • Density-based likelihood requires estimating n integrals, making optimization difficult; many methods instead optimize partial likelihood.Source: §2 · p. 3

3 EVALUATION CRITERIA AND PROPER SCORING RULES

The section defines propriety by whether the true distribution maximizes expected score, then contrasts commonly used survival metrics that may favor false distributions with right-censored log-likelihood, which is proper under an assumption.

3 EVALUATION CRITERIA AND PROPER SCORING RULES

  • A scoring rule is proper when the true distribution achieves the optimal expected score, including in the survival-regression setting.The survival definition applies across censoring distributions and covariates.Source: §3 · p. 3

3.1 Existing evaluation metrics are not proper scoring rules

  • Time-dependent concordance, Brier score, binomial log-likelihood and survival-CRPS can favor false distributions over the true one; integrated Brier and binomial log-likelihood are also non-proper.In the concordance example, a false distribution scores higher than the true distribution.Source: §3.1 · pp. 3, 5, Figure 1 · p. 4
  • Brier score and binomial log-likelihood are proper under independent censoring and correctly estimated censoring weights, but may fail when censoring depends on covariates.The paper notes that estimating covariate-dependent censoring weights introduces a regression-selection loop.Source: §3.1 · p. 4
  • Survival-CRPS can favor a false CDF: with deterministic censoring at c, minimizing the score sets the estimated CDF at c to 1 even when the true CDF is lower.The proposed weighted-CRPS argument does not recover survival-CRPS.Source: §3.1 · p. 5

3.2 The right-censored log-likelihood is a proper score

  • Under Assumption 2, which requires finite KL divergence, right-censored log-likelihood is proper, motivating direct training and evaluation instead of non-proper proxy scores.In the paper’s examples, it gives the true distribution a higher score than the fake distribution.Source: §3.2 · pp. 5–6

4 OPTIMIZING LIKELIHOOD USING PARTIALLY MONOTONIC NEURAL NETWORKS

SuMo-net adapts a monotonic neural density estimator to jointly model survival functions and densities, enabling direct training and evaluation on right-censored log-likelihood. It uses non-negative time-dependent weights to enforce monotonicity and differentiates its survival output to obtain the density.

Training

  • SuMo-net adapts a monotonic neural density estimator to jointly model survival and density, enabling direct training and evaluation on right-censored log-likelihood.The model is presented as scalable and flexible.Source: §4 · p. 6

Architecture

  • Covariates pass through hcov(x), then combine with time t in a fully connected mixed network whose scalar output h(t, x) defines the survival curve.The network models S(t|x) as 1 − sigmoid(h(t, x)).Source: §4 · p. 6, Figure 2 · p. 7

Training

  • Differentiating the modeled survival function with respect to t yields the conditional event-time density; the authors use automatic differentiation or a chain-rule approximation.Both derivative implementations performed comparably and outperformed a simple finite-difference estimate in the reported experiments.Source: §4 · pp. 6–7, Figure 2 · p. 7

Universality

  • Non-negative weights descending from time t ensure the network output is non-decreasing in t; with at least two mixed-network hidden layers, it can universally approximate survival functions given enough nodes.The implementation enforces non-negativity by squaring unrestricted weights.Source: §4 · p. 7

5 EXPERIMENTS

Across five real datasets, SuMo-net performs well on right-censored likelihood, though many baseline likelihoods are approximated by interpolating survival curves. The experiments also report inference speedups, score correlations, and calibration findings.

Results

  • SuMo-net performs well on right-censored likelihood against more complex NeuralODE methods, although many benchmark likelihoods are approximated by interpolating survival curves.Table 3 reports five-fold log-likelihood scores; SuMo-net, Weibull, Log-normal, and SODEN use exact likelihood, while other methods use approximations.Source: §5 · p. 7, Table 3 · p. 9
  • DeepHit generally has the best time-dependent concordance but the worst likelihood scores among the compared methods.The authors note this pattern agrees with their finding that time-dependent concordance is not a proper scoring rule.Source: §5 · p. 8

Correlation between likelihood and existing metrics

  • Concordance is almost independent of likelihood, while IBLL has the strongest rank correlation with likelihood across the datasets.The correlation matrix averages rank correlations over FLCHAIN, GBSG, METABRIC, SUPPORT, and KKBOX.Source: §5 · p. 8, Table 5 · p. 9

Calibration

  • SuMo-net generally provides a calibrated survival distribution; calibration is evaluated only for methods trained directly on likelihood.The authors state that empirical frequencies are only coarsely approximated without direct access to the cumulative distribution function.Source: §5 · pp. 8–9, Table 6 · p. 9

Inference and training timings

  • 20–100×: SuMo-net achieves this inference speedup over Cox-Time and SODEN by evaluating survival probability in a single forward pass.Cox-Time integrates its learned hazard over time; the comparison used identical architectures.Source: Table 7 · p. 9, §5 · p. 9

6 CONCLUSION

The paper finds that several common right-censored survival scores are not proper, while right-censored log-likelihood is proper. It proposes SuMo-net as a flexible model with tractable likelihood and points to extensions to other architectures and data types.

Conclusion

  • Several existing scores for right-censored regression may not faithfully reflect fitted-distribution accuracy, whereas right-censored log-likelihood is proper.The authors caution that other scores may still be useful for certain purposes, but their limitations should be considered.Source: §6 · p. 9
  • SuMo-net is a simple, scalable, and flexible survival-regression model for which right-censored log-likelihood is tractable by construction.The conclusion identifies more involved architectures and high-dimensional data, including genetic and image data, as future directions.Source: §6 · p. 9

Downstream applications

  • The authors propose extending time-to-event monotonicity constraints to other trainable-weight models so they can train on censored survival likelihoods.They identify epidemiology, medical statistics, and economics among fields that could use such models, particularly in safety-critical settings.Source: §6 · p. 9

A.1 Details on time-dependent concordance example

The example uses two covariate-specific event-time distributions built from a shared discrete uniform component and a uniform offset. Their hazard rates occupy disjoint time sets.

A.1 Details on time-dependent concordance example

  • For binary X, the event time is K + U when X = 0 and K + U + 1 when X = 1, with K uniform on {0, 2, 4, 6, 8} and U uniform on [0, 1].Censoring time C is exponential with mean 20.Source: §A.1 · p. 12
  • The two covariate-specific distributions have hazard rates that are nonzero on disjoint sets.Source: §A.1 · p. 12
  • The example’s false distribution is constructed so that, at each time, the distribution with nonzero hazard has the lowest survival probability.Source: §A.1 · p. 12

A.2 Details on Brier score example

The example compares true and false distributions under unequal censoring, evaluating Brier score at t = 4 and integrated Brier score from t = 0 to t = 40.

A.2 Details on Brier score example

  • With X ∼ Bernoulli(0.1), censoring is C ∼ Exp(1) when X = 0 and C = ∞ when X = 1; the true event time is T ∼ Exponential(10).The false distribution matches the truth for X = 1 but uses 1 − exp(−t/4) for X = 0; the sample size is n = 10^6.Source: §A.2 · p. 12

A.3 Discussion of survival-CRPS

This appendix discusses a proposed survival-CRPS proof strategy, gives an example, and records supplementary calculations, scores, implementation details, and toy-data definitions.

A.3.1 Discussion of proof given in Avati et al

  • Conditioning on C = c gives u(r) = 1{r ≤ c}, but the resulting expression omits a survival-CRPS term; evaluating only r > c favors F(r) = 1 beyond c.The proposed uncensored-region weight depends on the censoring time, although the weighted score is stated for non-random weight functions.Source: §A.3.1 · pp. 12–13

A.3.2 An example

  • The example compares survival-CRPS for a true event-time distribution with mean 100 and a false exponential distribution with mean 25 under likely censoring.Individuals are set to fail at t = 200, censoring is exponential with mean 10, and the comparison uses samples of size 10^6.Source: §A.3.2 · p. 13

A.4 Calculating life expectancies using SuMo-net

  • Life expectancies from SuMo-net are calculated by direct numerical integration of the learned survival distribution Sθ(t|x).The passage identifies this as using E[T|x], but the displayed expression is incomplete.Source: §A.4 · p. 13

A.5 Log-likelihood scores for d = 1, 4, 8

A.7 Implementation Details

  • The appendix provides SuMo-net implementation details, including network architecture and optimizer parameters, and points to the attached code base.The supplied passage does not give the parameter values.Source: §A.7 · p. 14

A.6 Other experiments

  • The additional experiments include concordance and IBS scores; SuMo-net is reported as fastest in training speed per epoch on larger datasets.The captions specify that higher concordance and lower IBS are better; they do not expose score values.Source: Table 11 · p. 14, Table 12 · p. 14, Table 13 · p. 15

A.7.1 Hyperparameter space

  • The appendix lists hyperparameter search spaces for Rot. & GBSG, SUPPORT, METABRIC, and FLCHAIN, plus KKBox configurations.The KKBox caption notes that starred parameters were found with a two-layer network with 128 nodes.Source: Table 14 · p. 15, Table 15 · p. 15

A.7.2 Toy datasets

  • Toy data include Weibull, Normal, and Checkerboard distributions; the checkerboard assigns alternating rows of event times across four covariate columns.The checkerboard divides the unit square into a 6 × 4 grid; toy-data parameters are chosen to produce censoring and crossing Normal and Weibull survival curves.Source: Table 16 · p. 15, §A.7.2 · pp. 15–16
Loading 2103.14755v2…