Source-linked AI summary
Random survival forests
Hemant Ishwaran, Udaya B. Kogalur, Eugene H. Blackstone, Michael S. Lauer
TL;DR
The paper addresses survival prediction for right-censored data without relying solely on restrictive conventional modeling assumptions. It introduces random survival forests with survival-specific splitting, ensemble mortality, and missing-data handling, and reports strong performance across real and simulated datasets. The paper also demonstrates complex prognostic relationships in coronary artery disease data.
Problem
Right-censored survival analysis requires methods that can handle nonlinear effects, interactions, and complex prognostic relationships beyond restrictive assumptions such as proportional hazards.
Method
RSF extends random forests with survival-specific tree splitting, conservation-of-events-based ensemble mortality, variable importance, and a missing-data algorithm for covariates and survival data.
Results
Across a wide range of real and simulated datasets, RSF was consistently better than, or at least as good as, competing methods.
Takeaways & Limitations
RSF can uncover complex relationships among renal function, body mass index, and long-term survival in coronary artery disease data.
Takeaways & Limitations
VIMP measures the test-error change when a variable is unavailable to a forest originally grown with it, which differs conceptually from regrowing the forest without that variable.
Abstract
from arXiv · showhide
We introduce random survival forests, a random forests method for the analysis of right-censored survival data. New survival splitting rules for growing survival trees are introduced, as is a new missing data algorithm for imputing missing data. A conservation-of-events principle for survival forests is introduced and used to define ensemble mortality, a simple interpretable measure of mortality that can be used as a predicted outcome. Several illustrative examples are given, including a case study of the prognostic implications of body mass for individuals with coronary artery disease. Computations for all examples were implemented using the freely available R-software package, randomSurvivalForest.
1. Introduction.
The paper introduces random survival forests (RSF), extending random forests to right-censored survival data while addressing nonlinearities, interactions, and restrictive modeling assumptions. It presents the method's outcome definition, empirical evaluation, variable-importance analysis, missing-data algorithm, and coronary-artery-disease case study.
- Motivation and contribution: RSF extends random forests to the analysis of right-censored survival data.The approach follows Breiman’s prescription by incorporating survival time and censoring information throughout forest construction, prediction, and accuracy measurement.
- Motivation and contribution: Survival analysis commonly relies on restrictive assumptions such as proportional hazards and requires specialized modeling of nonlinear effects and interactions.The paper notes that transformations, expanded design matrices, and ad hoc procedures are often used to represent nonlinearities and identify interactions.
- Motivation and contribution: Forests handle these modeling difficulties automatically and can uncover complex relationships among variables in survival data.The coronary artery disease case study examines body mass, smoking, reverse causation, and uncertain risk thresholds.
- Motivation and contribution: RSF differs from off-the-shelf survival-forest approaches because its splitting, prediction, and accuracy procedures explicitly incorporate survival outcomes and censoring.Other approaches may instead reformulate survival analysis within existing classification or regression-tree methodology.
- Objectives and outline: The paper defines ensemble mortality from a conservation-of-events principle and compares RSF prediction accuracy with competing methods across real and simulated datasets.It also investigates variable importance for selection and introduces a missing-data algorithm applicable to training and test data.
2. Random survival forests algorithm.
The RSF algorithm builds an ensemble of fully grown survival trees from bootstrap samples, using randomized candidate variables and survival-based splitting. Tree cumulative hazard functions are averaged, and out-of-bag data support prediction assessment.
- Algorithm overview: RSF draws B bootstrap samples, grows one survival tree per sample, and leaves approximately 37% of observations out of bag on average.The excluded observations are used as out-of-bag data.
- Algorithm overview: At each node, RSF randomly selects p candidate variables and splits on the candidate producing the largest survival difference between daughter nodes.This is the survival-specific split-selection step in the ensemble algorithm.
- Algorithm overview: Each tree grows to full size subject to a minimum of d0 > 0 unique deaths in every terminal node.This constraint prevents terminal nodes from containing no unique deaths.
- Algorithm overview: RSF calculates a cumulative hazard function (CHF) for each tree and averages the tree CHFs to obtain the ensemble CHF.The algorithm then uses out-of-bag data to calculate prediction error.
3. Ensemble cumulative hazard.
RSF predicts cumulative hazard by recursively routing cases into terminal nodes, estimating a Nelson–Aalen CHF there, and averaging predictions across survival trees. Both out-of-bag and bootstrap ensemble CHFs are defined, supporting prediction and error assessment.
- Binary survival tree: Survival trees are binary trees grown recursively, with each split dividing a node into left and right daughter nodes.The process continues until the tree reaches its saturation criterion.
- Binary survival tree: A node split is chosen to maximize survival difference between daughter nodes, separating cases with dissimilar survival.The intended result is increasing homogeneity of survival within later tree nodes.
- Terminal node prediction: A tree reaches saturation when no additional daughters can be formed because each node must contain at least d0 > 0 unique deaths.The extreme nodes of a saturated tree are its terminal nodes.
- Terminal node prediction: Within a terminal node, deaths and individuals at risk at each event time determine a Nelson–Aalen cumulative hazard estimate shared by all cases in that node.The terminal-node CHF is therefore the prediction assigned to cases routed there.
- Terminal node prediction: For a case xi, the tree CHF is obtained by routing xi to its unique terminal node and using that node’s Nelson–Aalen estimator.This defines the CHF prediction for every case and for the tree.
- Ensemble CHF: The ensemble CHF averages tree-specific CHFs, with the OOB estimate averaging trees where the case was out of bag and the bootstrap estimate using all trees.OOB CHFs are formed by dropping OOB cases through trees grown from in-bag data and averaging their terminal-node CHFs.
- Conservation of events: Conservation of events states that summing estimated CHF over observed survival times equals the total number of deaths, including within terminal nodes and across a tree.This principle underlies the paper’s predicted mortality measure.
4. Conservation of events.
The conservation-of-events principle shows that cumulative hazard summed over observed survival times equals the total number of deaths. This motivates ensemble mortality as an interpretable predicted outcome based on expected deaths under similar survival behavior.
- Summing cumulative hazard over observed survival times equals the total number of deaths within each terminal node.The result extends to conservation of deaths within the tree.
- For each tree, conservation of events implies that total deaths are conserved across terminal nodes.The corollary applies to trees grown from the original nonbootstrapped data.
- 4.1. Ensemble mortality: Ensemble mortality is the expected cumulative hazard summed over time for an individual under a null hypothesis of similar survival behavior.It provides a predicted measure of the expected number of deaths for a specific covariate profile.
- 4.1. Ensemble mortality: The survival-tree structure supports ensemble mortality because terminal-node members share a common estimated hazard function.The forest combines these tree-level hazard estimates into an ensemble estimate.
5. Prediction error.
Prediction error is estimated with Harrell’s C-index, which evaluates whether earlier failures receive worse predicted outcomes. The out-of-bag version provides an error estimate without evaluating cases on the data used to fit their relevant trees.
- The C-index estimates the probability that the case failing first has the worse predicted outcome in a randomly selected permissible pair.It is related to the area under the ROC curve.
- 5.1. C-index calculation: C-index calculation forms all case pairs, removes pairs unsuitable under censoring rules, and scores concordant, tied, or discordant predictions.The final index is Concordance divided by Permissible.
- For RSF, the OOB ensemble cumulative hazard function supplies the predicted outcome used to calculate OOB concordance and prediction error.This outcome is analogous to ensemble mortality but is derived from OOB data.
- OOB prediction error is defined as PE** = 1 − C**, with PE** = 0.5 representing prediction no better than random guessing.The measure ranges from 0 to 1.
6. Empirical comparisons.
The empirical comparison evaluates RSF splitting rules against Cox regression and another censored-data forest approach across eleven datasets, including increasingly noisy breast-cancer datasets. Log-rank and log-rank-score RSF generally had the lowest prediction error, while random log-rank splitting offered speed with good performance.
- Eleven datasets, including breast, breast10, breast50, and breast100, were used to evaluate prediction accuracy and high-dimensional noise stability.The noisy breast datasets added 10, 50, or 100 uncorrelated uniform variables.
- RSF used four splitting rules: log-rank, conservation-of-events, log-rank score, and random log-rank.The software implementation grew 1000 trees for each forest analysis.
- Cox regression and RF for censored data were included as benchmark methods, with RF regression growing 1000 trees from IPC-weighted bootstrap samples.Each method used its own predicted outcome when computing the C-index.
- In nearly all examples, RSF with log-rank or log-rank-score splitting had the lowest prediction error, while conservation-of-events splitting also performed very well.Prediction performance was assessed using the C-index.
- Random log-rank splitting performed well in all examples and may be preferred when computational speed is essential because it is significantly faster.
7. Variable importance (VIMP).
VIMP measures the change in test-set prediction error when a variable’s assignments are randomized after the original forest is grown. It is reasonably stable and can identify noise variables, but correlated predictors make its interpretation distinct from refitting a forest without the variable.
- Large VIMP values indicate predictive variables, whereas zero or negative values identify variables that may be filtered.
- VIMP is not the same as the prediction-error change from regrowing a forest without the variable, especially when predictors are highly correlated.A variable can have large VIMP while removing it and refitting leaves prediction error largely unchanged.
- VIMP rankings and relative sizes were reasonably stable across breast-cancer datasets, despite slight changes as noise variables increased.
- The average absolute VIMP for noise variables was 0.001, and breast100 had an average of 0.21 noise variables with VIMP exceeding 0.002.The authors suggest VIMP exceeding 0.002 as a threshold because such values are unlikely to be noise.
8. Missing data.
The paper introduces adaptive tree imputation for forests, using in-bag data during tree growth and enabling prediction with missing test values. Empirically, iterative and proximity imputation performed consistently well, while heavier missingness degraded non-iterated accuracy.
- 8.1. A new missing data algorithm: adaptive tree imputation.: Adaptive tree imputation draws missing values from nonmissing in-bag observations within each working node before splitting.Because out-of-bag data are not used for imputation, the method avoids optimistic bias in out-of-bag prediction error.
- 8.1. A new missing data algorithm: adaptive tree imputation.: The algorithm supports missing covariates and outcomes, and can dynamically impute missing values for test cases as they traverse the forest.Final summary imputations use averages for continuous variables and the most frequent values for integer variables.
- 8.4. Iterative imputation.: With increasing missingness, imputation accuracy may degrade, but iterating the algorithm can substantially improve accuracy.After the initial forest, missing values are updated using out-of-bag summary values before growing a new forest.
- 8.5. Empirical performance.: RMSE was consistently low and out-of-bag error closely matched test-set error for iterative missing-data imputation across all settings.Under moderate missingness, the non-iterated algorithm also produced accurate imputations and closely matched out-of-bag and test-set error.
- 8.5. Empirical performance.: Proximity imputation also produced accurate values and matched out-of-bag error rates to test-set errors in all settings.The authors report no observed bias in out-of-bag error rates for proximity imputation in these experiments.
9. Body mass index and long-term survival among patients undergoing coronary artery bypass grafting (CABG) surgery.
The CABG case study uses RSF to examine how body mass relates to long-term survival while conditioning on renal function, smoking, and graft count. Survival follows a hockey-stick pattern, and the low-BMI survival disadvantage weakens as renal function improves, supporting a reverse-causation hypothesis.
- Study population and analysis: In 15,586 Cleveland Clinic CABG patients, the analysis examined all-cause mortality using 36 variables, including body mass index, renal function, smoking, and graft count.Patients were accrued between 1990 and 2003, with mean follow-up of 6.5 ± 3.8 years.
- Study population and analysis: Predicted five-year survival was estimated from the RSF ensemble and plotted against body mass index, conditioned on smoking history and creatinine clearance.Curves were smoothed with lowess and stratified by the number of internal thoracic artery grafts.
- Body mass and survival: Survival was low at low body mass index, increased up to roughly 25 kg/m2, and then decreased, forming a distinctive hockey-stick pattern.For creatinine clearance above 90 ml/min, indicating healthy renal function, the pattern was much straighter.
- Body mass and survival: The association between low body mass and poor survival weakened as renal function improved, strengthening the hypothesis of reverse causation among underweight patients.The authors state that poor survival at low body mass may reflect systemic effects of renal disease rather than being underweight itself.
- Smoking and graft interactions: Smoking produced a more pronounced hockey-stick pattern without internal thoracic artery grafts, whereas at least one graft masked its lesser effect on survival.Smoking’s effect was also less apparent with abnormal creatinine clearance, where renal dysfunction masked it.
10. Discussion.
RSF combines random-forest ideas with a novel missing-data algorithm and was evaluated across diverse datasets and a complex coronary artery disease case study. The experiments found RSF consistently better than or at least as good as competing methods, while the case study revealed relationships among renal function, body mass index, and long-term survival.
- The missing-data algorithm supports both training and testing data and provides nearly unbiased error-rate estimates even with large amounts of missing data.
- Across a wide range of real and simulated datasets, RSF was consistently better than or at least as good as competing methods.
- The coronary artery disease case study identified relationships among renal function, body mass index, and long-term survival.These relationships helped explain confusion in the literature on the controversial topic.