PaperlayerView exact revision on arXiv

Source-linked AI summary

A review on longitudinal data analysis with random forest in precision medicine

Jianchang Hu, Silke Szymczak

arXiv:2208.04112v1stat.MLcs.LG

TL;DR

Longitudinal omics data create challenges for random-forest prediction because repeated measurements may be clustered, irregularly spaced, or missing. This review categorizes random-forest extensions by response structure and time effects, and reports that several tree- and forest-based extensions improve prediction over standard random forest and linear mixed models in simulations.

  • Problem

    Longitudinal measurements can be clustered, irregularly spaced, or missing, while ignoring their structure may produce spurious subgroups and inaccurate variable selection.

  • Method

    The review categorizes CART-based random-forest extensions by univariate or multivariate responses and whether time effects are modeled.

  • Results

    Tree- and forest-based extensions provided more accurate prediction than linear mixed models and standard random forest in reported simulations.

  • Takeaways & Limitations

    The review identifies multiple random-forest strategies for longitudinal prediction, including subject-level bootstrapping, historical summaries, mixed-effects models, and Bayesian approaches.

  • Takeaways & Limitations

    Most reviewed methods have not been evaluated on high-dimensional longitudinal datasets, limiting evidence for their use with omics data.

Abstract

from arXiv · show

Precision medicine provides customized treatments to patients based on their characteristics and is a promising approach to improving treatment efficiency. Large scale omics data are useful for patient characterization, but often their measurements change over time, leading to longitudinal data. Random forest is one of the state-of-the-art machine learning methods for building prediction models, and can play a crucial role in precision medicine. In this paper, we review extensions of the standard random forest method for the purpose of longitudinal data analysis. Extension methods are categorized according to the data structures for which they are designed. We consider both univariate and multivariate responses and further categorize the repeated measurements according to whether the time effect is relevant. Information of available software implementations of the reviewed extensions is also given. We conclude with discussions on the limitations of our review and some future research directions.

1 Introduction

Precision medicine uses patient characteristics and longitudinal molecular, demographic, and clinical data to tailor treatments and improve treatment efficiency. Because repeated measurements create dependent, variably spaced, and potentially incomplete observations, this review examines extensions of random forests for longitudinal prediction.

  • Precision medicine aims to customize treatments using patient characteristics to improve treatment efficiency while avoiding serious side effects.
  • Longitudinal data arise when patient measurements, including molecular, demographic, or clinical predictors, are collected repeatedly over time.Predictors may be measured once, remain time-invariant, or be collected at multiple study time points.
  • Longitudinal analysis is challenging because repeated observations are clustered and correlated, may occur at irregular intervals, and can contain missing values.Correlation can reflect patient-specific characteristics and serial dependence over time.
  • Random forest supports prediction with categorical, quantitative, or survival outcomes, varied predictor scales or distributions, and high-dimensional data.However, standard random forest assumes independently sampled observations, so ignoring longitudinal dependence can bias inference through underestimated standard errors.
  • The review presents CART-based random forest extensions for longitudinal data, focusing on predicting categorical and quantitative outcomes.The review considers extensions organized by longitudinal data structure, including univariate and multivariate responses and whether time effects are relevant.

2 Univariate response longitudinal data

This section examines longitudinal data with a univariate response, beginning with standard random forests and motivating extensions tailored to repeated measurements. It then presents and discusses several such extension methods.

  • 2 Univariate response longitudinal data: The section focuses on the simple case of a univariate response, corresponding to m = 1 in Table 1.
  • 2 Univariate response longitudinal data: It briefly reviews the standard random forest algorithm as a prediction model before considering extensions for longitudinal data analysis.
  • 2 Univariate response longitudinal data: Several random forest extension methods are presented and discussed to address the repeated measurements that characterize longitudinal data.

2.1 Standard random forest algorithm

Standard random forest ensembles bootstrapped decision trees whose splits minimize node impurity, then aggregate tree-level predictions by voting or averaging. Although useful for prediction and variable importance, standard RF can mishandle longitudinal clustering, motivating extensions that modify tree construction.

  • Standard random forest algorithm: Random forest builds an ensemble of decision trees from bootstrapped training samples, repeatedly partitioning nodes to minimize response impurity.Impurity is commonly measured by the Gini index for categorical responses and variance for quantitative responses.
  • Standard random forest algorithm: For prediction, each observation reaches one leaf in every tree, and the forest combines tree outputs through majority voting for classification or averaging for regression.
  • Standard random forest algorithm: Out-of-bag samples provide an estimate of forest prediction error because bootstrap sampling leaves some observations out of each tree’s construction.
  • Standard random forest algorithm: Random forest supports prediction and variable-importance measurement, which can aid predictor selection in high-dimensional omics data, but its ensemble results are difficult to interpret.Its predictive power comes from aggregating weaker learners, especially when correlations between trees are low.
  • Standard random forest algorithm: Applying standard RF directly to longitudinal data can produce correlated or homogeneous trees, overly optimistic OOB error, spurious subgroups, and inaccurate predictor selection because subject-level clustering is ignored.The review therefore motivates RF extensions that mainly modify decision-tree construction to better fit longitudinal data.

2.2 Clustered data

Clustered repeated measurements share subject-specific means but have little time effect, creating dependence among observations. Averaging replicates restores the standard one-observation-per-subject structure but loses intra-class variation and masks imbalance, motivating subject-level bootstrap extensions such as RF++.

  • Clustered data: Clustered data arise when multiple measurements are collected from the same subject in a single visit, with observation ordering largely irrelevant because time has little effect.The clustering effect results from observations sharing the same subject mean, while residual fluctuations are independent.
  • Clustered data: Averaging replicated measurements restores one observation per subject and the independence required by standard RF.Vlahou et al. (2004) used this approach for urine mass-spectrometry protein profiling.
  • Clustered data: Averaging loses intra-class variation and masks imbalance because subjects may contribute different numbers of observations carrying distributional information.After averaging, each subject contributes equally despite potentially different original replicate counts.
  • Clustered data: Standard RF bootstrapping can produce correlated or homogeneous trees and underestimate OOB prediction error when clustered observations are reused.These problems motivate extensions that use all observations while accounting for subject-level dependence.
  • Clustered data: RF++ replaces observation-level resampling with subject-level bootstrapping, while two-stage resampling selects one observation per chosen subject to further reduce intra-cluster correlation effects.Simulation studies reported that subject-level resampling with one observation per subject yielded the best prediction results compared with standard RF.

2.3 Time effects considered

Time-aware random-forest extensions use prior measurements or explicitly model within-subject dependence to improve longitudinal prediction. The section reviews historical RF methods and tree/forest extensions of mixed-effects models, including their computational and high-dimensional limitations.

  • Historical RF: Historical RF models predict each response using concurrent predictors together with all preceding observations from the same subject.This explicitly incorporates predictor history into longitudinal prediction.
  • Historical RF: Time-varying predictor histories are summarized over prior intervals before standard tree splitting selects variables using Gini impurity or sum-of-squares error.Summary functions can count past observations or use relative frequencies above thresholds.
  • Historical RF: Optimizing historical-summary parameters can substantially increase computation, particularly when many time-varying predictors occur in omics datasets.Random subsampling within intervals and subject-level resampling are used to mitigate this optimization burden.
  • Mixed-effects extensions: Mixed-effects RF extensions replace the fixed-effects linear component of (G)LMMs with a tree or forest while retaining random effects for dependence modeling.Common approaches include MERT and RE-EM trees, with Bayesian BiMM proposed for binary responses.
  • Mixed-effects extensions: Simulation studies found MERT- and RE-EM-based tree and forest algorithms applicable to high-dimensional data and more accurate than LMM and standard RF.The Bayesian approach can avoid convergence issues in high-dimensional data, whereas some frequentist approaches lack high-dimensional solutions.

3 Multivariate response longitudinal data

This section reviews random-forest extensions for multivariate longitudinal responses, including jointly modeled repeated measurements and multiple response types. These methods primarily modify tree-splitting impurity criteria to accommodate inter-dimensional dependence, while distance-based approaches can handle irregular observation times but complicate prediction.

  • Scope and data structure: Multivariate longitudinal responses can represent multiple health-related outcomes or repeated measurements treated jointly as one multidimensional response.The latter may also be viewed as a discretized response curve.
  • Methodological strategies: Random-forest extensions primarily modify node impurity and split criteria, incorporating covariance structure to account for correlations among response dimensions.The reviewed modifications are broadly categorized into distance-based and likelihood-based approaches.
  • Distance-based methods: Distance-based methods include covariance-weighted criteria such as Mahalanobis distance and pairwise-dissimilarity criteria based on a distance matrix.The distance-matrix approach is more general because its distance need not depend on the original response dimension.
  • Distance-based methods: Distance-matrix methods can analyze longitudinal responses observed at irregular time points when an appropriate distance measure is available.Prediction requires further consideration because leaves may contain responses with different dimensions.
  • Likelihood-based methods: Likelihood-based extensions use node log-likelihood or deviance for multivariate exponential-family, binary, ordinal, and normally distributed responses.For multivariate normal responses, the deviance-based impurity is equivalent to a Mahalanobis-distance form, as are MANOVA statistics such as Hotelling’s T^2.

4 Implementation

Table 3 summarizes software implementations for reviewed random-forest extensions, including access, problem type, tree or forest output, and variable-importance support. Nearly all are publicly available through R, but gaps remain in access, importance measures, and comparative efficiency studies.

  • Software implementations: Table 3 lists each extension’s implementation source, solvable problem type, tree or random-forest output, and available variable-importance measure.The table provides package or code locations alongside these implementation characteristics.
  • Software implementations: Almost all extensions are available through an R package on CRAN or an R program, whereas MERT and MERF require direct author contact.No public access is available for the MERT and MERF programs.
  • Limitations and future directions: Most extensions lack variable-importance measures, and no systematic comparisons or efficiency studies have been conducted for the listed packages and programs.The review identifies variable-importance support as a future research direction for longitudinal random-forest analysis.

5 Discussion

The review highlights how CART-based random forests can support longitudinal-data applications in precision medicine while identifying important methodological gaps. Future work should address missing values, longitudinal variable importance, high-dimensional data, and systematic benchmarking.

  • Scope and motivation: CART-based random forests may support patient stratification, disease progression prediction, and target biomarker identification, but repeated measurements induce clustering that negatively affects standard analysis.The review focuses on extensions of CART-based random forest for longitudinal data in clinical trials and precision medicine.
  • Methodological considerations: Subject-level bootstrap methods ignore time effects and random effects, whereas historical random forests summarize observation histories with concurrent observations to model conditional response means.Bootstrap resampling works well when observation ordering is unimportant but has limited application for prediction because it omits random effects.
  • Limitations: The review is limited to CART-based methods, although GUIDE and conditional inference approaches could require different longitudinal-data extensions.MERT and MERF can readily switch tree or forest construction approaches, while other extensions may not.
  • Future research: Missing values remain a major challenge because surrogate splitting may not suit all missing mechanisms, making tailored handling an important research area.Different missing mechanisms may require different approaches in statistical and machine-learning methods.
  • Future research: Future research should develop longitudinal variable-importance measures, handle high-dimensional data, and benchmark random-forest extensions for practitioners.Permutation-based importance is computationally expensive, correlated predictors may impair variable deletion, most methods lack high-dimensional evaluation, and systematic benchmark studies have not been published.
Loading 2208.04112v1…