Source-linked AI summary
ranger: A Fast Implementation of Random Forests for High Dimensional Data in C++ and R
Marvin N. Wright, Andreas Ziegler
TL;DR
Existing random-forest implementations had limitations for high-dimensional data, motivating ranger, a modular C++ and R framework optimized for such analyses. Across simulated scenarios, ranger outperformed the other packages and achieved the lowest runtime and memory usage in GWAS-mode analyses.
Problem
Existing random-forest implementations had differing strengths and weaknesses, with several not optimized for datasets containing many features.
Method
The paper introduces ranger, a modular C++ application and R package optimized for high-dimensional random-forest analysis through runtime- and memory-profiled algorithms.
Results
ranger outperformed the other four packages across simulated classification scenarios and was fastest in all three large-scale GWAS modes.
Takeaways & Limitations
The package provides a fast, memory-efficient option for high-dimensional random-forest analyses, with a GWAS mode that minimizes both runtime and memory usage.
Takeaways & Limitations
There is no single best random-forest implementation for all large datasets because packages are optimized for different feature and sample characteristics.
Abstract
from arXiv · showhide
We introduce the C++ application and R package ranger. The software is a fast implementation of random forests for high dimensional data. Ensembles of classification, regression and survival trees are supported. We describe the implementation, provide examples, validate the package with a reference implementation, and compare runtime and memory usage with other implementations. The new software proves to scale best with the number of features, samples, trees, and features tried for splitting. Finally, we show that ranger is the fastest and most memory efficient implementation of random forests to analyze data on the scale of a genome-wide association study.
1. Introduction
Random forests are widely used across biological, financial, and image-processing applications, but existing implementations have important limitations for high-dimensional data. The authors therefore introduce ranger, a platform-independent and modular RF framework designed for high-dimensional analysis, simple use in R, and performance at least comparable to Random Jungle 2.1.
- Random forests are widely used for gene expression, protein-protein interaction, biological sequence, genome-wide association, credit-scoring, and image-processing analyses.
- Existing RF implementations have distinct strengths and weaknesses, including recompilation requirements in Breiman and Cutler’s Fortran 77 implementation.The R package randomForest is feature-rich and widely used, but is not optimized for high-dimensional data; the passage also notes this limitation for other implementations.
- The authors introduce ranger as a platform-independent, modular framework for analyzing high-dimensional data with random forests.The software is also available as an R package under the same name.
- Ranger was designed to be simple to use in R while achieving computational performance not worse than Random Jungle 2.1.
2. Implementation
ranger is implemented in portable C++11 with cross-platform parallel processing and exposed as an R package through Rcpp. Its implementation supports diverse random-forest analyses, memory-efficient genetic data handling, defined split criteria, and profiling-driven optimization for high-dimensional data.
- Core implementation: The core uses standard-library C++ and C++11 features, including cross-platform threading for parallel processing and the random library for number generation.A standalone C++ version can be installed on any up-to-date platform.
- R integration: Rcpp exposes ranger as an R package installable with a single command, while keeping all data in R and supporting most randomForest features plus new ones.The passage states that external file handling is unnecessary.
- Genetic data: GenABEL objects can be loaded and analyzed directly, with genotypes stored in a memory-efficient binary format and genetic data mixable with dichotomous, categorical, and continuous variables.This supports analyses combining clinical variables with GWAS genotypes.
- Split criteria: Classification and regression forests use node-impurity decrease for splitting, survival forests use the log-rank test, and impurity is measured by the Gini index or estimated response variance.For probability estimation, trees are grown as regression trees.
- Optimization: Runtime and memory profiling identified node splitting as the crucial bottleneck because all values of all mtry candidate features must be evaluated as split candidates.ranger uses two splitting algorithms, including one that sorts feature values beforehand and accesses them by index.
- Software architecture: ranger is open-source software under the GNU GPL-3 license, with a modular implementation that facilitates adding or contributing tree types, splitting criteria, and other features.The passage explicitly describes extensibility for both the authors and other developers.
3. Usage and examples
The R package provides ranger() for growing forests and predict() for predicting new data, with similar interfaces across classification, regression, and survival forests. The C++ version offers equivalent computational speed and memory usage, but GWAS-specific memory-efficient storage is available only in R, which is generally recommended.
- R usage: The R interface centers on ranger() for growing forests and predict() for predicting responses for new datasets.Models use R’s formula interface and data.frame datasets.
- R usage: Permutation importance is enabled through the importance option, while training-test workflows use ranger() on training data and predict() with the resulting forest.The example samples 100 of 150 iris observations for training and compares test predictions with observed classes.
- Forest types: The tree type is determined by the dependent variable: factors yield classification trees, numeric values regression trees, and survival objects survival trees.A dichotomous 0/1 endpoint must be converted to a factor for classification, and survival forests use Surv().
- C++ usage: R and C++ have equal computational speed and memory usage, but extreme memory-efficient GWAS storage is available only in R, so the authors generally advise using R.The C++ interface requires compilation or an executable, uses ASCII files, and writes predictions to ranger_out.prediction.
4. Validation
ranger was validated against the R package randomForest using identical settings, comparing out-of-bag prediction error and variable-importance results. Simulation studies found no systematic difference in classification out-of-bag errors and showed closely aligned importance results.
- ranger and randomForest were compared using identical settings for out-of-bag prediction error and variable importance.
- No systematic difference was observed between the packages’ out-of-bag prediction errors across 200 simulated classification datasets.Each dataset contained 2000 samples and 50 features, with 5 effect features and 45 noise features; both forests used 5000 trees.
- Gini and permutation importance results were very similar for simulated data with 5 effect features and 45 noise features.The study grew 10,000 random forests with 500 trees each and used node impurity as the split criterion.
5. Runtime and memory usage
ranger was generally the fastest implementation across high-dimensional classification and regression benchmarks, including a GWAS-scale simulation where its GWAS mode minimized both runtime and memory usage. However, no implementation was best for every dataset: Rborist was faster for continuous features in low-dimensional data with large sample sizes.
- Classification benchmarks: ranger outperformed the other packages across all simulated classification scenarios, including variations in trees, features, samples, and mtry.Random Jungle was faster than randomForestSRC, while randomForestSRC generally exceeded randomForest except at high mtry values.
- Regression benchmarks: ranger was fastest in every simulated regression scenario, although Rborist scaled better than the other packages at high mtry values.Regression runtimes were generally lower than classification runtimes; Rborist was otherwise slowest, while randomForest, randomForestSRC, and Random Jungle were approximately equal.
- GWAS-scale analysis: In the GWAS-scale simulation, Rborist and bigrf without disk caching failed from memory shortage, whereas all other packages completed the analysis.The dataset contained 10,000 subjects and 150,000 features, with 1000 trees and mtry values of 5000, 15,000, and 135,000 features.
- GWAS-scale analysis: ranger was fastest in all three GWAS modes, and its GWAS mode achieved the lowest runtime and memory usage among the compared packages.The save.memory option reduced memory usage but increased runtime; Random Jungle used very little memory, while randomForestSRC used much more.
- Low-dimensional analysis: For low-dimensional data, ranger was fastest with features having few unique values, but Rborist was faster for continuous features and large sample sizes.Across 441 continuous-feature comparisons, ranger was faster for all feature counts when sample sizes were below 25,000; above that, a threshold emerged.
- Practical implications: The benchmarks indicate that runtime depends on dataset properties: Rborist suits low-dimensional data with n > 25,000, while ranger is preferred in other cases.The paper also identifies Random Jungle as optimized for GWAS data and emphasizes that no single implementation is best for every large dataset.
6. Conclusions
The paper introduced ranger, a C++ application and R package for random forests, and showed that its predictions and variable-importance measures closely matched a standard implementation. Simulation studies demonstrated ranger’s computational and memory efficiency, with runtime scaling favorably across key problem dimensions.
- ranger’s out-of-bag prediction errors and variable-importance measures were very similar to those from a standard implementation, supporting its validity.
- Simulation studies demonstrated ranger’s computational and memory efficiency, with runtime scaling best for features, samples, trees, and mtry.
- All computations used an exclusive 64-bit Linux node with two 2.7 GHz Intel Xeon CPUs, 16 cores, 128 GByte RAM, and R 3.1.2.
A. Additional figures
The additional figures validate ranger against randomForest for regression forests and examine both variable-importance measures and runtime across key dataset and model parameters.
- Validation: Regression-forest predictions from ranger and randomForest are compared using out-of-bag error correlations and Bland–Altman analysis.The Bland–Altman plot shows the mean difference and limits at ± 1.96 SD of the difference.
- Variable importance: Gini and permutation importance for five simulated non-zero-effect variables are compared between ranger and the randomForest R package.Boxplots display medians, quartiles, and largest non-outliers for both importance measures.
- Runtime analysis: Runtime is analyzed for ranger and four alternative implementations while varying trees, features, samples, and the percentage of features tried for splitting.Each runtime measures growing one forest, with results averaged over 20 simulation repeats.
B. Multicore randomForest code
The multicore wrapper divides the requested trees across cores, fits randomForest models in parallel, and combines the resulting forests.
- Function interface: mcrf accepts response data, predictors, tree count, core count, and additional arguments.The wrapper is defined as mcrf <- function(y, x, ntree, ncore, ...).
- Tree allocation: The wrapper distributes ntree as evenly as possible across ncore cores, assigning remainder trees across the first cores.It computes ntree%/%ncore for each core and adds one tree to ntree%%ncore cores.
- Parallel fitting and combination: It fits one randomForest model per allocation using mclapply and combines the resulting forests with do.call(combine, rfs).Parallel execution is controlled by mc.cores = ncore.