Randomization-based Machine Learning in Renewable Energy Prediction Problems: Critical Literature Review, New Results and Perspectives

• Intermittency limits renewable-energy integration, so accurate short- to medium-term production predictions are needed for grid compatibility.Source: §1 · p. 2
• Randomization-based methods use randomized structures or parameters and have grown because they combine strong prediction performance with extremely fast training.Source: §1 · p. 3
• The paper reviews randomization-based methods and their applications to solar, wind, marine/ocean, and hydropower prediction.Source: §1 · p. 3
• Across the reported case studies, randomization-based approaches outperform previous methods while requiring very little computation time.Source: §1 · p. 3
• The study evaluates large batteries of randomization-based algorithms against state-of-the-art and classical machine-learning methods on solar radiation, wind speed, and reservoir water-level prediction.Source: §1 · pp. 3–4

• Randomization-based ML methods randomize parts of training to improve accuracy, avoid overfitting, or make models more robust to outliers.Source: §2 · p. 4
• Randomization may affect training-set selection, model topology, or parameter initialization.Source: §2 · p. 4
• Bagging randomizes training data through bootstrap aggregation, while Echo State Networks randomize recurrent links.Source: §2 · p. 4
• ELM and RVFL networks randomly initialize weights, providing another way to exploit randomization when constructing data-based models.Source: §2 · p. 4

2.1. Ensemble methods

• Ensemble methods combine simpler coordinated learners to improve predictive performance, robustness, and generalization relative to individual complex models.Source: §2.1 · pp. 4–5

2.1.1. Bagging

• Bagging trains same-architecture models independently on bootstrap samples, then combines their outputs by averaging for regression or voting for classification.Source: §2.1.1 · pp. 5–6
• Independent bagging learners can be trained in parallel, substantially reducing ensemble training time.Source: §2.1.1 · p. 6
• Random Forest extends bagging with trees that may differ in length, topology, or input variables, increasing learner variability before voting or averaging predictions.Source: §2.1.1 · p. 6

2.1.2. Boosting

• Boosting trains same-topology learners sequentially on the full dataset, unlike bagging’s parallel bootstrap training.Source: §2.1.2 · p. 7
• Boosting reweights training examples according to previous prediction accuracy and usually weights learner contributions by their accuracy.Source: §2.1.2 · pp. 7–8
• AdaBoost focuses each successive learner on samples misclassified by its predecessor and supports binary, regression, and multiclass variants.Source: §2.1.2 · pp. 8–9

• ELM randomly sets first-layer weights, computes the hidden-layer matrix, and obtains output weights by multiplying its Moore-Penrose pseudoinverse by the target.Source: §2.2 · pp. 10–11
• ELM can outperform the training efficiency of multilayer perceptrons and Support Vector Machines while achieving competitive results.Source: §2.2 · p. 10
• ELM training constructs H from randomly assigned weights and biases, applies an activation function, and reduces learning to output-weight optimization.Source: §2.2 · p. 11
• The ELM output for a new input is computed as Hβ after the output-weight vector has been determined.Source: §2.2 · p. 11
• The hidden-node count is a free parameter that must be selected through validation over a scanned range.Source: §2.2 · p. 11

2.3. Multihidden-Layer ELMs (ML-ELM)

• ML-ELM addresses vanishing gradients by avoiding iterative hidden-layer weight adjustment during training.Its construction is described as simpler than backpropagation-trained multilayer networks.Source: §2.3 · p. 11
• The first hidden layer applies a nonlinear transformation after random assignment of input weights and biases, then computes output weights with a Moore-Penrose pseudoinverse.The initial output weights are β = H†Y^T.Source: §2.3 · p. 12
• Subsequent hidden layers use the training output and previous representations to calculate new hidden outputs, interlayer weights, and updated output weights.The procedure updates β after constructing each additional hidden layer.Source: §2.3 · p. 12
• For K hidden layers, the algorithm repeats the intermediate steps K −1 times and produces the final prediction as ˆY (x) = Hkβ.The final output uses the representation from the last hidden layer.Source: §2.3 · p. 12

2.4. Random Vector Functional Link Networks (RVFL)

• RVFL fixes randomly initialized hidden-layer weights and biases, training only the output-layer weights to minimize estimation error.This preserves the fast, non-iterative hidden-layer training characteristic described for ELM-like methods.Source: §2.4 · pp. 12–13
• The hidden-layer matrix is concatenated with the training features as P = [H;X], and output weights are obtained by pseudoinverse or ridge regression.The pseudoinverse gives β = P†Y, while regularized least squares uses the ridge formulation.Source: §2.4 · p. 13
• Unlike ELMs, RVFL output nodes receive both nonlinear hidden-layer features and the original input features through direct links.With n input features and Ñ hidden neurons, each output node receives n + Ñ inputs.Source: §2.4 · p. 13
• RVFL and ELM have similar internal structures, but their distinction centers on whether direct input-to-output links are included.The paper notes an ongoing debate about the novelty and pioneering contribution of the two techniques.Source: §2.4 · p. 13

2.5. Deep RVFL (dRVFL)

• dRVFL extends RVFL with multiple hidden layers whose inputs are formed from predecessor-layer outputs.The network allows flexible choices for the number of layers and neurons per layer.Source: §2.5 · p. 14
• The network randomly assigns and freezes weights and biases, then computes the first hidden representation as H(1) = g(XW(1)).Here X is the input training-data matrix and g(⋅) is the neuron activation function.Source: §2.5 · pp. 14–15
• For deeper layers, dRVFL recursively computes H(K) = g(H(K−1)W(K)) and trains only the output weights β.The final prediction is defined as ˆY (x) = Iβ.Source: §2.5 · p. 15

2.6. Ensemble dRVFL (edRVFL)

• edRVFL addresses dRVFL’s potentially excessive time and memory demands by decomposing β into independently learned smaller submodels βed.The approach is motivated for settings with large amounts of data.Source: §2.6 · p. 15
• Each edRVFL layer combines a nonlinear transformation of the predecessor output with the original input features, retaining RVFL-style direct access to inputs.This differs from dRVFL, where each layer input is formed from the predecessor layer output.Source: §2.6 · p. 15
• The final ensemble averages intermediate outputs for regression and uses majority voting for classification.Output weights are solved independently for the component models.Source: §2.6 · p. 15

Architecture

• ESNs use input, recurrent reservoir, and output layers to model long-term sequence relationships without backpropagation-based training drawbacks.The reservoir nonlinearly expands input dimensions, and predictions combine reservoir states with trained output weights.Source: §2.7 · p. 16

Dynamics

• The recurrent update combines current inputs, previous reservoir states, and feedback through activation function g(⋅), with leaky rate α controlling response speed.The leaky rate adjusts how quickly reservoir states react to changes in input dynamics.Source: §2.7 · pp. 16–17

Training

• Only the output weights Wout are trained, using a Moore-Penrose pseudoinverse or regularized least-squares optimization.Input, recurrent, and feedback weights are initialized randomly and kept fixed during training.Source: §2.7 · pp. 16–17

Inference

• After output weights are fitted on training sequences, ESNs generate predictions for subsequent input points using the learned output mapping.The output combines the current reservoir state and input before applying the output activation function.Source: §2.7 · p. 17

Reservoir design

• Reservoir size and connectivity must balance accuracy against complexity, while scaling W helps enforce stable echo-state dynamics.The echo state property requires input effects on reservoir outputs to fade over time and is associated with conditions involving spectral radius ρmax < 1.Source: §2.7 · pp. 17–18

Architecture

• Deep ESNs stack standard ESNs into multiple reservoirs, enabling feature representations at different temporal scales and granularities.Successive reservoirs receive preceding reservoir outputs, increasing nonlinear and multiscale representation of complex sequences.Source: §2.8 · pp. 18–20

Reservoir hierarchy

• A dESN comprises K sequentially connected reservoirs, each with its own input, recurrent, feedback, and leaky-rate parameters.The first reservoir receives the original input, while later reservoirs process preceding reservoir states.Source: §2.8 · p. 19

Output layer

• The standard dESN output concatenates hidden states from all reservoirs before computing the output weight matrix Wout.Alternative formulations may use only the final reservoir's hidden state.Source: §2.8 · p. 19

Representation

• Compared with standard ESNs, dESNs increase the level of nonlinear and granular feature mapping available for sequences with high nonlinear complexity.Their main advantage is capturing patterns at multiple representation scales as data propagates through the stacked reservoirs.Source: §2.8 · pp. 19–20

3.1. Bibliographic methodology for the literature review

• The review systematically searched Scopus, Web of Science, and Google Scholar using method- and renewable-energy-specific queries, removed duplicates, and assessed papers individually for relevance.Figure 6 summarizes the reviewed works and guides the section’s structure.Source: §3.1 · pp. 20–21, Figure 6 · p. 21

3.2.1. ELMs in solar radiation prediction problems

• ELM studies report strong solar-radiation prediction across satellite, meteorological, and PV inputs, frequently outperforming back-propagation networks, SVR, MLPs, and related baselines.Applications span Turkey, Indonesia, Iran, Burkina Faso, Australia, Spain, and other locations, using variables including temperature, humidity, wind, geography, and PV output.Source: §3.2.1 · pp. 21–22

3.2.2. ELMs in PV power systems production prediction

• ELM-based PV forecasting studies cover weather-conditioned, day-ahead, hourly, and 24-hour-ahead prediction, with optimized, pruned, decomposed, and wavelet-KELM variants evaluated against neural and support-vector models.Reported extensions include entropy preprocessing, PSO weight optimization, node pruning, and gravitational-search optimization for PV forecasting and diagnosis.Source: §3.2.2 · p. 24

3.2.3. Hybrid ELMs techniques in solar radiation prediction problems

• Hybrid ELM solar models combine metaheuristic optimization, feature selection, signal decomposition, satellite or reanalysis data, and similar-day analysis to improve radiation or PV-power prediction.Examples use CRO, grouping genetic algorithms, mutual information, differential evolution, ant-colony optimization, VMD, wavelets, bat algorithms, GA, and PSO.Source: §3.2.3 · pp. 24–25

3.2.4. RF approaches in solar radiation prediction

• RF approaches achieve leading solar-forecasting accuracy in several reviewed comparisons, while ensemble and optimized RF models improve predictions across radiation, solar thermal, PV, and geographically distributed energy tasks.RF is reported as best among compared methods for 1-to-6-day hourly forecasting in India and for all tested horizons and radiation components in France; RF and Extra Trees also obtain the best accuracy in a French solar-thermal study.Source: §3.2.4 · pp. 26–28

3.2.5. ESNs in solar radiation prediction

• ESN research addresses multi-step and multi-timescale solar forecasting, with ESNs outperforming ELMs in one PV study and multiple-reservoir or chain-structured ESNs surpassing conventional recurrent, Elman, and classical ESN models.The reviewed systems use real-time forecasting, restricted-Boltzmann-machine features, PCA, multitask learning, and spatial autocorrelation to improve representations and scalability.Source: §3.2.5 · pp. 28–29

3.2.6. RVFL networks in solar energy prediction

• Only two RVFL solar-prediction studies were identified: a standard RVFL outperformed SLFN and RWSLFN baselines, while a kernelized RVFL used local and global kernels plus water-cycle optimization.The kernelized model retained RVFL’s fast training and reduced architecture while targeting improved short-term solar-power accuracy at two U.S. plants.Source: §3.2.6 · p. 29

3.3.1. ELMs in wind speed prediction

• Wind studies apply ELMs to speed estimation, turbine control, wind-power density, and forecasting, while hybrid methods use decomposition, feature selection, optimization, ensembles, error correction, and probabilistic modeling.Decomposition-based hybrids generally improve on single ELM models; reported variants include ELM-CRO, ELM-wavelet, optimized KELM, OR-ELM, AdaBoost.RT-ELM, and multi-stage CEEMDAN-VMD-ELM systems.Source: §3.3.2 · pp. 31–34, §3.3.1 · p. 31

3.3.2. Hybrid ELM models for wind speed prediction

• ELM-based hybrids combine decomposition, cross-validation, Grey systems, or artificial bee colony optimization for multi-step, long-term, and short-term wind-speed forecasting.Reported applications include Yunnan, China, and Zanjan, Iran, using forecasting horizons from 24 hours to long-term behavior.Source: §3.3.2 · p. 35

3.3.3. RF for wind speed prediction problems

• RF methods improve wind forecasting through ensembles, seasonal inputs, feature selection, ANN-derived features, wavelets, and optimization, outperforming or improving classical neural, SVR, and persistence baselines.Applications span individual wind farms, larger geographical scales, offshore platforms, and multiple-hour-ahead forecasting.Source: §3.3.3 · pp. 35–36

3.3.4. ESNs for wind speed prediction problems

• ESN wind-prediction systems use wavelets, PCA, genetic or multi-objective optimization, LSTM blocks, ensembles, and deep architectures for speed, direction, ramp-event, and power forecasting.These studies report favorable comparisons with ESN, ANFIS, and conventional methods while reducing training, tuning, or computational demands in some designs.Source: §3.3.4 · pp. 37–38

3.3.5. RVFL for wind speed prediction problems

• RVFL approaches address wind-ramp detection and multi-step speed or power forecasting using Chebyshev expansions, EMD, Legendre-RBF networks, and firefly optimization.The reviewed systems were evaluated on real datasets from the United States, Spain, and other wind-farm locations.Source: §3.3.5 · pp. 38–39

3.4.1. ELMs in marine and ocean energy

• Marine-energy studies primarily use hybrid or ensemble ELMs for wave height, wave-energy flux, and related prediction tasks, with Bayesian optimization improving GGA-ELM solution quality by over 6%.ELM slightly outperformed MLPs and SVRs in one wave-height modeling comparison.Source: §3.4.1 · pp. 39–40

3.4.2. RF in marine and ocean energy

• RF and gradient-boosting methods are relatively uncommon in marine energy but improve wave-height forecasting over ANNs, including when combined with physics-based inputs and Bayesian hyperparameter optimization.Applications include electricity prediction at the Mutriku wave farm and significant-wave-height forecasting in Biarritz.Source: §3.4.2 · p. 40

3.5.1. ELMs in hydro-power prediction

• Hydro-power applications use ELM, KELM, and ESN ensembles for discharge, streamflow, reservoir operation, production capacity, power, and leakage forecasting, incorporating decomposition, PSO, Monte Carlo expansion, and multi-objective optimization.The studies address robustness, limited data, physical trends, variable sensitivity, and bias–variance tradeoffs across reservoirs and rivers in China, Malaysia, Brazil, Denmark, and the United States.Source: §3.5.1 · pp. 40–42

3.6. Critical literature analysis

• The literature’s rapid growth since 2014 is driven by randomization-based methods’ balance between predictive performance and computational cost, but many studies lack novelty and rigorous statistical testing.The review also notes that practical factors such as interpretability, physical compliance, computational efficiency, and epistemic uncertainty are often overlooked.Source: §3.6 · pp. 42–43

• The experimental study applies randomization-based ML algorithms to prediction problems in solar radiation, wind speed, and water level.Source: §4 · p. 44
• Hyperparameter values are specified for each randomization-based model, while competing non-randomization methods use settings from their original studies.Source: §4 · p. 44, Table 1 · p. 44

4.1.1. Dataset description

• The Toledo dataset contains one year of hourly global radiation observations and 39 inputs combining reflectivity, cloud indices, clear-sky information, and Heliosat-2 and CAMS-based models.Source: §4.1.1 · pp. 45–46, Table 2 · p. 46

4.1.2. Experimental results

• The comparison includes alternative regressors from and, including SVR, GPR, and evolutionary neural-network configurations.Source: §4.1.2 · p. 47
• dRVFL achieves the best overall solar-radiation result, with MAE = 36.82 W/m2 and R2 = 0.968, outperforming every compared model.Source: §4.1.2 · p. 47, Table 3 · p. 48
• Randomization-based models generally obtain MAE under 40 W/m2 and R2 > 0.96, except AdaBoost, which performs slightly worse than the other randomized approaches.Source: §4.1.2 · p. 47
• Randomization-based algorithms clearly outperform SVR and GPR, while remaining competitive with evolutionary neural networks that require more difficult training.Source: §4.1.2 · p. 47
• Figure 7 compares true and predicted values for RFR, ML-ELM, and dRVFL, with dRVFL producing the most compact scatter and improvements on specific samples.Source: §4.1.2 · p. 47, Figure 7 · p. 48
• Wilcoxon testing identifies dRVFL as the best-over-all randomized algorithm, with statistically significant differences from the other tested algorithms.Source: §4.1.2 · pp. 47–48, Table 4 · p. 49

4.2.1. Dataset description

• The dataset comprises long hourly wind-speed series from farms A, B, and C across northern, central, and southern Spain, with distributions approximated by Weibull models.Source: §4.2.1 · pp. 49–50, Figure 8 · p. 50, Figure 9 · p. 50

4.2.2. Experimental results

• Forecasts use only previous wind-speed observations for horizons up to 8 hours, whereas longer horizons require exogenous atmospheric variables.Source: §4.2.2 · p. 51
• At 1 hour, persistence dominates overall, although randomized methods slightly improve it, including edRVFL with R2 = 0.835 at farm A.Source: §4.2.2 · p. 51
• At 4 hours, edRVFL exceeds persistence at farms A, B, and C with R2 values of 0.464 versus 0.395, 0.565 versus 0.551, and 0.705 versus 0.676.Source: §4.2.2 · p. 51
• At 8 hours, randomized methods outperform persistence, but predictions remain poor without atmospheric information; edRVFL, ESN, and dESN are best at farms A, B, and C respectively.Source: §4.2.2 · p. 52
• Wilcoxon tests show randomized methods generally outperform alternatives at farms B and C, while farm A shows statistical similarity among traditional ML methods.Source: §4.2.2 · p. 52
• Performance worsens as the horizon grows, and the 8-hour forecasts are clearly poor across the illustrated farms without atmospheric-state information.Source: §4.2.2 · p. 52

4.3. Dammed water level prediction for hydro-power

• The experiment compared randomized methods with classical SVR, MLP, and GPR for short-term prediction of the Belesar reservoir level.Source: §4.3 · pp. 52, 56, §4.3.1 · p. 59

4.3.1. Dataset Description

• Predictors included upstream and tributary height and flow, precipitation, electricity-generation water use, snow depth, and spring snowmelt.The target was Belesar reservoir water level, measured in hm3.Source: §4.3.1 · pp. 56–57
• The study used 306 weekly samples from January 2009 through September 2015, with the first 80% for training and the final 20% for testing.Four datasets represented different groups of input variables.Source: §4.3.1 · pp. 57–58, Table 10 · p. 59

4.3.2. Experimental results

• 31 inputs yielded the best overall result: dRVFL achieved MAE 9.14 Hm3, improving previous classical SVR, MLP, and GPR results by over 20%.The multi-level RVFL was second with MAE 10.26 Hm3, followed closely by edRVFL and basic RVFL.Source: §4.3.2 · p. 59, Table 11 · p. 60
• RVFL-family approaches were statistically superior to most alternatives across the four datasets, although some RVFL algorithms performed similarly.Source: §4.3.2 · p. 59
• The best algorithm varied with input dimensionality: multiRVFL led with 19 inputs, whereas dRVFL led with 28, 29, and 31 inputs.The 31-input dataset also produced the best overall performance.Source: §4.3.2 · p. 59

Conclusions

• Randomization-based models achieved superior performance across the study’s solar, wind, and hydro-power prediction problems, with statistically significant improvements in most cases and training times of a few seconds.The conclusions characterize the experimental evidence as outperforming alternative algorithms while reducing benchmark computation time.Source: §5 · pp. 60, 62

Experimental evidence

• At Belesar, Table 12 reports Wilcoxon signed-rank p-values for different datasets in the water-level prediction problem; gray cells indicate no declared significance at α = 0.05.Source: Table 12 · p. 61

Challenges and open questions

• Randomness remains an active research area, with open questions concerning how initialization randomness contributes to epistemic uncertainty and prediction trustworthiness.The issue is especially relevant to Echo State Networks, where recurrence can amplify small differences in random reservoir weights.Source: §5 · pp. 62–63
• Explainability remains limited because randomization and ensemble aggregation can make predictions difficult for non-expert users to understand and trust.The paper highlights feature attribution and counterfactual generation as relevant XAI directions for renewable-energy applications.Source: §5 · pp. 63–64

Research directions

• The authors advocate hybrid models that combine randomized data-driven methods with domain knowledge, engineered features, biased initialization, or numerical-simulator outputs.These strategies are proposed to incorporate established knowledge about renewable-energy phenomena into randomization-based models.Source: §5 · p. 64
• Future work should investigate newer randomization-based models and additional renewable-energy domains, including tidal, hydrogen, geothermal, and biomass energy.The paper identifies oblique and isolation forests, random-feature methods, and newer deep-learning concepts as potential opportunities.Source: §5 · pp. 64–65

Conclusions

• Model assessment should consider power consumption, memory requirements, training time, and inference latency alongside predictive accuracy and precision.The authors frame these design factors as decisive for practical renewable-energy systems and environmental sustainability.Source: §5 · p. 66