Source-linked AI summary
Neural basis expansion analysis with exogenous variables: Forecasting electricity prices with NBEATSx
Kin G. Olivares, Cristian Challu, Grzegorz Marcjasz, Rafał Weron, Artur Dubrawski
TL;DR
Electricity price forecasting needs models that use exogenous factors without sacrificing accuracy or interpretability. The paper extends NBEATS into NBEATSx, incorporates temporal and static covariates, and evaluates it across electricity markets. NBEATSx achieves state-of-the-art performance, improving accuracy by nearly 20% over NBEATS and up to 5% over specialized LEAR and DNN models, while decomposing predictions into trend, seasonality, and exogenous components.
Problem
Deep-learning forecasters lacked time-dependent exogenous inputs and interpretable outputs, while exogenous covariates are fundamental for accurate electricity price forecasting.
Method
NBEATSx extends NBEATS to incorporate temporal and static exogenous variables and uses an interpretable architecture for time-series decomposition.
Results
NBEATSx achieves state-of-the-art performance across five EPF tasks, improving over NBEATS by nearly 20% and over LEAR and DNN models by up to 5%.
Takeaways & Limitations
NBEATSx can explain time-series variation by separating trend, seasonality, and exogenous components while supporting forecasting tasks where exogenous variables are fundamental.
Takeaways & Limitations
The paper identifies future extensions including wavelet-based harmonic functions, spline-based covariate encoders, and smoothness regularization for non-interpretable outputs.
Abstract
from arXiv · showhide
We extend the neural basis expansion analysis (NBEATS) to incorporate exogenous factors. The resulting method, called NBEATSx, improves on a well performing deep learning model, extending its capabilities by including exogenous variables and allowing it to integrate multiple sources of useful information. To showcase the utility of the NBEATSx model, we conduct a comprehensive study of its application to electricity price forecasting (EPF) tasks across a broad range of years and markets. We observe state-of-the-art performance, significantly improving the forecast accuracy by nearly 20% over the original NBEATS model, and by up to 5% over other well established statistical and machine learning methods specialized for these tasks. Additionally, the proposed neural network has an interpretable configuration that can structurally decompose time series, visualizing the relative impact of trend and seasonal components and revealing the modeled processes' interactions with exogenous factors. To assist related work we made the code available in https://github.com/cchallu/nbeatsx.
1. Introduction
NBEATSx addresses limitations in deep-learning forecasting by incorporating exogenous variables and improving interpretability. Across five electricity price forecasting tasks, it achieves state-of-the-art accuracy while decomposing predictions into interpretable components.
- Motivation: Deep-learning forecasters still lacked time-dependent exogenous inputs and interpretable outputs despite strong forecasting performance.These limitations restrict use in applications where covariates matter and where understanding predictions is important.
- Method: NBEATSx extends NBEATS to incorporate temporal and static exogenous variables through a convolutional substructure that cleans and encodes covariate information.The design respects temporal dependencies in the covariates.
- Interpretability: NBEATSx combines nonlinear neural transformations with multiple-seasonality modeling and interactions involving holidays and other covariates while remaining interpretable.Its architecture can separate level, trend, seasonality, and exogenous effects.
- Evaluation: Five EPF tasks achieve state-of-the-art performance on all considered datasets.The evaluation covers a broad set of electricity markets and time periods.
- Evaluation: Nearly 20% accuracy improvement is obtained over the original NBEATS and ESRNN architectures, with up to 5% over established machine-learning and EPF-tailored methods.The reported comparisons concern the five electricity price forecasting tasks.
2. Literature Review
Deep learning has become effective for sequence modeling and forecasting, while electricity price forecasting remains a challenging application with volatile, complex dynamics. Prior EPF studies have faced limited evaluation periods, weak benchmarks, and incomplete methodological reporting, motivating broader and more reproducible comparisons.
- Deep Learning for Sequence Modeling: Deep-learning sequence models use nonlinear temporal representations and have recently outperformed classical statistical forecasting methods.Examples include MQCNN, ESRNN, and NBEATS.
- Deep Learning for Sequence Modeling: Recurrent, convolutional, and skip-connected architectures established building blocks for deeper sequence models with longer effective memory.WaveNet, DilRNN, and TCN illustrate these developments.
- Electricity Price Forecasting: Electricity price forecasting targets spot and forward wholesale-market prices, with most research focusing on next-day 24-hour day-ahead forecasts.Studies also address very short-, mid-, and long-term horizons.
- Electricity Price Forecasting: Renewable generation and large-scale battery storage have added complex dynamics to already volatile electricity spot prices.These dynamics make EPF a productive setting for testing forecasting methods and trading strategies.
- Research Gaps: EPF research has often used short, single-market test periods, weak established benchmarks, or incomplete training descriptions that reduce reproducibility.The paper addresses these shortcomings with two-year out-of-sample periods across five power markets and competitive benchmarks.
3. NBEATSx Model
NBEATSx extends NBEATS with exogenous inputs while retaining residual basis-expansion forecasting. Its interpretable and generic configurations trade constrained decomposition against flexible data-driven representations.
- Architecture: NBEATSx projects target and covariate inputs onto basis functions through fully connected blocks, whose forecasts aggregate into the final prediction.Backcasts clean inputs for subsequent blocks, while forecasts are summed across stacks.
- Architecture: Doubly residual stacking subtracts each block’s backcast component from subsequent inputs and aggregates partial forecasts within each stack.This supports sequential signal decomposition and prepares downstream forecasts.
- Interpretable Configuration: The interpretable configuration uses polynomial, harmonic, and exogenous bases to decompose predictions into level, trend, seasonality, and covariate effects.Harmonic coefficients act as Fourier-transform coefficients, while exogenous expansion resembles time-varying local regression.
- Interpretable Configuration: NBEATSx adds an exogenous stack to the original trend-and-seasonality arrangement, producing three interpretable stacks for trend, seasonal, and exogenous factors.These variants are called NBEATS-I and NBEATSx-I.
- Generic Configuration: The generic configuration learns unconstrained basis representations and uses an encoder convolutional substructure to derive context from time-dependent covariates.The paper names this configuration NBEATSx-G and notes that greater flexibility may reduce interpretability and increase overfitting risk.
4. Empirical Evaluation
The evaluation tests NBEATSx on five day-ahead electricity markets using hourly prices and influential temporal covariates. It also examines interpretable decomposition and the role of exogenous information in forecasting.
- Datasets: Five major electricity markets provide six years of hourly prices and two influential temporal exogenous variables, with the final two years held out for testing.The markets include Nord Pool, PJM, and three EPEX markets.
- Datasets: The datasets span differing volatility and price-event patterns, including frequent negative prices and spikes in some EPEX markets.Nord Pool is described as least volatile, while PJM is more volatile and EPEX markets show more pronounced price events.
- Signal Decomposition: NBEATSx-I produces substantially smaller forecast residuals than NBEATS-I on a high-load Nord Pool day by incorporating exogenous information.The example specifically uses electricity load and production forecasts to explain price variation.
- Evaluation Design: Evaluation uses train, validation, early-stopping, and test partitions, with daily recalibration that incorporates all data available before each prediction.The recalibration procedure uses an early-stopping set of 42 weeks randomly selected from the updated training data.
- Signal Decomposition: The interpretable decomposition displays level, polynomial trend, nonlinear Fourier seasonality, exogenous load effects, and unexplained variation.The figure’s caption identifies these components across successive rows.
4.3. Comparative Analysis
The comparative analysis evaluates NBEATSx and benchmark forecasts using standardized metrics, rolling data splits, automated tuning, recalibration, and statistical significance tests. Across five electricity markets, NBEATSx ensembles generally deliver the strongest accuracy while retaining practical computational performance.
- Training methodology: Training uses three years, validation uses the intervening year, and testing uses the final two years for each of five markets.An early-stopping set is randomly sampled or consists of the 42 weeks following the training period.
- Training methodology: Validation-guided Bayesian optimization selects configurations, while daily recalibration retrains the selected model with newly available information before test inference.Early-stopping data regularizes retraining during recalibration.
- Forecasting results: 18.77% average improvement across all metrics and markets over original NBEATS, versus 20.6% over ESRNN without time-dependent covariates.For the NBEATSx ensemble, average improvements were 4.68% in RMSE, 2.53% in MAE, 1.97% in rMAE, and 1.25% in sMAPE.
- Forecasting results: NBEATSx-I and NBEATSx-G significantly outperformed selected benchmarks in several markets, and no benchmark significantly outperformed either model.NBEATSx-G outperformed DNN in NP and EPEX-DE; NBEATSx-I outperformed DNN in NP, EPEX-FR, and EPEX-DE.
- Forecasting results: 3% average accuracy improvement resulted from ensembling NBEATSx compared with the single best component models across all markets.The appendix reports similar results for single best configurations using MAE accuracy and Giacomini-White significance analyses.
- Computational performance: Day-ahead prediction takes milliseconds and is comparable to LEAR and DNN, while recalibration requires about 50% more time than DNN.The reported computational comparison appears in Appendix Table A1.
5. Conclusions
NBEATSx extends NBEATS with exogenous variables, producing interpretable forecasts and strong performance on tasks where such variables matter. In electricity price forecasting benchmarks, it improves substantially over NBEATS and specialized models, while remaining open to further refinement.
- NBEATSx extends NBEATS for univariate forecasting with exogenous variables and is suitable for tasks where those variables are fundamental.
- The method is mainly composed of fully connected layers and is fast to optimize while producing interpretable results.
- Nearly 20% improvement over NBEATS and up to 5% over LEAR and DNN models were observed in electricity price forecasting.
- The interpretable configuration separates forecasts into trend, seasonality, and exogenous components for analyst insight.
- Future improvements include wavelet-like harmonic functions, spline-based covariate encoders, and smoothness regularization for non-interpretable outputs.
Appendix A.1. Forecast and Backast Basis
The interpretable NBEATS configuration uses slowly varying polynomial and harmonic bases to model trends and seasonalities. Forecast and backcast components rely on similar basis functions, differing in the span of their time indexes.
- Polynomial bases model trends, while harmonic bases model seasonalities in the interpretable configuration.
- The basis functions are slowly varying, supporting the modeling of trend and seasonal structure.
- Forecast and backcast components use similar basis functions, with their difference determined by the span of their time indexes.
Appendix A.2. Training and validation curves
Training diagnostics compare NBEATSx-G with exogenous variables against NBEATS without them using training and validation MAE curves over optimization iterations. The curves are shown for multiple random initialization seeds.
- Training and validation MAE curves compare NBEATSx-G with exogenous variables against NBEATS without exogenous variables.
- The curves track model errors as optimization iterations progress during hyperparameter optimization.
- Four curves are defined by different random seeds used for initialization.
- Models trained with and without exogenous variables form separate clusters in their training and validation errors.
Appendix A.3. Computational Time
The appendix reports computational-time measurements for four highly accurate models in the NP day-ahead electricity price forecasting task. It evaluates ensemble recalibration and prediction computation, with neural models run on a GPU and LEAR on a CPU.
- The timing study concerns the top four models for day-ahead electricity price forecasting in the NP market, averaged across four ensemble elements.
- Computational time is measured using ensemble recalibration and prediction computation for the four most accurate models.
- Neural network models use a GeForce RTX 2080 GPU, while LEAR uses an Intel Xeon Silver 4210 CPU.
- NBEATSx ensemble recalibration takes 75 and 81 seconds in the reported experiments, with an increase of 30 seconds noted.
Appendix A.4. Best Single Models
NBEATSx models substantially improve forecast accuracy over NBEATS and ESRNN without time-dependent covariates, and significantly outperform LEAR and DNN across most markets and metrics. The appendix reports these comparisons using forecast-error tables and Giacomini-White significance tests.
- 0.75% to 7.2% statistically significant accuracy gains over LEAR and DNN occur across metrics and markets, except EPEX-BE.
- The Giacomini-White plots compare six models across five markets, with colored cells indicating significant MAE outperformance and color intensity reflecting significance.The evaluated models include LEAR, DNN, ESRNN, NBEATS, NBEATSx-G, and NBEATSx-I.
- 1000 optimization steps were used for comparability, while restricting optimization to 300 steps produced similar results.
Appendix A.5. Comments on Hyperparameter Optimization
The appendix identifies recurring choices in interpretable bases and neural-network optimization. Quadratic trends commonly fit day-ahead profiles, while activation, batch-size, normalization, and architecture choices showed systematic selection patterns.
- Interpretable configuration observations: Quadratic trends were the most common choice for day-ahead EPF, and quadratic trends combined with harmonics described average daily profiles successfully.Linear trends were excluded because they were restrictive; longer horizons beyond 24 hours benefited from greater trend flexibility.
- Interpretable configuration observations: Nhr showed no preferred value because the harmonic basis already covered a broad frequency spectrum.Nhr = 1 was considered a suitable default unless sampling and time-series frequencies appeared mismatched.
- Hyperparameter optimization regularities: SeLU, PreLU, and Sigmoid were selected most often, whereas ReLU, TanH, and LReLU were consistently outperformed.Sigmoid activations can make optimization difficult as network depth increases.
- Hyperparameter optimization regularities: Batch sizes of 256 and 512 were preferred over 128, while batch normalization was often detrimental with doubly-residual stacks.Residual signals close to zero can make normalization numerically unstable; robust median normalization was preferred for exogenous variables.