Source-linked AI summary
Deep Learning for Time Series Forecasting: The Electric Load Case
Alberto Gasparin, Slobodan Lukovic, Cesare Alippi
TL;DR
Accurate load forecasting matters for smart-grid management, but nonlinear demand and limited comparative evidence complicate model selection. The paper surveys and experimentally compares deep architectures and forecasting strategies across real-world household and aggregated-load datasets. Its recurrent-network findings indicate that simple ERNNs can perform comparably to gated networks, while strategy and exogenous-variable effects vary by setting.
Problem
Electric load forecasting is challenging because demand is nonlinear and affected by time-varying factors, while comprehensive comparisons of deep architectures on common benchmarks are limited.
Method
The paper reviews and evaluates recurrent, sequence-to-sequence, and temporal convolutional architectures, including recurrent multi-step strategies, on real-world household and aggregated-load scenarios.
Results
Simple ERNNs perform comparably to gated GRU and LSTM networks, while recurrent strategy performance differs with multi-step forecasting conditions and exogenous variables.
Takeaways & Limitations
The study provides a systematic basis for comparing deep-learning architectures and forecasting strategies in individual and aggregated short-term load forecasting.
Takeaways & Limitations
ERNN training is constrained by vanishing and exploding gradients, especially for long-term connections.
Abstract
from arXiv · showhide
Management and efficient operations in critical infrastructure such as Smart Grids take huge advantage of accurate power load forecasting which, due to its nonlinear nature, remains a challenging task. Recently, deep learning has emerged in the machine learning field achieving impressive performance in a vast range of tasks, from image classification to machine translation. Applications of deep learning models to the electric load forecasting problem are gaining interest among researchers as well as the industry, but a comprehensive and sound comparison among different architectures is not yet available in the literature. This work aims at filling the gap by reviewing and experimentally evaluating on two real-world datasets the most recent trends in electric load forecasting, by contrasting deep learning architectures on short term forecast (one day ahead prediction). Specifically, we focus on feedforward and recurrent neural networks, sequence to sequence models and temporal convolutional neural networks along with architectural variants, which are known in the signal processing community but are novel to the load forecasting one.
1 Introduction
Electric load forecasting supports smart-grid operations but remains difficult because demand depends on nonlinear, time-varying influences. This work addresses limited comparative evidence by evaluating recent deep architectures across individual and aggregated load scenarios.
- Electric load forecasting supports pricing, renewable integration, maintenance planning, and energy-efficiency decisions in smart grids.
- Day-ahead forecasting is important for efficient power supply, while individual-household demand is more volatile than aggregated consumption.
- Traditional ARMA, ARIMA, seasonal ARIMA, ARMAX, and ARIMAX families impose linearity assumptions that generally do not hold for load profiles.
- Existing reviews do not provide a comprehensive performance comparison of deep architectures on common load-forecasting benchmarks.
- The survey contrasts recurrent, sequence-to-sequence, and temporal convolutional architectures on two real-world scenarios: volatile household demand and aggregated consumption.
- The contributions include a comprehensive review, comparison of recurrent multi-step prediction strategies, and systematic assessment across individual and aggregated demand.
2 Problem Description
The paper frames multi-step load forecasting as supervised learning by sliding a fixed historical window across the time series. Each window supplies lagged load values, optional exogenous features, and a future target sequence.
- A sliding window converts the target signal into supervised input-output pairs indexed across the available time range.
- The predictor receives a fixed window of nT lagged values and forecasts the next nO values of the univariate load series.
- The output vector contains future measurements, while the model produces a prediction vector by applying f to the input and learned parameters.
- With exogenous variables, each input timestep combines the scalar load measurement with aligned features such as temperature.
- The paper defines notation for the forecasting formulation and model variables in its nomenclature table.
3 Feed Forward Neural Networks
Feedforward neural networks provide the paper’s baseline architecture for mapping fixed lagged inputs to multi-step forecasts. Their parameters are learned by minimizing a loss, typically with iterative gradient-based optimization and regularization.
- FNNs use layered input, hidden, and output blocks, with hidden-layer count determining depth and layer width determining model complexity.
- The network maps h0 = xt to the forecast vector ˆyt = hL, whose dimension matches the multi-step horizon nO.
- Each layer has its own weight matrix and bias parameters, collectively grouped into the network parameter vector Θ.
- Training estimates parameters by minimizing a loss that measures discrepancy between predicted outputs and observed measurements.
- Mean squared error is widely used for time-series prediction, while L2 regularization controls model complexity and helps prevent overfitting.
- Because the objective is nonconvex, parameter estimation proceeds iteratively using methods such as gradient descent, SGD, RMSProp, Adagrad, or Adam.
- Deep FNNs serve as the baseline, while prior work compares multi-output and recursive forecasting strategies and reports benefits from extra information in some settings.
4 Recurrent Neural networks
Recurrent neural networks process ordered inputs through state updates, while unfolded computation enables gradient-based training with shared weights across time. The section reviews ERNNs, LSTMs, GRUs, deep recurrent architectures, and multi-step forecasting strategies.
- 4.1 Elmann RNNs (ERNN): ERNNs process one sequence element at a time, updating their internal state from the previous state and current input while preserving temporal order.Time unfolding converts the recurrent graph into an acyclic structure for analysis and training.
- 4.1 Elmann RNNs (ERNN): Unfolding represents an RNN as a feedforward network whose layers correspond to input time intervals and share the same weight matrices.The shared weights preserve the recurrent architecture across time steps.
- 4.1 Elmann RNNs (ERNN): ERNN parameters are typically learned with Backpropagation Through Time, but long-term dependencies can cause vanishing or exploding gradients.Gradient propagation is effective for short-term connections but may shrink to zero or diverge for long-term ones.
- 4.2 Long Short-Term Memory (LSTM): LSTMs address standard RNN gradient problems with gated cells that control information flow and can preserve internal memory over long intervals.Their cell state combines old and new state information under gate control.
- 4.3 Gated Recurrent Units (GRU): GRUs simplify LSTMs by merging forget- and input-gate functionality, yielding two gates and a more parsimonious architecture.The reviewed literature reports comparable performance to LSTMs with generally faster training due to lighter computation.
- 4.4 Deep RNNs and 4.5 Multi-Step Prediction Schemes: Deep RNNs stack recurrent layers to model temporal hierarchies, and multi-step forecasting can use independent direct predictors or DirRec models that add previous forecasts to later inputs.DirRec combines multiple predictors with progressively enlarged input sets.
5 Sequence To Sequence models
Sequence-to-sequence models use an encoder to compress an input sequence into a context representation and a decoder to generate an output sequence. In load forecasting, teacher forcing and self-generated training differ in the decoder inputs and address the mismatch between training and inference.
- Encoder-decoder architecture: The encoder reads the input sequence one timestep at a time and produces a fixed-dimensional context representation.The context can be the encoder’s last hidden state or a function of it.
- Encoder-decoder architecture: The decoder uses the context vector to generate the forecast sequence, typically through a recurrent neural network trained end-to-end with the encoder.The decoder output is the predicted sequence of length nO.
- Training and inference: During teacher forcing, the decoder receives the ground-truth value from the previous timestep when generating the next estimate.This training path differs from inference, where true previous values are unavailable.
- Training and inference: During inference, decoder inputs are replaced by model estimates, creating exposure bias because errors can accumulate over time.Scheduled sampling, professor forcing, and reinforcement learning are cited as approaches for addressing this discrepancy.
- Load forecasting applications: The study investigates one seq2seq architecture trained with teacher forcing and another trained with self-generated samples.The two procedures differ in whether the decoder uses ground-truth values or its own predictions during training.
6 Convolutional Neural Networks
Convolutional neural networks apply shared filters to grid-structured data, while temporal convolutional networks adapt causal, dilated, and residual convolutions for sequence forecasting. These design choices expand historical context efficiently while supporting parallel and stable training.
- CNN foundations: One-dimensional CNNs apply convolutional filters to time-series data, with local connectivity and parameter sharing reducing the number of parameters.Each output feature-map location uses the same kernel over a local input patch.
- Temporal convolutional networks: The paper evaluates a Wavenet-inspired autoregressive convolutional architecture that had not previously been proposed for electric-load forecasting.The architecture is designed to process arbitrary-length sequences and output sequences of the same length.
- Temporal convolutions: Causal convolutions ensure that an autoregressive estimate depends only on past samples, while padding preserves the input sequence length.This prevents future values from influencing predictions.
- Temporal convolutions: Dilated convolutions enlarge the receptive field and improve access to long-term dependencies without requiring proportionally greater network depth.With dilation, receptive-field growth is exponential rather than linear in depth; the illustrated three-layer example uses history size 8.
- Residual connections: Residual connections recast the desired mapping as a residual function plus the input, helping deep CNNs address performance degradation with increasing depth.The shortcut structure allows stacked layers to learn identity mappings by driving residual-layer weights toward zero.
- Temporal convolutional networks: TCNs combine causal dilated convolutions and residual connections to exploit long histories, with lower training memory, parallel later-timestep predictions, and more stable training than RNNs.The paper’s architecture separately processes load and available exogenous information before combining transformed features.
7 Performance Assessment
The assessment compares forecasting architectures across household and aggregated-load scenarios using RMSE, MAE, normalized RMSE, and R2. Aggregated load is easier to predict; strategy and feature choices affect model performance, while TCNs offer a promising recurrent-network alternative.
- Evaluation setup: The evaluation uses two real-world datasets, three use cases, and standard error metrics for architecture comparison.Models are assessed with RMSE, MAE, normalized RMSE, and R2; implementations use Keras with TensorFlow.
- Use Case I: IHEPC contains 2.07M one-minute household measurements, resampled to 15-minute intervals for day-ahead forecasting 96 timesteps ahead.The test set comprises the last year, while earlier data support repeated training-validation splits and hyperparameter selection.
- Use Case I: For IHEPC, MIMO recurrent strategies generally outperform recursive strategies because recursive error accumulation is especially damaging for volatile household load.Among MIMO models, gated networks outperform the simple Elman network, although most non-recursive models have comparable performance.
- Use Cases II–III: Aggregated GEFCom2014 load produces substantially better accuracy across models than volatile household load because its consumption pattern is smoother.GEFCom2014 contains 60.6k hourly measurements, and temperature variables are available as exogenous features.
- Use Cases II–III: Exogenous variables improve most models except plain FNNs; recursive RNNs particularly outperform MIMO RNNs when temperature inputs are used.The paper attributes this difference to recursive RNNs' better ability to leverage exogenous temperature series, while MIMO recurrent gains without extra features are negligible.
- Cross-use-case findings: TCNs perform well across all use cases and are presented as promising alternatives to RNNs because of lower training-memory requirements and inherent parallelism.Household predictions still smooth steep consumption peaks; the paper links this partly to MSE optimization or potentially unpredictable user-behavior noise.
8 Conclusions
The work surveys and experimentally evaluates deep learning models for short-term load forecasting, emphasizing standardized assessment across recurrent, sequence-to-sequence, and temporal convolutional architectures. Its findings show that model effectiveness and cost depend on the forecasting scenario.
- The study evaluates recurrent neural networks, sequence-to-sequence architectures, and temporal convolutional neural networks for short-term load forecasting.The work combines a survey with experimental evaluation of the three main deep-learning model families.
- The evaluation covers three real-world use cases spanning individual-household and geographically aggregated load forecasting.One use case concerns a single household, while two concern aggregated load curves from dispersed meters.
- In aggregated forecasting, ERNN performs comparably to GRU and LSTM, making it a less costly alternative without a remarkable accuracy impact.The reported advantage is reduced training time relative to gated networks in this scenario.
- For single-household forecasting, gated networks outperform Elman recurrent networks.The conclusion contrasts this result with the comparable performance observed for aggregated load forecasting.