Source-linked AI summary
Building Energy Load Forecasting using Deep Neural Networks
Daniel L. Marino, Kasun Amarasinghe, Milos Manic
TL;DR
Accurate aggregate and individual-site load forecasts are needed for flexible energy management, but individual building-level forecasting is difficult. The paper evaluates standard LSTM and LSTM-based S2S architectures on one residential customer's one-minute and one-hour electricity data. S2S performs well at both resolutions, while standard LSTM fails on minute-resolution data.
Problem
Accurate future energy-demand predictions are needed for efficient buildings and flexible smart-grid management, while individual building-level load forecasting remains difficult.
Method
The paper evaluates standard LSTM and LSTM-based Sequence to Sequence architectures for individual building-level load forecasting.
Results
The S2S architecture performs well on one-minute and one-hour datasets, whereas standard LSTM fails on minute-resolution data and performs well at one-hour resolution.
Takeaways & Limitations
The S2S architecture provides a flexible forecasting model that can use arbitrary numbers of previous measurements to estimate arbitrary numbers of future time steps.
Abstract
from arXiv · showhide
Ensuring sustainability demands more efficient energy management with minimized energy wastage. Therefore, the power grid of the future should provide an unprecedented level of flexibility in energy management. To that end, intelligent decision making requires accurate predictions of future energy demand/load, both at aggregate and individual site level. Thus, energy load forecasting have received increased attention in the recent past, however has proven to be a difficult problem. This paper presents a novel energy load forecasting methodology based on Deep Neural Networks, specifically Long Short Term Memory (LSTM) algorithms. The presented work investigates two variants of the LSTM: 1) standard LSTM and 2) LSTM-based Sequence to Sequence (S2S) architecture. Both methods were implemented on a benchmark data set of electricity consumption data from one residential customer. Both architectures where trained and tested on one hour and one-minute time-step resolution datasets. Experimental results showed that the standard LSTM failed at one-minute resolution data while performing well in one-hour resolution data. It was shown that S2S architecture performed well on both datasets. Further, it was shown that the presented methods produced comparable results with the other deep learning methods for energy forecasting in literature.
I. INTRODUCTION
The paper motivates accurate load forecasting as necessary for more efficient buildings and adaptable smart grids, then investigates deep-learning methods for individual building loads. It compares standard LSTM and LSTM-based S2S forecasting on one residential customer's data at one-minute and one-hour resolutions.
- Motivation: Buildings consume 20%-40% of total energy production and account for significant energy wastage, making accurate future-consumption prediction important for efficiency.The paper links reduced energy wastage and building energy efficiency to sustainability.
- Motivation: Smart grids require continuous forecasting at aggregate and modular levels to adapt demand and distribute generation efficiently.The motivation includes generation from sources such as renewables.
- Motivation: Individual building-level forecasting supports local demand response, while smart meters make building- and site-level consumption data acquisition feasible.The paper distinguishes building-level forecasting from aggregate-load forecasting.
- Related work: Load forecasting is difficult, with individual building-level forecasting harder than aggregate forecasting; prior work includes physics-based, statistical, and machine-learning models.The paper focuses on statistical and machine-learning-based load forecasting and reviews ANN, SVM, kernel, and deep-learning approaches.
- Contribution: This work investigates deep learning for individual building-level load forecasting, using multilayer models to learn representations at multiple abstraction levels.The paper frames deep learning as an alternative methodology for the forecasting task.
- Contribution: Standard LSTM and LSTM-based S2S methods were tested on one residential customer's benchmark electricity data at one-minute and one-hour resolutions.The S2S architecture performed well on both datasets, whereas standard LSTM failed on minute-resolution data; both methods produced results comparable to prior CRBM and FCRBM methods.
II. LONG SHORT TERM MEMORY
LSTM is introduced as a recurrent neural network designed to address vanishing gradients and retain information over long periods. Its memory cells regulate temporal information through input, forget, and output gates.
- LSTM background: LSTM was designed to overcome vanishing-gradient problems in recurrent neural-network training and store information for long periods.The passage also identifies exploding gradients as a training problem for conventional RNN methods.
- LSTM cell: An LSTM network uses memory cells with self-loops to store temporal information in the cell state.Information is written, erased, and read from the memory state.
- LSTM cell: Input, forget, and output gates respectively control writing, erasing, and reading operations in the cell memory.The input gate uses a sigmoid as a soft switch for whether an update modifies the memory state.
- LSTM cell: The cell notation defines the input, forget, and output gates, the state and output at time t, and the update signal.These quantities are used to describe a single LSTM cell's operation.
- Multilayer architecture: Stacked LSTM cells form multilayer networks that predict ŷ[t] from all inputs available through time t.The architecture is described as using previous inputs to predict the outcome at the current time.
III. LOAD FORECASTING USING DEEP NEURAL NETWORKS
The methodology section investigates two LSTM variants for deep-neural-network load forecasting: standard LSTM and an LSTM-based sequence-to-sequence architecture.
- Methodology: The paper investigates two LSTM variants for load forecasting: standard LSTM and LSTM-based Sequence to Sequence (S2S).The section presents the standard LSTM methodology first and then the S2S architecture.
A. Load Forecasting using Standard LSTM
The methodology estimates future active-power load from historical measurements, using time information and recurrent predictions to forecast multiple future steps.
- The model estimates electricity load for one or multiple future time steps from historical load measurements.
- Standard LSTM inputs combine the previous active-power measurement with the target prediction's day, weekday, and hour.
- The network predicts the next time step, then feeds predictions back as inputs to extend the forecast into the future.
- Training uses back-propagation through time by unrolling the recurrent network into a deep feedforward network with shared parameters.
- ADAM is used as the gradient-based optimizer, while norm clipping addresses the exploding-gradient problem and unrolling uses 50 steps.
B. Load forecasting using LSTM based sequence to sequence
The paper explores an LSTM-based Sequence to Sequence architecture for flexible load forecasting. It encodes variable-length inputs and produces forecasts for multiple future steps.
- Architecture: S2S maps sequences of different lengths using an encoder and decoder built from two LSTM networks.The encoder converts variable-length inputs into a fixed-length vector used as the decoder’s input state.
- Architecture: The decoder generates an output sequence of length n representing the energy load forecast for the next n steps.
- Flexibility: The architecture accepts an arbitrary number of historical load measurements and predicts an arbitrary number of future time steps.
- Inputs: The encoder receives load measurements and corresponding date-time inputs, while the decoder receives only date-time inputs.This design prevents the decoder from learning a naïve direct input-to-output mapping.
- Training: During training, backpropagation flows from the decoder to the encoder so both networks update their weights.Encoder pre-training alone is described as insufficient for good performance.
IV. DATASET AND EXPERIMENTAL RESULTS
This section introduces the testing dataset before presenting experimental results for the two investigated forecasting models.
- Dataset: The experimental section first introduces the dataset used for testing.
- Experimental results: It then presents experimental results obtained for the two investigated models.
- Experimental results: The section combines dataset description with evaluation of both forecasting approaches.
A. Dataset
The dataset uses an initial training period and a subsequent testing period, with ranges selected for comparability to prior work.
- Data split: The first three years of data were used to train the model.
- Data split: The last year of data was used as testing data.
- Data split: The training and testing ranges were chosen to be comparable with the work in.
B. Experimental Results using standard LSTM
Standard LSTM forecasting was accurate one step ahead but became inaccurate for longer recursive forecasts. Delayed inputs were explored for one-minute data, but their insufficiency led to testing an S2S architecture.
- One-step and recursive forecasting: Standard LSTM produced low test error for one-step-ahead forecasting but failed when its predictions were recursively used for farther horizons.The model was accurate for the next step yet inaccurate when predictions replaced measured inputs.
- One-step and recursive forecasting: For a 60-hour forecast, standard LSTM was accurate while historical measurements were supplied, then became inaccurate after predictions were fed back as inputs.The recursive forecast began after hour 60 and covered the following 60 hours.
- One-minute resolution: With one-minute resolution, consecutive measurements were very similar, enabling a naïve mapping from the current input to the next output.This similarity helped the network predict the next step without learning a useful longer-term representation.
- One-minute resolution: Using measurements farther in the past was tested to make inputs and outputs more different, but delayed input was insufficient for one-minute forecasting.The delayed-input experiment used earlier measurements such as five steps back.
C. Experimental Results using LSTM based S2S architecture
The LSTM-based S2S model uses an encoder and decoder to forecast multiple future load steps. Experiments evaluated its errors across network capacities and one-hour and one-minute datasets, with dropout improving testing accuracy and results comparable to FCRBM.
- Architecture: The S2S architecture encodes variable-length load sequences into a fixed-length vector, which initializes a decoder generating n future energy-load steps.The encoder receives available load measurements, while the decoder generates the forecast sequence.
- Architecture: The decoder receives date and time rather than load measurements, preventing it from learning a naïve direct input-to-output mapping.Available load measurements are introduced only to the encoder.
- Model capacity and regularization: Increasing layers and units reduced training error but did not improve testing performance for the one-hour S2S dataset.Table I reports RMSE at training end and the lowest testing error observed during training.
- Model capacity and regularization: Dropout was used as regularization to improve testing accuracy, with predictions generated 60 time steps into the future for one-minute and one-hour datasets.The resulting errors were reported as comparable to FCRBM results on the same dataset.
- Prediction examples: S2S prediction results were illustrated separately for the training dataset and the testing dataset.Figures 6 and 7 provide examples of the model’s training and testing predictions.
V. CONCLUSIONS
The paper evaluates standard and S2S LSTM architectures for building-level load forecasting at one-hour and one-minute resolutions. Standard LSTM failed at one-minute resolution, whereas S2S performed well on both datasets and produced results comparable to FCRBM; broader dataset evaluation remains future work.
- Conclusions: The study investigated two LSTM-based neural-network architectures for building-level energy-load forecasting at one-hour and one-minute resolutions.Both architectures were trained and tested on both time-step datasets.
- Conclusions: Standard LSTM was unable to accurately forecast one-minute-resolution loads, while the S2S architecture performed well on both datasets.The conclusion contrasts the architectures across the two time resolutions.
- Conclusions: S2S accepts an arbitrary number of previous load measurements and estimates an arbitrary number of future time steps.This flexibility is stated as a property of the presented S2S model.
- Conclusions: The S2S model produced results comparable to FCRBM results reported for the same dataset.The comparison concerns the same residential-consumption dataset.
- Limitations and future work: The authors identify testing both algorithms on different real-world datasets as future work.They also plan to investigate other deep-learning algorithms and regularization approaches.