Source-linked AI summary
Neural NILM: Deep Neural Networks Applied to Energy Disaggregation
Jack Kelly, William Knottenbelt
TL;DR
NILM seeks to estimate individual-appliance electricity demand from a single whole-home meter, and this paper tests whether deep neural networks can address that problem. It adapts three architectures, evaluates them against established baselines on five appliances and seen or unseen houses, and finds strong performance, especially for two architectures on unseen houses.
Problem
NILM must estimate individual-appliance demand from a single aggregate meter, and the paper investigates whether deep neural networks can provide good performance for this task.
Method
The paper adapts LSTMs, denoising autoencoders, and a network that regresses appliance-activation start time, end time, and average power demand.
Results
The denoising autoencoder and rectangles architectures outperform combinatorial optimisation and factorial hidden Markov models broadly, especially on unseen houses.
Takeaways & Limitations
Deep neural networks show promise for NILM, with the denoising autoencoder and rectangles architectures performing well on unseen houses.
Takeaways & Limitations
The study uses data from only five houses, and the authors state that broader generalisation requires training on many more appliances.
Abstract
from arXiv · showhide
Energy disaggregation estimates appliance-by-appliance electricity consumption from a single meter that measures the whole home's electricity demand. Recently, deep neural networks have driven remarkable improvements in classification performance in neighbouring machine learning fields such as image classification and automatic speech recognition. In this paper, we adapt three deep neural network architectures to energy disaggregation: 1) a form of recurrent neural network called `long short-term memory' (LSTM); 2) denoising autoencoders; and 3) a network which regresses the start time, end time and average power demand of each appliance activation. We use seven metrics to test the performance of these algorithms on real aggregate power data from five appliances. Tests are performed against a house not seen during training and against houses seen during training. We find that all three neural nets achieve better F1 scores (averaged over all five appliances) than either combinatorial optimisation or factorial hidden Markov models and that our neural net algorithms generalise well to an unseen house.
1. INTRODUCTION
Energy disaggregation estimates individual-appliance demand from a single aggregate meter, but extracting robust appliance signatures is difficult. This paper investigates whether deep neural networks can learn useful features and generalise to unseen houses.
- Motivation: Energy disaggregation estimates individual appliance demand from a meter measuring combined household demand.The technique is also called non-intrusive load monitoring (NILM).
- Motivation: Hand-engineering detectors for rich appliance signatures is time-consuming and may be vulnerable to noise and artefacts.Washing-machine demand can include rapid oscillations and spin-cycle ramps.
- Motivation: Deep neural networks automatically learn hierarchies of features from raw inputs rather than relying on hand-engineered detectors.This approach had recently improved performance in image classification and other machine-learning fields.
- Contribution: The paper adapts LSTMs, denoising autoencoders, and a start-time, end-time, and power-regression network to NILM.One network is trained for each target appliance.
- Contribution: The networks are compared with combinatorial optimisation and factorial hidden Markov models using seven metrics, including tests on unseen houses.The authors argue that generalisation to houses without ground-truth appliance data is essential for field deployment.
- Deployment: Once trained, the neural networks require only aggregate data from an end-user’s house, not house-specific ground-truth appliance data.Training is computationally expensive, whereas inference is described as much cheaper.
2. INTRODUCTION TO NEURAL NETS
The paper introduces neural networks as layered computational graphs whose weights are learned by reducing prediction error. Its architectures use nonlinear feature transformations and convolutional processing to extract patterns from time-series inputs.
- Neural-network basics: An artificial neural network is a directed graph of weighted connections linking input, hidden, and output layers.Information flows forward through the network, while learning updates connection weights during the backward pass.
- Neural-network basics: Each neuron computes a weighted input sum, adds a learned bias, and applies an activation function to produce its output.The paper uses linear, ReLU, and tanh activation functions.
- Representation: Deep nonlinear hidden layers can re-represent inputs by learning a hierarchy of feature detectors.This hierarchical representation gives deep networks substantial expressive power.
- Learning: Training computes mean squared error against a target and modifies weights in a direction intended to reduce that error.The paper uses gradient descent and backpropagation to calculate weight updates efficiently.
- Time-series processing: All networks use at least one one-dimensional convolutional layer to extract local low-level features across the time-series input.The convolutional layer uses small receptive fields over the input sequence.
3. TRAINING DATA
The networks are trained from real appliance activations combined into synthetic aggregate sequences alongside real aggregate data. The training design normalises inputs and varies appliance-specific windows, but its limited appliance diversity and simplified simulation constrain generalisation.
- Synthetic aggregate data: The networks are trained on a 50:50 mixture of synthetic and real aggregate data, while validation and testing use only real data.The authors report that synthetic data acts as a regulariser for unseen-house generalisation.
- Dataset: The dataset uses UK-DALE, with appliance selection limited to five appliances present in at least three houses.This enables training on at least two houses and testing on another house for each appliance.
- Training setup: Each target appliance has its own network, with input windows ranging from 128 samples for the kettle to 1536 samples for the dishwasher.The paper reports that larger windows can hurt performance for short-duration appliances.
- Synthetic aggregate data: Synthetic aggregate sequences are created by randomly combining real activations from kettles, washing machines, dishwashers, microwaves, and fridges.The target appliance appears with 50% probability, while each distractor appears with 25% probability.
- Limitations: The synthetic-data simulator ignores real temporal structure, such as kettle and toaster activations occurring close together.The authors expect a more realistic simulator could improve neural-network performance.
- Standardisation: Independently centering each sequence removes absolute-power information, although the authors found it may improve generalisation.This preprocessing also avoids requiring the networks to model always-on loads.
4. NEURAL NETWORK ARCHITECTURES
The paper adapts recurrent neural networks, including LSTMs and bidirectional layers, for sequential energy disaggregation. The selected architecture combines convolutional processing with recurrent memory, while bidirectionality limits natural online use.
- Recurrent neural networks: RNNs map input histories to outputs through recurrent connections, making them suitable for sequential data.The networks are trained using backpropagation through time.
- Recurrent neural networks: LSTMs address vanishing or exploding gradients with a gated memory cell, input, output, and feedback loop.The architecture is described as a differentiable latch.
- Recurrent neural networks: Bidirectional RNNs process sequences forwards and backwards, combining the outputs by concatenation or element-wise summation.The experiments settled on concatenation, although element-wise summation appeared nearly as effective and cheaper.
- Recurrent neural networks: Bidirectional RNNs are not naturally suited to online disaggregation, but can be used with frequent small batches of offline processing.
- Selected architecture: The final energy-disaggregation architecture was selected after experimenting with both RNNs and LSTMs.Its listed design begins with an appliance-duration-dependent input followed by a 1D convolutional layer.
6. Fully connected (N=1, activation function=linear)
This section presents three NILM formulations: sequence-to-sequence recurrent prediction, denoising autoencoding, and regression of appliance-activation rectangles. The models use convolutional feature extraction and are trained end-to-end, with substantial computational demands.
- LSTM formulation: At each time step, the network maps aggregate power to a target appliance’s power output.
- LSTM formulation: Adding a convolutional layer slightly improved performance, although it was theoretically unnecessary for LSTMs to retain context.A hierarchical subsampling variant showed promise but was excluded from final experiments.
- Denoising autoencoders: Denoising autoencoders reconstruct an appliance’s clean power demand from aggregate demand treated as noise from other appliances.Unlike standard artificial corruption, NILM uses the other appliances’ demand as the corruption source.
- Denoising autoencoders: The dAE uses convolutional input and output layers to learn position-invariant low-level features before deconvolution.The stated architecture includes a compact middle code layer and end-to-end training without layerwise pre-training or tied weights.
- Activation-rectangle regression: The regression network estimates the start time, end time, and mean power of the first appliance activation in a window.Outputs are normalized to [0, 1], with zeroes indicating no target appliance.
- Implementation: The networks were trained end-to-end on a GPU, taking 1–12 hours per appliance and ranging from 1M to 150M trainable parameters.Training used random initialization without layerwise pre-training.
5. DISAGGREGATION
To process arbitrarily long aggregate sequences, the disaggregation procedure pads the signal and applies each network to sliding, potentially overlapping windows. Overlapping predictions are then aggregated into appliance-level outputs.
- Sliding-window processing: The procedure pads sequence boundaries with zeros and slides each network across the aggregate signal by a manually chosen STRIDE.When STRIDE is shorter than the input window, the network receives overlapping segments.
- Sliding-window processing: Overlapping windows produce multiple estimated values for each time step during disaggregation.
- Output aggregation: For the first two network architectures, multiple values per timestep are combined by taking their mean.
- Output aggregation: For the rectangle-regression network, predicted appliance rectangles are layered, overlap is normalized to [0, 1], and power and probability are thresholded.This converts the rectangle predictions into one output vector per appliance.
6. RESULTS
On an unseen house, the denoising autoencoder and rectangles architecture outperform the benchmark algorithms across several metrics, while LSTM performance depends on appliance state complexity. On houses seen during training, the denoising autoencoder performs strongly across appliances and metrics, with one stated exception.
- Unseen house: On the unseen house, denoising autoencoder and rectangles architectures outperform CO and FHMM for every appliance on F1, precision, energy assignment, and mean absolute error.The benchmarks are combinatorial optimisation and factorial hidden Markov models.
- Unseen house: LSTM outperforms CO and FHMM on kettle, fridge, and microwave but falls behind them on dish washer and washing machine.The distinction is between two-state and multi-state appliances.
- Seen houses: On houses seen during training, the denoising autoencoder outperforms CO and FHMM on every appliance and metric except relative error in total energy.The reported evaluation uses multiple disaggregation metrics.
- Seen houses: On houses seen during training, rectangles architecture outperforms CO and FHMM on every appliance except the microwave across five listed metrics.The metrics are F1, precision, accuracy, proportion of total energy correctly assigned, and mean absolute error.
7. CONCLUSIONS & FUTURE WORK
The adapted neural networks show promise for NILM, especially the denoising autoencoder and rectangles architectures, but comparisons and generalisation remain constrained by model size, appliance complexity, and limited training diversity.
- The denoising autoencoder and rectangles architectures perform well, especially on unseen houses.
- The architecture comparison is not entirely fair because LSTMs used 1M parameters, whereas the larger dAE and rectangles networks used over 150M.
- LSTMs work best for two-state appliances but perform poorly on multi-state appliances such as dishwashers and washing machines.Long gaps between informative events may challenge LSTMs, requiring further investigation.
- Figure 2 compares measured aggregate and appliance demand with raw and overlapped outputs from the three neural architectures.The overlapped outputs are produced by sliding each network over aggregate data with STRIDE=16.
- Figure 4 reports disaggregation performance on houses seen during training using a testing time window different from the training window.
- Generalisation is limited by UK-DALE’s data diversity: it contains many hundreds of days from only five houses, motivating training on hundreds or thousands of appliances.The paper also identifies pre-training on unlabelled data as a possible future direction.