Source-linked AI summary
NILMTK: An Open Source Toolkit for Non-intrusive Load Monitoring
Nipun Batra, Jack Kelly, Oliver Parson, Haimonti Dutta, William Knottenbelt, Alex Rogers, Amarjeet Singh, Mani Srivastava
TL;DR
NILM aims to estimate appliance-level consumption from aggregate household electricity data, but inconsistent data sets, benchmarks, and metrics hinder reproducible comparison. NILMTK addresses this gap with an open-source toolkit that standardises data handling and evaluation, and demonstrates comparisons across multiple public data sets using benchmark algorithms.
Problem
Inconsistent data sets, missing reference implementations, and varied accuracy metrics make empirical comparison of disaggregation algorithms difficult.
Method
NILMTK provides a common data format, data-set parsers, diagnostics and preprocessing, benchmark disaggregation algorithms, and accuracy metrics.
Results
The toolkit enables analysis of six publicly available data sets and evaluation of benchmark disaggregation algorithms across them.
Takeaways & Limitations
NILMTK enables reproducible comparative analysis of energy disaggregation algorithms across diverse data sets.
Takeaways & Limitations
Combinatorial optimisation is computationally tractable only for a small number of modelled appliances because its complexity is exponential in appliance count.
Abstract
from arXiv · showhide
Non-intrusive load monitoring, or energy disaggregation, aims to separate household energy consumption data collected from a single point of measurement into appliance-level consumption data. In recent years, the field has rapidly expanded due to increased interest as national deployments of smart meters have begun in many countries. However, empirically comparing disaggregation algorithms is currently virtually impossible. This is due to the different data sets used, the lack of reference implementations of these algorithms and the variety of accuracy metrics employed. To address this challenge, we present the Non-intrusive Load Monitoring Toolkit (NILMTK); an open source toolkit designed specifically to enable the comparison of energy disaggregation algorithms in a reproducible manner. This work is the first research to compare multiple disaggregation approaches across multiple publicly available data sets. Our toolkit includes parsers for a range of existing data sets, a collection of preprocessing algorithms, a set of statistics for describing data sets, two reference benchmark disaggregation algorithms and a suite of accuracy metrics. We demonstrate the range of reproducible analyses which are made possible by our toolkit, including the analysis of six publicly available data sets and the evaluation of both benchmark disaggregation algorithms across such data sets.
1. INTRODUCTION
NILM research expanded rapidly, but inconsistent data sets, benchmarks, sampling choices, and metrics made direct algorithm comparisons difficult. NILMTK addresses these obstacles through an open, modular toolkit for reproducible cross-data-set evaluation.
- Motivation: NILM separates aggregate household electricity consumption into estimates for individual appliances.The process can support appliance-specific energy feedback and advice about appliance use.
- Research gap: Rapid field growth exposed three obstacles: limited cross-data-set evaluation, difficult-to-reproduce data selection, and inconsistent benchmarks and metrics.These obstacles made it difficult to assess generalisation and compare numerical performance across publications.
- Toolkit: NILMTK is an open-source toolkit designed to enable comparative analysis of energy disaggregation algorithms across diverse data sets.It provides a complete pipeline from data sets to accuracy metrics.
- Toolkit design: Its modular design supports reuse or replacement of components, while flat-file and binary formats support compatibility with tools written for different languages and platforms.The toolkit also provides documentation and was released as open-source software to encourage community contributions.
- Toolkit components: NILMTK-DF standardises energy-disaggregation data and includes parsers for six existing data sets.The format is modelled loosely on REDD to support adoption within the community.
- Toolkit components: The toolkit provides data-set statistics, diagnostic and preprocessing functions, two benchmark algorithms, and accuracy metrics.The benchmark algorithms use combinatorial optimisation and factorial hidden Markov models.
2. BACKGROUND
NILM research uses diverse data sets, algorithms, and evaluation metrics, but incompatible formats and inconsistent benchmarks limit comparability. NILMTK provides a domain-specific basis for analysing these resources and standardising evaluation.
- Public data sets: Public data sets differ in household coverage, measurements, and formats, creating an engineering barrier to cross-data-set evaluation.These differences contribute to studies using only one data set and rarely investigating generality across households.
- Public data sets: Different data-set aims and formats have slowed uptake of data sets and prevented algorithms from being tested across multiple data sets.BLUED and AMPds, for example, had limited subsequent use for algorithm evaluation.
- Algorithms and benchmarks: Reference implementations and benchmark consensus are lacking, so publications often compare new methods against different or basic baselines.Examples include decoupled HMMs, Bayesian classifiers, variants of authors’ own approaches, and FHMMs.
- Evaluation metrics: Evaluation metrics vary across application areas, including multiple metrics labelled energy correctly assigned.This variety further complicates comparisons of disaggregation performance.
- Toolkit context: NILMTK’s pipeline runs from data-set import through disaggregation and evaluation over multiple metrics, with intermediate data and results storable on disk.Its domain-specific functionality complements rather than replaces general machine-learning toolkits.
- NILM formulation: NILM estimates each appliance’s power demand and state from household aggregate power readings.Appliances are commonly modelled with discrete states such as off, on, and intermediate.
3. NILMTK
NILMTK is a Python toolkit that standardizes data import and processing while supporting diagnostics, preprocessing, benchmark disaggregation, and accuracy evaluation across datasets.
- NILMTK provides a complete pipeline from dataset import through disaggregation evaluation over multiple accuracy metrics.The pipeline covers data formats, diagnostics and statistics, preprocessing, disaggregation, model import and export, and metrics.
- 3.1 Data Format: NILMTK-DF converts diverse public datasets into a common format, with importers for REDD, Smart*, Pecan Street, iAWE, AMPds, and UK-DALE.BLUED, Tracebase, and HES were excluded for stated data or time constraints.
- 3.1 Data Format: Standardized nomenclature and metadata enable appliance-data analysis across multiple datasets and support queries spanning countries or households.Labels for equivalent appliances and measured parameters are converted to a standard vocabulary, while metadata captures household, appliance, and meter properties.
- 3.2 Data Set Diagnostics: NILMTK diagnoses gaps, dropout rates, up-time, and other dataset characteristics before disaggregation evaluation.Its diagnostic functions include detecting gaps, quantifying recorded versus expected samples, isolating wireless-sensor dropout, and measuring recording duration.
- 3.4 Preprocessing of Data Sets: NILMTK preprocesses datasets through downsampling and voltage normalization, and can identify the top-k energy-consuming appliances for modeling.Downsampling supports mean, mode, and median aggregation; voltage normalization addresses differences in country-specific voltage fluctuations.
- 3.5 Training and Disaggregation Algorithms: NILMTK includes combinatorial optimisation and factorial hidden Markov model benchmarks, plus metrics combining general detection and energy-disaggregation measures.CO treats each time slice independently and becomes computationally tractable only for a small number of modeled appliances because its complexity is exponential in appliance count.
4. EVALUATION
NILMTK enables reproducible evaluation by diagnosing data quality, characterising appliance and household usage patterns, and comparing benchmark disaggregation algorithms across diverse data sets. The analyses show both cross-dataset variation and algorithm-specific trade-offs, including accuracy differences, computational cost, and model limitations.
- Data Set Diagnostics: NILMTK diagnostics revealed substantial data-quality differences, including missing-sample gaps and dropout periods in REDD, while AMPds and Pecan Street showed robust recording uptime.REDD house 1 had four large mains gaps and periods with dropout rates around 10%; AMPds and Pecan Street had 100% uptime and zero dropout rates.
- Data Set Statistics: Appliance power distributions range from simple on/off states in toasters and kettles to many-state profiles in washing machines, vacuum cleaners, and computers.The histograms compare selected appliances from UK-DALE and iAWE and illustrate why appliance models may require different state structures.
- Appliance Energy Profiles: Energy-use profiles varied strongly across countries: iAWE’s example household assigned almost half of total energy to two air conditioners, unlike the UK-DALE example household.REDD and UK-DALE showed some similarities in household energy breakdown, whereas iAWE differed substantially.
- Appliance Usage Patterns: Daily appliance usage patterns exposed relationships among appliances and household routines, including similar TV and Home theatre PC schedules and boiler activity in mornings and evenings.The boiler pattern was associated with occupancy and the hot-water timer, while the Home theatre PC was the only video source for the TV.
- External Dependencies: Boiler usage and external maximum temperature showed a strong correlation (R2 = 0.73), with an x-axis intercept of approximately 19 °C near the boiler thermostat set point.The relationship illustrates how external dependencies can be extracted from data sets for use in disaggregation analysis.
- Disaggregation Across Data Sets: FHMM outperformed CO across all three metrics on REDD, Smart* and AMPds, whereas their performance was similar on iAWE, Pecan Street and UK-DALE.The reported pattern was attributed to dataset-dependent appliance contributions and power demands; CO was exponentially quicker than FHMM, making it potentially preferable for some households.
- Detailed Disaggregation Results: A two-state model performed worse for iAWE air conditioner 2 because it frequently occupied an intermediate compressor-off, fan-on state.Both algorithms learned approximately 1.1 kW rather than the rated approximately 1.6 kW; the authors suggest a three-state model, at higher computational and memory cost.
5. CONCLUSIONS AND FUTURE WORK
NILMTK provides an open-source framework for empirical comparisons of energy disaggregation algorithms across multiple data sets. The paper demonstrates analyses supported by the toolkit and identifies broader data, algorithm, metadata, and simulation support as future work.
- Conclusions: NILMTK enables empirical comparisons of energy disaggregation algorithms across multiple data sets through a common data format, benchmarks, and accuracy metrics.It includes parsers for six publicly available data sets, preprocessing and statistics functions, and benchmark implementations based on combinatorial optimisation and factorial hidden Markov models.
- Conclusions: The demonstrated analyses include detecting missing data, learning appliance models, comparing algorithms across data sets, and breaking down performance by appliance.
- Future work: Future work will add recently proposed training and disaggregation algorithms and additional data sets, including larger data sets such as HES.
- Future work: Future extensions include comparing unsupervised-training algorithms, maintaining communal appliance metadata, and evaluating algorithms with a household simulator.The simulator would support evaluation in settings beyond those represented by publicly available data sets.
A. SAMPLE CODE FOR NILMTK PIPELINE
The sample code illustrates NILMTK’s complete pipeline, from loading and cleaning a data set through resampling, feature selection, splitting, training, disaggregation, and metric calculation.
- A. SAMPLE CODE FOR NILMTK PIPELINE: Algorithm 1 illustrates NILMTK’s pipeline through a minimal complete code example.
- A. SAMPLE CODE FOR NILMTK PIPELINE: The pipeline loads an HDF5 data set and removes records where voltage is below 160.
- A. SAMPLE CODE FOR NILMTK PIPELINE: Data are downsampled to one-minute intervals, and active power is selected as the disaggregation feature.
- A. SAMPLE CODE FOR NILMTK PIPELINE: The building data are divided into training and test sets before an FHMM disaggregator is trained and used for disaggregation.
B. NILMTK-DF
NILMTK-DF models data hierarchically, organizing households, sensors, metadata, utilities, electrical components, and physical measurements. Figure 9 depicts this hierarchy for structured data access.
- B. NILMTK-DF: Each data set contains one or more households, with utility, ambient, external, and household metadata components.
- B. NILMTK-DF: Electrical utility data are organized into mains, circuits or panels, and appliances.
- B. NILMTK-DF: Figure 9 presents the NILMTK-DF format hierarchy used to model data sets.
C. QUERY EXAMPLES
NILMTK supports a wide range of queries for exploring and analyzing the data represented in the toolkit.
- C. QUERY EXAMPLES: The section introduces examples of the wide range of queries supported by NILMTK.
C.1 Across data sets
This section examines cross-country differences in household energy use and appliance composition. It asks whether some appliances are specific to particular countries.
- Daily energy consumption is compared across countries.
- The analysis examines how instances of an appliance vary across countries.
- The analysis asks whether any appliances are country specific.
C.2 Within a data set
This section examines variation in power consumption within a data set. It considers seasonal, weekly, and temperature-related patterns.
- Power consumption is compared across seasons.
- Power consumption is compared between weekdays and weekends.
- HVAC power consumption is analyzed for correlation with temperature.
D. FUNCTIONS IN NILMTK
NILMTK provides statistical, diagnostic, and preprocessing functions for analyzing and preparing data. These functions are summarized in Tables 5–7.
- Statistical functions: NILMTK includes statistical functions for describing data sets.These functions are summarized in Table 5.
- Diagnostic functions: NILMTK includes diagnostic functions.These functions are summarized in Table 6.
- Preprocessing functions: NILMTK includes preprocessing functions.These functions are summarized in Table 7.
E. ADDING A NEW NILM ALGORITHM
NILMTK standardizes how new disaggregation algorithms are added through a scikit-learn- and R lm-inspired interface. Algorithms train on aggregate and sub-metered streams, disaggregate using learned appliance models, and support model import and export.
- NILMTK models its algorithm interface on scikit-learn and the R lm package.The design aims to make new algorithms easy to add.
- The train function receives a building, disaggregation features, an aggregate stream, and a sub-metered stream.Features may include active power or active and apparent power.
- The disaggregate function uses appliance models learned during training to convert an aggregate feed into individual appliance streams.
- The import model function imports JSON appliance models into the NILMTK disaggregator.
- The export model function writes the learned model to an external destination.