PaperlayerView exact revision on arXiv

Source-linked AI summary

Forecasting Electricity Smart Meter Data Using Conditional Kernel Density Estimation

Siddharth Arora, James W. Taylor

arXiv:1409.2856v1stat.AP

TL;DR

The paper addresses modelling electricity consumption densities from individual smart meters to support energy and financial decision-making. It proposes conditional kernel density methods that capture seasonality, outperform unconditional kernel density estimation, and support tariff switching that yields considerable cost savings.

  • Problem

    Modelling the consumption density of individual smart meters matters because detailed smart-meter information can support energy and financial savings.

  • Method

    The paper proposes kernel density and conditional kernel density estimators, with conditioning designed to capture seasonality.

  • Results

    The considered methods convincingly outperformed the unconditional KD estimator, while switching between tariffs based on cost densities produced considerable overall savings.

  • Takeaways & Limitations

    Density forecasts can support tariff-switching decisions that reduce electricity costs compared with assigning consumers a single tariff for all periods.

  • Takeaways & Limitations

    The approach does not assume weather predictions are available and affordable near each smart meter.

Abstract

from arXiv · show

The recent advent of smart meters has led to large micro-level datasets. For the first time, the electricity consumption at individual sites is available on a near real-time basis. Efficient management of energy resources, electric utilities, and transmission grids, can be greatly facilitated by harnessing the potential of this data. The aim of this study is to generate probability density estimates for consumption recorded by individual smart meters. Such estimates can assist decision making by helping consumers identify and minimize their excess electricity usage, especially during peak times. For suppliers, these estimates can be used to devise innovative time-of-use pricing strategies aimed at their target consumers. We consider methods based on conditional kernel density (CKD) estimation with the incorporation of a decay parameter. The methods capture the seasonality in consumption, and enable a nonparametric estimation of its conditional density. Using eight months of half-hourly data for one thousand meters, we evaluate point and density forecasts, for lead times ranging from one half-hour up to a week ahead. We find that the kernel-based methods outperform a simple benchmark method that does not account for seasonality, and compare well with an exponential smoothing method that we use as a sophisticated benchmark. To gauge the financial impact, we use density estimates of consumption to derive prediction intervals of electricity cost for different time-of-use tariffs. We show that a simple strategy of switching between different tariffs, based on a comparison of cost densities, delivers significant cost savings for the great majority of consumers.

1. Introduction

Smart-meter data are highly variable and seasonal, motivating density forecasts and nonparametric methods that capture seasonality without distributional assumptions. The paper evaluates kernel-based forecasts and uses consumption densities to compare time-of-use tariffs.

  • 1. Introduction: Density forecasts can assess decision risk and support consumption estimates at varying aggregation levels.The introduction motivates densities rather than point forecasts for complex decisions and demand-side management.
  • 1. Introduction: The paper addresses the lack of studies modelling the density of individual smart-meter electricity consumption.Individual-meter data are highly variable and exhibit intraday and intraweek seasonality.
  • 1. Introduction: The study proposes KD and CKD methods that capture seasonality through conditioning on temporal and lagged-consumption variables.The CKD implementations condition on period of week, period of day, and lagged consumption.
  • 1. Introduction: A decay factor models temporal evolution in the relationship between consumption and conditioning variables.
  • 1. Introduction: The methods are evaluated on eight months of half-hourly data from one thousand meters across lead times from one half-hour to one week.
  • 1. Introduction: Density estimates are used to derive cost prediction intervals and select tariffs with the greatest potential savings.The strategy compares future costs across available tariffs and selects the lowest-cost option.

2. Smart meter data

The study uses eight months of half-hourly consumption data from 1,000 Irish smart meters, with separate residential and SME patterns. The data exhibit recurring intraday and intraweek seasonality, which motivates density forecasts that capture both cycles.

  • Consumption patterns: Residential consumption was volatile, whereas SME consumption displayed a prominent seasonal pattern across the eight-month sample.
  • Consumption patterns: SME consumption was higher on weekdays and concentrated during working hours, while residential consumption peaked during evenings without noticeable day-of-week differences.
  • Seasonality: The data showed repeating intraday and intraweek cycles, so the proposed density methods targeted double seasonality with cycle lengths of 48 and 336 half-hour observations.
  • Data preparation: Seven special days were identified and adjusted using historical comparisons, while the authors note holiday effects as an area for future work.
  • Scalable estimation: To scale estimation across consumers, parameters were estimated for up to 10% within each tariff-and-stimulus category and median values were applied category-wide.

3. Forecasting methods

The forecasting framework compares unconditional and conditional kernel density estimators for smart-meter consumption. Conditional methods estimate the full nonlinear density by weighting observations according to conditioning variables such as seasonality and lagged consumption.

  • Implementation: The study applies these estimators to standardized individual-consumer electricity observations and uses Gaussian kernels to construct full density estimates.
  • Kernel estimators: Unconditional kernel density estimation uses one kernel direction, whereas conditional estimation applies kernels in both the conditioning-variable and outcome directions.
  • Conditional kernel density estimation: Conditional kernel density estimation models the entire conditional density nonparametrically, unlike linear regression, which focuses on a conditional expectation.
  • Related applications: The forecasting literature contains relatively few CKD applications, including studies of temperature, respiratory response, and wind-power density forecasting.

3.2 Kernel bandwidth selection and boundary correction

Bandwidth choice balances smoothness against local detail, while boundary correction addresses bias caused by consumption being standardized to the interval from 0 to 1. The study selects bandwidths by cross-validation and reduces them near boundaries.

  • Bandwidth selection: A large bandwidth can over-smooth the density, whereas a small bandwidth can produce an overly rough estimate and allocate mass outside the valid range.
  • Bandwidth selection: The study uses data-based cross-validation, minimizing one-step-ahead density prediction error on a one-month hold-out sample to estimate bandwidths.
  • Boundary correction: Because consumption values lie between 0 and 1, symmetric kernels create boundary bias and may extend estimated density beyond that interval.
  • Boundary correction: The adopted boundary correction reduces bandwidths near the lower and upper limits, with the reduction determined by proximity to a boundary.
  • Boundary correction: Boundary correction was incorporated during both parameter estimation and post-sample evaluation for all kernel-based methods.

3.3 Unconditional kernel density estimation (KD-U)

KD-U is the unconditional kernel-density benchmark, estimated over a moving window of recent observations. Unlike the remaining methods, it does not attempt to capture consumption seasonality.

  • KD-U specification: KD-U applies unconditional kernel density estimation to the most recent observations within a six-month moving window.
  • KD-U specification: KD-U does not account for seasonality, whereas all remaining methods aim to accommodate seasonal structure in the consumption series.

3.4 Separate kernel density estimation for each period of the week (KD-W)

KD-W estimates a separate consumption density for each weekly period, using nearby historical observations and exponentially decaying weights that emphasize recency.

  • 3.4 Separate kernel density estimation for each period of the week (KD-W): KD-W estimates each weekly period’s density from historical observations belonging to the same period of the week.The method applies kernel density estimation within a moving window of periods.
  • 3.4 Separate kernel density estimation for each period of the week (KD-W): An exponential decay parameter gives greater emphasis to recent observations within the density estimation framework.A lower decay parameter produces faster weight decay and stronger emphasis on recent observations.
  • 3.4 Separate kernel density estimation for each period of the week (KD-W): KD-W uses a bandwidth for observations from the same weekly period and applies uniform decay weights within each week.The seasonal discounting is included across the methods described in the section.
  • 3.4 Separate kernel density estimation for each period of the week (KD-W): CKD-W extends the approach by conditioning consumption on the period of the week and weighting observations according to their temporal proximity.Its weekly-period distance treats the beginning and end of the week as adjacent rather than far apart.
  • 3.4 Separate kernel density estimation for each period of the week (KD-W): Fig. 4 shows 24 weeks of SME consumption plotted against the period of the week for the CKD-W construction.The plot provides the observation layout to which the conditional estimator is applied.

3.6 CKD estimation conditional on the period of week and period of day (CKD-WD)

CKD-WD extends weekly-period conditioning by also conditioning consumption on the period of the day, while distinguishing weekday and weekend intraday cycles.

  • 3.6 CKD estimation conditional on the period of week and period of day (CKD-WD): CKD-WD conditions consumption on both the period of the week and the period of the day.The method extends the CKD-W estimator to accommodate intraweek and intraday cycles.
  • 3.6 CKD estimation conditional on the period of week and period of day (CKD-WD): The period-of-day conditioning uses a separate bandwidth for the day-period dimension.This bandwidth is defined for conditioning on the period of the day.
  • 3.6 CKD estimation conditional on the period of week and period of day (CKD-WD): The method treats weekdays separately from weekends and uses distinct period-of-day bandwidths for the two groups.Weekdays share one intraday cycle, while Saturdays and Sundays share another.
  • 3.6 CKD estimation conditional on the period of week and period of day (CKD-WD): CKD-WD builds on prior intraday-cycle modelling that classifies days according to similarity in their intraday consumption patterns.The related parametric approach is identified as intraday cycle exponential smoothing.

3.8 CKD estimation conditional on type of intraday cycle (CKD-IC)

CKD-IC pools observations sharing weekday or weekend intraday cycles, combining cycle-based conditioning with separate smoothing for weekdays and weekends.

  • 3.8 CKD estimation conditional on type of intraday cycle (CKD-IC): CKD-IC synthesizes CKD-W with an intraday-cycle classification that models weekdays separately from weekends.The method uses distinct bandwidths for weekday and weekend observations.
  • 3.8 CKD estimation conditional on type of intraday cycle (CKD-IC): For each weekday period, CKD-IC combines 24 same-period observations with 24 observations from each corresponding period on other weekdays.This produces 120 plotted observations for each weekday period; weekend periods pool Saturday and Sunday observations.
  • 3.8 CKD estimation conditional on type of intraday cycle (CKD-IC): Separate bandwidths smooth weekday and weekend observations differently along the relevant dimension.This design preserves distinct weekday and weekend treatment within the estimator.
  • 3.8 CKD estimation conditional on type of intraday cycle (CKD-IC): The method applies CKD to observations grouped by matching intraday-cycle types across the weekly period.Fig. 5 displays the corresponding 24-week observation structure for an SME.
  • 3.8 CKD estimation conditional on type of intraday cycle (CKD-IC): CKD-Lag conditions a period’s consumption density on consumption observed during the same period of the previous week.The method is described as a nonparametric analogue to seasonal autoregressive modelling.
  • 3.8 CKD estimation conditional on type of intraday cycle (CKD-IC): The decay factors are close to one, with 0.942 corresponding to an approximately twelve-week half-life and 0.977 to approximately thirty weeks.These values indicate slowly decaying weights over time.

3.11 Sophisticated Benchmark

The sophisticated benchmark uses seasonal Holt-Winters-Taylor exponential smoothing to model nested intraday and intraweek cycles and generate point and density forecasts.

  • 3.11 Sophisticated Benchmark: The benchmark is a seasonal Holt-Winters-Taylor exponential smoothing method with nested intraday and intraweek cycles.It has previously been used to model total national consumption.
  • 3.11 Sophisticated Benchmark: HWT estimates smoothing parameters for level, intraday seasonality, intraweek seasonality, and residual autocorrelation.The parameters are denoted λ, δ, ω, and ø, respectively.
  • 3.11 Sophisticated Benchmark: The HWT parameters are estimated by maximizing a Gaussian likelihood using the same training set and categorization scheme as the kernel methods.This aligns the benchmark’s estimation setup with the KD and CKD methods.
  • 3.11 Sophisticated Benchmark: HWT density forecasts are generated with Monte Carlo simulation using 10000 iterations.
  • 3.11 Sophisticated Benchmark: Average HWT parameters are reported separately for residential consumers and SMEs.Residential averages are λ = 0.009, δ = 0.019, ω = 0.1203, and ø = 0.420; SME averages are λ = 0.014, δ = 0.050, ω = 0.139, and ø = 0.592.

4. Evaluating post-sample forecast accuracy

Post-sample evaluations show that kernel-based density and point forecasts capture consumption seasonality, with performance varying between residential consumers and SMEs.

  • Density forecasts: Residential-consumer density-forecast performance was similar across KD-W, CKD-W, CKD-WD, KD-IC, and CKD-IC.KD-IC was presented as a representative method because it was among the most accurate.
  • Density forecasts: CRPS increased during periods with relatively large consumption changes, while forecast figures retained the seasonal patterns identified earlier.The evaluation covered half-hourly lead times from one half-hour to one week ahead.
  • Density forecasts: KD-W, CKD-W, and CKD-WD were the best-performing kernel methods for SME density forecasts, while HWT was also competitive.KD-IC and CKD-IC performed similarly but were outperformed by CKD-W and CKD-WD; KD-U was substantially outperformed.
  • Quantile forecasts: KD-IC and HWT produced very good one-day-ahead unconditional coverage across residential-consumer quantiles.For six-hour horizons, KD-IC coverage was closer to perfect than KD-U, while CKD-Lag performed worst among CKD methods.
  • Point forecasts: KD-IC was slightly more accurate than HWT at nearly all residential-consumer horizons under MAE.For SMEs, HWT was slightly more accurate than CKD-W for lead times shorter than one day and at horizons with the highest MAE.

5. Deriving prediction intervals for electricity cost

The study converts consumption density forecasts into cost-density forecasts under multiple tariffs and uses them to compare tariff-selection strategies.

  • Cost-density construction: Cost-density forecasts were generated by sampling consumption realizations, multiplying by tariff-specific electricity costs, and forming a histogram of the resulting costs.Six cost-density estimates were derived for each consumption-density forecast, one for each tariff.
  • Data scope: The tariff analysis included only 580 residential consumers outside the control group because the control group’s applicable tariffs were unavailable.SMEs were excluded because their specific CER-allocated TOU tariffs were also unavailable.
  • Tariff switching: Switching tariffs reduced costs for the great majority of consumers under all three selection criteria.The comparison evaluated switching against retaining each consumer’s allocated tariff across four post-sample weeks.
  • Tariff switching: Switching based on the 75% and 95% predicted-cost quantiles produced noticeably greater average savings than switching based on the predicted mean.The quantile criteria represent risk-averse and very risk-averse strategies, respectively.

6. Summary and concluding remarks

The study models smart-meter consumption densities with seasonality-aware kernel methods and a decay parameter, then evaluates their forecasting and tariff applications.

  • Summary: The evaluated methods convincingly outperformed the unconditional KD estimator and accommodated weekly seasonality in consumption.The framework used a decay parameter to emphasize more recent observations.
  • Summary: HWT exponential smoothing produced particularly strong point and density forecasts and was competitive for SME series with strong seasonal patterns.The authors attribute this SME performance to the relatively strong seasonality typical of those series.
  • Summary: The study derived electricity-cost density forecasts for six tariffs and found considerable savings from switching instead of retaining a single allocated tariff.Three tariff-selection criteria were evaluated empirically.
  • Future work: Future work could model holiday effects separately and investigate clustering methods to scale the framework to many consumers without huge computational costs.These extensions are presented as potential research directions rather than evaluated components.
Loading 1409.2856v1…