PaperlayerView exact revision on arXiv

Source-linked AI summary

Behind-the-Meter Photovoltaic Visibility from Net Smart Meters: Deployment Regimes and Operational Insights

Jack Zheng, Hao Wang

arXiv:2610.12527v1cs.LG

TL;DR

Residential net smart meters report consumption minus PV generation, leaving behind-the-meter PV visibility underdetermined and making deployment setting important to interpreting disaggregation error. The paper evaluates six sequence models across four deployment regimes on Ausgrid and adapts across datasets to Pecan Street. Deployment regime explains more error variation than model family, and fine-tuning with five target households reduces transfer error.

  • Problem

    Net smart meters report the difference between consumption and PV generation, while household-specific signatures can limit transfer and random timestamp splits leave new-customer and future-period risk unmeasured.

    Source: §I · p. 1

  • Method

    The study evaluates six sequence models across four regimes separating household transfer from temporal extrapolation on Ausgrid, with cross-dataset adaptation to Pecan Street.

    Source: §I · p. 1, B. Operational Deployment Regimes · p. 3

  • Results

    Deployment regime explained 78.6% of error variation versus 18.0% for model family; best-model daylight MAE rose from 0.104 kW on seen to 0.185 kW on unseen households under identical future conditions.

    Source: §I · pp. 1–2

  • Takeaways & Limitations

    Visibility error budgets should match deployment regime; longer windows help known-customer estimates, while new-customer context-length effects vary by model.

    Source: §I · p. 2, §VI · p. 6

Abstract

from arXiv · show

Distribution operators need visibility of behind-the-meter photovoltaic (PV) generation, yet most residential customers are observed only through net smart meters. Deep-learning-based PV disaggregation can restore this visibility, but its operational value depends on whether the model serves known customers, new customers, future periods, or new regions. We quantify this dependence using four deployment regimes that separate household transfer from temporal extrapolation, applied to Ausgrid smart-meter data with cross-dataset adaptation to the Pecan Street data. Using a fixed training protocol, we benchmark representative deep sequence architectures for PV disaggregation under regime-matched evaluation. At a fixed 48-hour context length, daylight mean absolute error (MAE) of the best model rises from 0.104~kW on seen households in the future test period to 0.185~kW on unseen households over the same period, and under a fixed training contract the deployment regime explains far more of the error variation than model choice. Under target-domain normalization, fine-tuning on five target-region households reduces source-model transfer error by 28\%--36\%. Long input windows help known-customer extrapolation, while their new-customer benefits vary by model. These findings suggest regime-matched error budgets, limited target supervision for model adaptation in new regions, and use-case-dependent context selection for energy data analytics.

I. INTRODUCTION

Behind-the-meter PV disaggregation is underdetermined, and deployment-specific error matters because familiar-customer tests can miss the risks of serving new customers or future periods. Across the evaluated settings, deployment regime matters more than model family, while limited target-region supervision reduces transfer error and context-length benefits depend on the customer setting.

I. INTRODUCTION

  • A net-load measurement is underdetermined because infinitely many nonnegative consumption-generation pairs can produce the same value, while household-specific signatures can limit transfer.PV orientation, capacity, shading, appliance composition, and occupancy contribute to household-specific signatures.Source: §I · p. 1
  • The study evaluates six sequence models across four regimes separating household transfer from temporal extrapolation on Ausgrid, with cross-dataset adaptation to Pecan Street.Source: §I · p. 1
  • Daylight MAE rose from 0.104 kW on seen households to 0.185 kW on unseen households under identical future conditions.The comparison uses the same final test period, isolating household familiarity.Source: §I · p. 1
  • Deployment regime explained 78.6% of error variation, compared with 18.0% for model family across six sequence models under a fixed development practice.Source: §I · p. 2
  • Approximately 28%–36%: fine-tuning an Ausgrid model on five Pecan Street households reduced source-model transfer error under target-domain normalization.Returns saturated rapidly.Source: §I · p. 2
  • Longer input windows consistently improve estimates for known customers, while new-customer results vary by model.Source: §I · p. 2

II. RELATED WORK

Prior PV-disaggregation work spans physical or proxy-based methods and learned approaches, while deployment-regime reporting often leaves household and time overlap unclear. The paper positions its regime analysis as complementary to architectural work and related transfer concerns in non-intrusive load monitoring.

II. RELATED WORK

  • Physical and proxy-based methods use irradiance references, clear-sky models, or capacity estimates, while learned approaches include CNN-BiLSTM, graph, refinement, diffusion, and transfer-oriented models –,,.Physical approaches can require proxy systems, capacity information, or assumptions about orientation and shading that may be unavailable for residential customers.Source: §II · p. 2
  • Many studies report splits without enough household/time-overlap detail to identify the tested deployment regime; where splits are clear, most evaluate seen households or one regime.Table I audits which regimes are used, marks insufficient overlap detail, and identifies cross-dataset transfer.Source: §II · p. 2, Table I · p. 2
  • Held-out-house and cross-dataset transfer concerns also arise in non-intrusive load monitoring, and this work complements architectural studies through operational-regime analysis.Source: §II · p. 2

III. DISTRIBUTION-GRID VISIBILITY PROBLEM AND DEPLOYMENT REGIMES

The study frames PV visibility as predicting generation from net-meter and solar-weather windows, with deployment regimes separating household transfer from temporal extrapolation.

III. DISTRIBUTION-GRID VISIBILITY PROBLEM AND DEPLOYMENT REGIMES

B. Operational Deployment Regimes

The four deployment regimes distinguish household familiarity from time period, enabling comparisons that align evaluation with intended use.

B. Operational Deployment Regimes

  • S2 and S4 share the final 183-day test period, isolating the effect of household familiarity.S2 evaluates future visibility for known customers, while S4 evaluates future visibility for customers absent from model development.Source: B. Operational Deployment Regimes · p. 3
  • The study holds out 20% of households per split seed, yielding 239 seen and 60 unseen households; S1 and S3 use pre-test days.S1 holds out calendar days from seen households, while S3 holds out households and serves as a household-transfer diagnostic.Source: B. Operational Deployment Regimes · p. 3
  • Validation uses pre-test data from seen households, and samples crossing split boundaries are rejected to prevent temporal-window leakage.Held-out households are excluded from training, validation, early stopping, and normalization statistics.Source: B. Operational Deployment Regimes · p. 3

IV. STUDY DESIGN

The study benchmarks six deep sequence models on Ausgrid under a fixed training contract, then evaluates cross-dataset transfer to Pecan Street.

A. Datasets and Features

  • The Ausgrid panel contains 299 customers at 30-minute resolution from January 2011 to June 2013, while Pecan Street contributes 17 Austin households over one year.Net load is synthesized as consumption minus gross PV generation, and solar-weather features are aligned to meter timestamps.Source: A. Datasets and Features · p. 3

B. Visibility Models and Training Contract

  • Six sequence models receive the same six input channels: CNN-BiLSTM, TCN, TFT, TimesNet, Mamba, and PatchTST.Source: B. Visibility Models and Training… · p. 3
  • Hyperparameters are selected on the Ausgrid S1 validation split at W = 96 and then frozen across all regimes.The fixed contract uses early stopping on validation daylight MAE, nonnegative softplus outputs, and zero-clamping at night.Source: B. Visibility Models and Training… · p. 3
  • The main Ausgrid comparison fixes W at 96 half-hourly steps and trains with daylight MAE plus an integrated PV-energy penalty.For varying context lengths, the study uses a window-normalized energy loss with fixed λE = 0.35.Source: B. Visibility Models and Training… · p. 3
  • Pecan Street adaptation fine-tunes on reference-household records, with target-normalized source weights used as the unadapted comparison.The evaluation draws k ∈ {1, 3, 5, 10} reference households and fine-tunes for 20 epochs.Source: B. Visibility Models and Training… · pp. 3–4

C. Uncertainty and Reproducibility

Reported uncertainty summarizes variability across household splits for both datasets.

C. Uncertainty and Reproducibility

  • Ausgrid results average five household splits with 95% BCa intervals, while Pecan Street reports mean and 95% t intervals across five target-household splits.Each Pecan Street split evaluates three households, with 10 unique evaluation households overall.Source: C. Uncertainty and Reproducibility · p. 4

V. RESULTS AND DISCUSSION

On Ausgrid, moving the same pipeline from known to unseen customers raises daylight PV-visibility error, while deployment regime accounts for more error variation than model family. Model rankings remain useful for coarse screening, but error budgets require regime-matched evaluation.

A. Ausgrid: New-Customer Visibility Error is Much Higher

  • 0.104 kW under S2 rises to 0.185 kW under S4 for CNN-BiLSTM on the same final test period, isolating the effect of household familiarity.Leading models remain close within each regime.Source: Figure 3 · p. 4, A. Ausgrid: New-Customer Visibility… · p. 4
  • 78.6% of cell-mean variation is attributed to deployment regime, versus 18.0% to model family and 3.4% to residual interaction.This decomposition applies to the tested grid under the fixed training contract.Source: Figure 4 · p. 4, A. Ausgrid: New-Customer Visibility… · p. 4
  • S1 and S4 model rankings are highly correlated, but planning studies using uninstrumented customers should budget against S4-type error, nearly twice the known-customer figure.Known-customer tests remain useful for coarse model screening; deployment error levels still require regime-matched evaluation.Source: A. Ausgrid: New-Customer Visibility… · p. 4

B. Target-Region Fine-Tuning Improves Cross-Region Trans-

Under target-domain normalization, adapting an Ausgrid model with a small Pecan Street reference cohort reduces transfer error, with gains saturating as more households are added.

B. Target-Region Fine-Tuning Improves Cross-Region Transfer

C. Operational Value of Historical Context

Context length has different value across deployment regimes: longer histories consistently help known-customer future estimates, while new-customer preferences vary by model and gains are less stable.

C. Operational Value of Historical Context

  • The context-length sweep uses fewer split seeds, a different loss normalization, and restricted timestamp support, so its absolute MAE values differ from Table IV.The analysis is intended to compare trends across context lengths.Source: C. Operational Value of Historical… · p. 5
  • Under S2, all five models achieve their best mean MAE at W = 192, improving over W = 24 by 0.0146 to 0.0346 kW.Under S4, CNN-BiLSTM and TimesNet favor W = 48, TCN W = 96, TFT W = 24, and Mamba W = 192.Source: Table VI · p. 5, C. Operational Value of Historical… · pp. 5–6
  • Under S4, context preferences vary by model, and the differences are smaller and less stable than the S2 gains.The authors suggest choosing new-customer window size according to data-availability and latency constraints.Source: C. Operational Value of Historical… · p. 6

D. Data Quality and Deployment Considerations

The experiments use cleaned, synchronized 30-minute records, and the conclusion calls for regime-matched error budgets, leakage-safe splits, and household-level uncertainty in PV-visibility results.

D. Data Quality and Deployment Considerations

  • The experiments use cleaned, synchronized 30-minute records; sensitivity to outages, delays, misalignment, and measurement noise remains future work.Online deployment also requires causal windows, inference-time weather, and testing under representative faults.Source: D. Data Quality and Deployment… · p. 6

VI. CONCLUSION

  • PV-visibility error nearly doubled for customers outside the training population, while longer input windows helped known-customer estimates and new-customer context effects varied by model.The conclusion calls for regime-matched error budgets, leakage-safe splits, and household-level uncertainty; adaptation gains remain subject to validation on larger datasets.Source: §VI · p. 6

Related papers

Loading 2610.12527v1…