PaperlayerView exact revision on arXiv

Source-linked AI summary

A Novel Combined Data-Driven Approach for Electricity Theft Detection

Kedi Zheng, Qixin Chen, Yi Wang, Chongqing Kang, Qing Xia

arXiv:2411.06649v1eess.SYcs.LGeess.SP

TL;DR

Electricity-theft detection must handle increasingly diverse tampering without relying on labeled data or difficult additional system information. The paper combines MIC for NTL–load associations with CFSFDP for abnormal load shapes, and reports strong, steady performance across FDI types, including improved MIX detection.

  • Problem

    Existing electricity-theft detection methods may require labeled datasets or difficult additional system information, while clustering can miss tampered profiles with normal shapes.

  • Method

    The framework combines MIC to detect associations between area NTL and tampered load profiles with CFSFDP to detect abnormal load-profile shapes, then combines their suspicion ranks.

  • Results

    The combined methods achieved the best MIX detection accuracy, increasing MIX AUC from 0.748 to 0.816 and MAP@20 from 0.693 to 0.831.

  • Takeaways & Limitations

    The combined method maintains the techniques’ specialized strengths while providing high and steady detection accuracy across FDI1 to FDI6.

Abstract

from arXiv · show

The two-way flow of information and energy is an important feature of the Energy Internet. Data analytics is a powerful tool in the information flow that aims to solve practical problems using data mining techniques. As the problem of electricity thefts via tampering with smart meters continues to increase, the abnormal behaviors of thefts become more diversified and more difficult to detect. Thus, a data analytics method for detecting various types of electricity thefts is required. However, the existing methods either require a labeled dataset or additional system information which is difficult to obtain in reality or have poor detection accuracy. In this paper, we combine two novel data mining techniques to solve the problem. One technique is the Maximum Information Coefficient (MIC), which can find the correlations between the non-technical loss (NTL) and a certain electricity behavior of the consumer. MIC can be used to precisely detect thefts that appear normal in shapes. The other technique is the clustering technique by fast search and find of density peaks (CFSFDP). CFSFDP finds the abnormal users among thousands of load profiles, making it quite suitable for detecting electricity thefts with arbitrary shapes. Next, a framework for combining the advantages of the two techniques is proposed. Numerical experiments on the Irish smart meter dataset are conducted to show the good performance of the combined method.

NOMENCLATURE

The nomenclature defines the user sets, data objects, indices, variables, parameters, and functions used throughout the paper.

  • A denotes all users, B benign users, F fraudulent users, and D ordered data pairs.
  • t indexes time intervals, i electricity users, p and q load profiles, and j days.
  • x_i,t and x_i are ground-truth consumption and load profiles, while ˜x_i,t and ˜x_i are recorded values.
  • u_i,j is a normalized daily load profile, E_t is area ground-truth consumption, and e_t and e represent non-technical loss.
  • G discretizes ordered data pairs, M is a characteristic matrix, and ρ_p, δ_p, d_p,q, d_c, and ζ_p parameterize CFSFDP.
  • MIC measures maximum information coefficient, χ is the CFSFDP kernel, Corr measures correlation, and I denotes mutual information.

I. INTRODUCTION

Electricity theft increasingly exploits smart-meter data, while existing detection methods face trade-offs involving labels, system information, and detection coverage. The paper proposes combining correlation-based and unsupervised approaches using broadly available observer-meter data.

  • Smart meters enable high-frequency consumption analysis but can be tampered with digitally or through cyber attacks, diversifying theft forms.
  • Traditional inspections and video surveillance are time-consuming and labor-intensive, motivating detection methods that exploit Energy Internet information flow.
  • AI classification requires labeled theft data, clustering misses tampered profiles with normal shapes, and state-based methods require difficult-to-obtain measurements and system information.
  • The proposed framework combines suspicion ranks from shape similarity and magnitude correlation while using observer-meter data available to many DSOs and retailers.
  • The study applies MIC and CFSFDP to abnormal detection and evaluates the framework across different electricity-theft behaviors against several existing methods.

II. PROBLEM STATEMENT

The problem setting assumes an observer meter for an area containing multiple customers and models theft through time-varying false-data injections in smart-meter records.

  • A. Observer Meters: An observer meter records area-level consumption for a customer group and is assumed difficult for fraudulent users to tamper with.
  • A. Observer Meters: The framework assumes distribution operators or electricity retailers can access observer-meter data.
  • B. False Data Injection: False-data injections simulate theft by modifying smart-meter records, generally reducing the quantity of billed electricity.
  • B. False Data Injection: The six modeled FDI types either preserve original-curve features and fluctuations or create new patterns.

C. A State-based Method of Correlation

The correlation method uses area non-technical loss and recorded user load profiles to identify suspicious associations, while the overall framework combines these rankings with shape-based clustering.

  • Area non-technical loss is obtained by subtracting the observer-meter reading from the sum of customers’ recorded smart-meter consumption.
  • When tampering preserves positive correlation with true consumption, fraudulent users should show stronger correlation between recorded load and area NTL than benign users.
  • MIC is used to detect associations between area NTL and tampered profiles, including relationships that are fuzzy rather than strictly linear.
  • The correlation relation may fail for FDI6, which produces a totally random curve, so correlation alone cannot detect every theft type.
  • The combined framework merges suspicion ranks from MIC and CFSFDP to exploit both correlation and abnormal-shape information.

A. Maximum Information Coefficient

MIC measures associations beyond linear correlation by maximizing mutual information over grid partitions. In this framework, it is applied to relate consumer load behavior to area non-technical loss and detect thefts with strong relevance to that loss.

  • Maximum Information Coefficient: MIC detects diverse associations, whereas Pearson correlation cannot detect sophisticated or time-variant relations such as quadratic and cubic associations.Mutual information detects all types of associations, and MIC is reported to outperform MI in many occasions.
  • Maximum Information Coefficient: MIC is computed by maximizing normalized mutual information over grids partitioning the x- and y-values of a finite paired dataset.The characteristic matrix records the resulting grid-based values before MIC selects the maximum under the grid-size constraint.
  • Maximum Information Coefficient: MIC values range from 0 to 1, with larger values indicating stronger associations.The method uses the grid-size bound B(|D|) = |D|^0.6.
  • Maximum Information Coefficient: The framework applies MIC as its correlation measure to detect consumer behaviors strongly relevant to area NTL.This targets thefts whose consumption behavior is associated with the area's non-technical loss.

B. CFSFDP-based Unsupervised Detection

CFSFDP detects abnormal load profiles by exploiting local density and distance structure rather than assuming a particular cluster shape. Its abnormality score highlights profiles that are sparse and far from high-density regions, complementing MIC for theft patterns MIC may miss.

  • CFSFDP-based Unsupervised Detection: CFSFDP uses clustering to identify outliers among numerous load profiles that correlation-based detection cannot capture.The method is described as suitable for large-dataset clustering and outlier detection.
  • CFSFDP-based Unsupervised Detection: For each load profile, CFSFDP defines local density ρp and distance δp from other profiles with higher density using pairwise distances.The cut-off distance dc and kernel function χ(·) determine the density calculation.
  • CFSFDP-based Unsupervised Detection: A Gaussian kernel can estimate the discrete local density ρp to avoid conflicts, while δp is defined separately for profiles relative to higher-density profiles.The highest-density profiles receive a conventional definition of δp.
  • CFSFDP-based Unsupervised Detection: Abnormal profiles typically have small ρp and large δp because they have few neighbors and lie farther from high-density regions.The example distribution identifies points #26–28 as abnormal and links their spatial distribution to the ρp–δp pattern.
  • CFSFDP-based Unsupervised Detection: Density-based clustering accommodates arbitrary-shaped clusters without parameter selection, and high ζp values flag strange profiles that MIC may not consider.CFSFDP obtains δp and ζp without iteration once local densities are calculated.

C. Combined Detecting Framework

The combined framework computes complementary MIC- and CFSFDP-based suspicion measures for normalized daily load profiles. It clusters daily scores into suspicious and less-suspicious groups, ranks consumers separately, and combines the ranks using arithmetic and geometric means.

  • Combined Detecting Framework: The framework first calculates area NTL, normalizes each consumer-day load profile, and computes MIC, ρi,j, δi,j, and ζi,j for every consumer and day.Normalization divides each recorded profile by its maximum and preserves load-curve shape for arbitrary-shape clustering.
  • Combined Detecting Framework: For each consumer, k-means with k = 2 separates the m daily MIC or ζ values into suspicious and less-suspicious groups.The mean score of the more suspicious group becomes that consumer's suspicion degree.
  • Combined Detecting Framework: The method produces two consumer suspicion ranks by comparing the n×m MIC and ζ values across consumers.The two ranks represent the correlation-based and density-based detection results.
  • Combined Detecting Framework: The two ranks are combined using arithmetic and geometric means following the Rank Product approach.The combined rank integrates the outputs of the two independent but complementary methods.
  • Combined Detecting Framework: A consumer is considered to commit electricity theft when its combined rank is high.

IV. NUMERICAL EXPERIMENTS

The experiments evaluate theft detection on Irish smart-meter data using correlation and unsupervised outlier baselines. They use AUC and MAP, benchmark against random guessing, and average results across 100 randomly generated scenarios.

  • NUMERICAL EXPERIMENTS: The dataset contains 391 SMEs observed over 30 days, yielding 11 730 load profiles from the Irish CER Smart Metering Project.The broader project contains over 5000 users with more than 500 days of data, while this experiment uses the SME subset from July 15 to August 13, 2009.
  • NUMERICAL EXPERIMENTS: Comparisons include Pearson correlation, Kraskov’s mutual-information estimator, Fuzzy C-Means, and Density-based Local Outlier Factor.These methods represent alternative correlation-analysis and unsupervised outlier-detection approaches.
  • NUMERICAL EXPERIMENTS: Evaluation uses AUC and MAP, with MAP@20 adopted for ranking-based assessment.AUC measures area under the ROC curve, while MAP@N averages precision at the positions of ranked thieves.
  • NUMERICAL EXPERIMENTS: Random guessing provides benchmarks of AUC = 0.5 and MAP equal to the proportion of electricity thieves among all users.These values follow from equal true- and false-positive rates under random guessing.
  • NUMERICAL EXPERIMENTS: All numerical experiments are repeated for 100 randomly generated scenarios, and reported AUC and MAP values are means across scenarios.

C. Numerical Results

The numerical experiments compare correlation-based, clustering-based, and combined methods across six FDI types, showing complementary strengths and improved overall performance from combining MIC with CFSFDP.

  • Method comparisons: MIC provides balanced AUC and MAP@20 performance for FDI1 and performs strongly on type MIX.Correlation-based methods perform poorly on FDI5 and FDI6, where tampered profiles become random and correlations disappear.
  • Method comparisons: CFSFDP performs best among clustering methods for FDI5, FDI6, and type MIX, with higher MAP@20 than LOF.Clustering methods have zero performance on FDI1 because normalized tampered profiles can appear identical to original profiles.
  • Combined performance: The type MIX AUC increased from 0.748 to 0.816 and MAP@20 increased from 0.693 to 0.831 with the combined method.These correspond to approximately 10% and 20% increases, respectively.
  • Stability: Arith and Geo combined methods improve both accuracy and stability over the original methods in type MIX experiments.Arith has minimum σAUC of 3.08%, while σMAP@20 is 9.16% for Arith and 9.13% for Geo.
  • Computational cost: The combining process takes less than 1 s, whereas Kraskov’s mutual-information estimator has the greatest time consumption among the evaluated methods.Timing covers one detection of all 11 730 load profiles on the stated desktop computer.

D. Sensitivity Analysis

Sensitivity analysis varies the numbers of thieves and consumers per area under type MIX, finding generally stable or improved combined-method performance across conditions.

  • Experimental setup: The analysis varies thieves per area from 1 to 7 and consumers per area from 30 to 98, while presenting results only for type MIX.The baseline setting is 5 thieves and approximately 39 users per area.
  • Varying thieves: As the number of thieves changes, MIC and PCC perform well with fewer thieves, MI is more robust in AUC, and MIC performs better in MAP@20.CFSFDP is consistently best among the three unsupervised clustering methods, while Arith maintains excellent AUC and MAP@20 performance.
  • Varying consumers: As consumers per area increase, most methods remain stable against the benchmark, while the combined methods improve over other methods in all conditions.MIC is the best correlation-based method overall, and CFSFDP is the best clustering-based method.
  • Stability sensitivity: The combined methods’ standard-deviation changes remain within a rather small range, although they do not always have the smallest standard deviation.σMAP@20 generally increases as consumers per area increases, while σAUC decreases slightly as the theft problem becomes more severe.

V. CONCLUSION

The paper proposes a combined MIC–CFSFDP framework for electricity-theft detection against AMI, using complementary correlation and outlier-detection signals.

  • Conclusion: MIC quantifies the association between tampered load profiles and area non-technical loss.The correlation-based method uses observer-meter data in the detection framework.
  • Conclusion: CFSFDP detects outliers for FDI types with little association to the original data through unsupervised analysis of smart-meter profiles.This complements MIC when tampered profiles have weak correlation with non-technical loss.
  • Conclusion: The methods ensemble suspicion ranks from MIC and CFSFDP to improve detection accuracy and stability.The numerical results report good and steady performance for all FDI types under various conditions.
Loading 2411.06649v1…