PaperlayerView exact revision on arXiv

Source-linked AI summary

Distinguishing Process Change, Instrument Fault and Data-Path Corruption in Streaming Telemetry: A Reference Implementation and Benchmark

Oleksandr Klymenko

arXiv:2610.18904v2eess.SPcs.LG

TL;DR

Monitoring systems must distinguish process change, instrument failure, and data-path corruption, which can look alike in a single reading but call for different responses. The paper proves some one-channel limits and combines online detectors with a deterministic attribution rule. On synthetic streams, correct attribution is 91.2% from one channel and 99.0% from two; recorded signals with injected faults reach 84.5%.

  • Problem

    Monitoring streams need to distinguish process change, instrument failure, and data-path corruption, but these causes can look alike in a single reading and prior work often treats them separately.

    Source: §I · p. 1

  • Method

    The paper proves one-channel identifiability limits and combines classical online detectors in a deterministic attribution rule that returns a cause or an undetermined verdict.

    Source: §I · p. 1, B. What one channel cannot separate · p. 2

  • Results

    91.2% of detected events are attributed correctly from one channel and 99.0% from two on synthetic streams; recorded signals with injected faults reach 84.5% correct attribution.

    Source: B. Semi-synthetic streams · p. 5, §IX · p. 7

  • Takeaways & Limitations

    The authors conclude that the method detects quickly on real data but alarms too often for unattended use.

    Source: §IX · p. 7

    One channel cannot separate sensor drift from a process trend or a bias step below the stated identifiability bound.

    Source: §VIII · p. 5

Abstract

from arXiv · show

A reading departs from what a monitoring system expects for one of three reasons: the physical process changed, the instrument is failing, or the value was corrupted between the instrument and the software. The three call for different responses and look alike in a single sample. We state the three-cause problem, prove which pairs of causes one channel cannot separate, and propose a bank of online detectors - a Kalman tracker with CUSUM and EWMA charts on its innovation, a Hampel median with generalized-ESD confirmation, a robust noise statistic with ADWIN, exact-value checks and an optional two-channel complementary filter - combined by a deterministic, inspectable attribution rule. The detectors were merged into a public algorithm library. On 2,942 injected events of thirteen fault types the method detects 96.3% (95% interval 95.7-97.0) at 0.22 false alarms per hour and attributes 91.2% (90.6-91.8) of detected events correctly from one channel and 99.0% (98.6-99.4) with a second, rate channel; from one channel only 2 of 207 detected sensor drifts are attributed to the instrument, as the problem statement predicts. Injected into recorded industrial signals, the same catalogue is detected at 80.8% and attributed correctly at 84.5%, at 3.6 false alarms per hour. On the Skoltech Anomaly Benchmark the method detects 32 of 33 experiments with a median delay of 15 s, at 46 false alarms per hour against 11 for a rolling z-score.

I. Introduction

A monitoring system must distinguish real process change, instrument failure, and data-path corruption because each calls for a different response. The paper frames these causes together at the software receiving the stream, where prior work often addresses them separately or detects departures without attribution.

I. Introduction

  • Confusing process change, instrument failure, and data-path corruption can prompt the wrong operational response, maintenance action, or data-path repair.The software receiving the stream may not be able to audit the route a reading traveled.Source: §I · p. 1
  • The paper treats the three causes as one streaming problem under one-pass, bounded-memory, no-retraining, and bounded-decision-delay constraints.Prior work addresses individual causes or detects departures without naming their cause.Source: §I · p. 1

II. Related Work

  • Related work detects distribution changes, model residuals, sensor-fault signatures, or point anomalies, but attribution of a detected departure to its cause is outside some anomaly-detection work's scope.The paper uses a random-walk state with a physical rate bound and keeps outlier detection separate so rejected samples remain visible for attribution.Source: §II · pp. 1–2

A. Observation model

  • The observation model distinguishes departures in the process quantity, instrument bias or noise, and the data-path operator.It defines the received value as what the software observes and identifies the controller-to-cloud chain as part of the path in industrial architectures.Source: A. Observation model · p. 2

B. What one channel cannot separate

The paper proves that some process changes and instrument faults are observationally indistinguishable from one channel. A physical rate bound or a second rate channel can supply information that breaks some of these ambiguities.

B. What one channel cannot separate

  • A process level shift and an instrument bias step induce the same distribution of received readings, so no one-channel test separates them better than chance.The equivalence is stated for an identity measurement function and a healthy data path.Source: B. What one channel cannot separate · p. 2
  • A bias step larger than the process could produce under the rate bound in the observed time is identifiable; a smaller step is not.If the process changes at most ρ per sample, a step completing in m samples can be a process change only when |a| ≤ mρ.Source: B. What one channel cannot separate · p. 2
  • A process trend and instrument drift are indistinguishable on one channel when the drifted process remains admissible, but a rate channel makes them separable as time grows.The rate channel measures process change rather than bias.Source: B. What one channel cannot separate · p. 2
  • Noise growth, stuck-at values, saturation, point anomalies, exact repeats, and out-of-range values change the received-reading law in ways an admissible process path cannot reproduce.The paper therefore expects one channel to attribute these signatures and measures that performance later.Source: B. What one channel cannot separate · p. 2

C. Constraints

The method is designed for streaming constraints and calibrates thresholds from each channel's own data. Calibration uses an initial noise estimate followed by silent detector operation to set thresholds.

C. Constraints

  • The system is constrained to one pass, bounded state independent of stream length, bounded decision delay, no labeled training data, and channel-specific threshold calibration.Source: C. Constraints · p. 2

A. Calibration

  • During phase A, each channel estimates its noise scale from first differences and keeps their median with P2.For Gaussian noise, the median absolute first difference is scaled by 0.6745σ; quantized channels use the larger of that estimate and the sample standard deviation.Source: A. Calibration · p. 2
  • During phase B, detectors run silently while P2 records the 99.9th percentiles of two statistics, and thresholds are set to 1.25 times those percentiles subject to configured floors.Source: A. Calibration · p. 2

B. Detectors

The detector bank combines exact-value guards, point-anomaly screening, level tracking, and noise-growth monitoring. Its statistics provide distinct evidence for different anomaly signatures.

B. Detectors

  • Exact-value checks flag six identical consecutive values, four repeats of earlier values within 256 samples, saturation patterns, and missing, out-of-range, or declared boundary values.The range guard acts before any filter.Source: B. Detectors · p. 2
  • A Hampel median over the last 11 samples flags point anomalies, and a generalized ESD test confirms flags before they count as evidence.Flagged samples are replaced downstream; the scale uses the calibrated σ estimate rather than the window's MAD.Source: B. Detectors · p. 2
  • A scalar random-walk Kalman filter tracks the cleaned value, and CUSUM and EWMA charts operate on its standardized innovation.The charts reset on alarm and after each verdict.Source: B. Detectors · pp. 2–3
  • The noise statistic compares current noise with calibration, while ADWIN monitors the mean of clipped normalized first differences.The design uses a median of 40 differences so smooth process movement and a single outlier have limited effect on the statistic.Source: B. Detectors · p. 3

C. Second channel

With a second channel measuring the same quantity’s rate, a complementary filter combines the integrated rate with the level. The resulting δ_t stays near zero during process movement and follows bias steps or drifts.

C. Second channel

  • The complementary filter fuses the integrated rate from a second channel with the level measurement.Source: C. Second channel · p. 3
  • By Proposition 2, δ_t stays near zero when the process moves and follows the bias when it steps or drifts.Source: C. Second channel · p. 3

D. Attribution rule

The attribution rule opens a 40-sample decision window after the first detector fires, then evaluates three features of the cleaned series. It uses matching evidence to assign a verdict and latches evidence after verdicts to avoid repeated decisions for persistent conditions.

D. Attribution rule

  • The first detector to fire opens a 40-sample decision window, after which the rule reads three cleaned-series features.Source: D. Attribution rule · p. 3
  • The features are net level change, least-squares slope over 60 samples, and the largest change between 5-sample medians five samples apart.Source: D. Attribution rule · p. 3
  • A level is considered moving when |γ| > 0.03; the procedure also evaluates net change, least-squares slope over the last 60 samples, and changes between five-sample medians five samples apart.Source: D. Attribution rule · p. 3
  • After a verdict, matching evidence is latched for a 40-sample hold-off, while a nearby change alarm can continue a dismissed window and its baseline.Each verdict carries the evidence behind the matched rows so an operator can inspect why the cause was assigned.Source: D. Attribution rule · p. 3

V. Reference Implementation

The reference implementation uses reviewed Java detectors merged into a public algorithm library, with a frozen benchmark configuration and bounded per-channel state. The evaluated bank is documented separately from components in the same series that were not used.

V. Reference Implementation

  • The Java detectors were reviewed and merged into TheAlgorithms/Java, and the benchmark vendors them unmodified from a pinned commit.The configuration was frozen before test runs, with development-data changes listed in the repository.Source: §V · p. 3
  • Per-channel state is bounded independently of stream length, using configuration-sized arrays, bounded extremum aggregators, and logarithmically growing ADWIN buckets.The reported ADWIN bucket count is at most 31 over 10^6 samples.Source: §V · p. 3
  • One update takes a median of 5.8 μs on one core, measured over 10^6 samples after warm-up rather than with JMH.Source: §V · pp. 3–4
  • Table III identifies the library components used in the evaluated bank and names extended and unscented Kalman filters and dynamic time warping as unused components from the same series.Source: Table III · p. 4

VI. Benchmark

The benchmark evaluates injected faults in synthetic and recorded industrial streams, as well as detection on SKAB, using event matching, delay, attribution, and false-alarm measures. The supplied passages describe the evaluation design and baselines, but not its outcome values.

VI. Benchmark

  • Synthetic tests use tank and heated-vessel process models with Gaussian noise, process wander, and a noisy second channel measuring true rate.Runs contain 20,000 samples, with one sample representing one second.Source: §VI · p. 4
  • The semi-synthetic tests inject the catalogue into eight anomaly-free SKAB channels, yielding 2,040 events without a rate channel.The file was also used during development to fix two calibration rules, but no attribution threshold was tuned on it.Source: §VI · p. 4
  • SKAB evaluation uses eight water-rig sensors at 1 Hz and excludes one of 34 labeled files because its anomaly begins before calibration ends.The benchmark labels anomalies rather than causes, so attribution is reported descriptively.Source: §VI · p. 4
  • An event is matched by the first decision triggered from onset through 120 samples after its end, while false alarms count decisions outside event windows per monitored hour.Attribution precision is over matched decisions and recall is over all events; intervals use 2,000 bootstrap resamples of 100 test runs.Source: §VI · p. 4
  • The comparison baselines are a 300-sample rolling z-score with |z| > 4 and a restarted raw-value CUSUM; neither attributes causes.Source: §VI · p. 4
  • The synthetic test catalogue uses shuffled fault rounds separated by at least 420 samples, with test seeds 2000–2049 per model run once after configuration freeze.Each process model draws its own random stream and fault schedule for a given seed.Source: Table IV · p. 5

VII. Results

Results span synthetic streams, recorded industrial signals, and SKAB, with lower detection and attribution on real noise and a speed–false-alarm tradeoff on SKAB.

A. Synthetic streams

  • Synthetic-stream median detection delay was 3 s (P90 33 s): 0 s for data-path, 4 s for instrument, and 20 s for process events.The verdict follows after the 40-sample window; process events develop at a bounded rate.Source: A. Synthetic streams · p. 5
  • With one channel, only 2 of 207 detected drifts were attributed to the instrument; with the rate channel, 198 of 213 detected drifts (93.0%) were.The one-channel results were 176 process, 28 undetermined, and 1 data-path verdict.Source: A. Synthetic streams · p. 5
  • Of 108 one-channel misses, 47 were absorbed by an earlier trigger’s open decision window and 61 were silent, with no point-like or sequence-level fault silently missed.Silent misses were slow changes and drifts whose rates kept innovation within chart limits, plus six level shifts.Source: A. Synthetic streams · p. 5

B. Semi-synthetic streams

  • 80.8% of injected events were detected and 84.5% attributed correctly on real noise, versus 96.3% and 91.2% on synthetic streams; false alarms rose from 0.22 to 3.6 per hour.Recorded background sometimes moved faster than the declared plausible rate, costing process attribution; data-path attribution precision remained 98.6%.Source: B. Semi-synthetic streams · p. 5
  • Of 912 false alarms across semi-synthetic channels, 300 were attributed to the process and 490 to the instrument.False alarms concentrated on the accelerometer 1 and current channels, at about 10 per hour each.Source: B. Semi-synthetic streams · p. 5

C. Baselines and SKAB

  • 32 of 33 SKAB experiments were detected, with a 15 s median delay against 35 s for the z-score, but four times its false alarms.SKAB carries no cause labels, so the reported cause verdicts are not scored.Source: C. Baselines and SKAB · p. 5

D. Bias step versus process shift on one channel

On one channel, bias steps and process shifts become distinguishable only above an identifiability boundary; a second rate channel improves attribution. Recorded-signal results and validity constraints qualify the synthetic benchmarks.

D. Bias step versus process shift on one channel

  • From 3σ, process shifts are attributed to the process at 91.8–99.4%; below the crossover, a bias step cannot be distinguished from a rate-limited process move.At 2σ, tank bias steps are attributed to the instrument at 24%, versus 12% for the vessel; attribution rises to 97.1–100% from 5σ.Source: D. Bias step versus process shift on… · p. 5
  • With Δ = 20 or 80, correct attribution is 88.7% or 91.0% versus 91.2% at Δ = 40; without the plausible-rate test, bias-step attribution falls to 0%.Without deferral, level-shift attribution falls to 0% and 28.5% of detected events become undetermined.Source: D. Bias step versus process shift on… · p. 5

VIII. Discussion and Limitations

  • One channel detects 96.3% [95.7, 97.0] of events and correctly attributes 91.2% [90.6, 91.8] of detected events; two channels raise correct attribution to 99.0% [98.6, 99.4].One-channel false alarms are 0.216 per hour [0.160, 0.278].Source: Table V · p. 7
  • On recorded SKAB channels, pooled F1 is 65.1% for process, 71.2% for instrument, and 87.8% for data-path; quantized channels and fast backgrounds impair performance.Most false alarms are labeled process movements but remain alarms; deployment would require a minimum event duration per channel.Source: §VIII · pp. 5, 7, Table VI · p. 7
  • A learned forecaster could sharpen the innovation and its charts, at the cost of training and inspectability.The authors’ rule uses a simpler expectation model.Source: §VIII · p. 7
  • The injected assumptions and author-designed benchmark may bias results; separating test seeds limits but does not remove that bias.The semi-synthetic base file was seen during development, and SKAB supports detection but not attribution.Source: §VIII · p. 7

IX. Conclusion

The conclusion reports accurate attribution on synthetic and recorded signals but notes that real-data alarms remain too frequent for unattended use. It identifies per-channel duration requirements, multichannel physical models, and labeled industrial data as next steps.

IX. Conclusion

  • 91.2% of detected events were attributed correctly from one channel and 99.0% from two, with 0.22 false alarms per hour on synthetic streams and 84.5% correct attribution on recorded signals with injected faults.On real data, the method detects quickly but alarms too often for unattended use.Source: §IX · p. 7
  • The proposed next steps are per-channel duration requirements, multichannel plants with a physical model, and labeled industrial datasets as they become available.The conclusion also notes that the method fails where the one-channel propositions say causes cannot be separated.Source: §IX · p. 7
  • The z-score detects 1 of 227 replayed segments and almost no dropout, while restarted CUSUM detects every process change but raises 9.6 false alarms per hour.The table also reports 2.00 h of unlabeled rows in labeled SKAB files and 2.50 h in the anomaly-free file.Source: Table VII · p. 8

Code and Data Availability

  • The implementation, benchmark harness, frozen configuration, and result files are publicly available, with one command reproducing every paper number, table, and figure.SKAB is not redistributed; a repository script downloads it and verifies its checksums.Source: Code and Data Availability · p. 8
Loading 2610.18904v2…