PaperlayerView exact revision on arXiv

Source-linked AI summary

Verifiable abstention makes AI leak diagnosis accountable in water distribution networks

Tianwei Mu, Yue Wang, Mingzhe Yuan, Manhong Huang, Wenhong Wang, Xuerui Yin, Qing Luo, Min Xiao, Hui Yang, Jun Li, Dan Xue

arXiv:2608.18836v1cs.AI

TL;DR

Leak-localization systems lack a verifiable basis for deciding when not to dispatch crews, limiting trust in autonomous excavation. This paper introduces executor–supervisor diagnosis with auditable abstention and reports selective, high-precision decisions, including 44% survey recovery at full district precision.

  • Problem

    Leak localizers are seldom trusted for autonomous dispatch because existing methods cannot prove when they should abstain amid ambiguous, non-leak anomalies.

  • Method

    An executor falsifies competing leak, demand, sensor, and valve hypotheses in a hydraulic digital twin, while an independent supervisor audits evidence against a code-verifiable contract.

  • Results

    44% survey recovery at 100% district precision was achieved on audited field events, alongside five excavation dispatches, three of which were correct.

  • Takeaways & Limitations

    Verifiable abstention makes “do not act” a provable, auditable outcome for leak diagnosis and supports a two-tier field response.

  • Takeaways & Limitations

    The study uses twin-simulated pressures and flows rather than measured utility telemetry, and its external L-Town test acts on only four events.

Abstract

from arXiv · show

Utilities lose a substantial share of treated water to leakage, yet rarely trust artificial-intelligence localizers to dispatch crews: guessing everywhere cannot justify excavation. The gap is accountability, not accuracy: no method proves when it should not act. Here we recast leak localization as decision-making under verifiable abstention. A physics-grounded executor agent falsifies hypotheses (leak, demand, sensor, valve) against a digital twin; an independent supervisor agent, with a large-language-model (LLM) auditor, checks evidence against a code-verifiable contract, then certifies a dispatch, requests evidence or abstains. Under field-grade noise, a 32% forced baseline becomes 96% decision precision on acted events. On an independently generated benchmark it acts on only 4 of 33 leaks, all correct. A 194-event register of audited real leak locations with twin-simulated pressures and flows yields five excavation dispatches, three correct, and 44% survey recovery at full district precision. Accountable abstention offers a defensible route to autonomous water-infrastructure operation.

Introduction

The introduction frames leak localization as accountable decision-making under verifiable abstention rather than forced prediction. It proposes an executor–supervisor architecture that uses physics-grounded falsification, code-verifiable contracts and independent auditing to justify action or abstention.

  • Motivation: Non-revenue water is dominated by physical leakage, while costly wrong excavations and 30 to 40% accuracy under field-grade noise undermine trust in autonomous dispatch.Treated water that is produced but never billed wastes urban production and billions of dollars each year.
  • Problem: The inverse problem is ill-posed because candidate locations vastly outnumber sensors, and demand surges, sensor drift and valve mis-operation can mimic leaks.Existing methods also lack a way to state and prove when they should not act.
  • Novelty: The proposed objective shifts from prediction to accountable decision-making under verifiable abstention, making “do not act” a provable, auditable outcome.Selective classification, learning-to-defer and conformal methods do not provide the required auditable justification for operators or regulators.
  • Method: A deterministic physics-grounded executor tests competing leak, demand, sensor and valve hypotheses in a hydraulic digital twin, while an independent supervisor audits numeric evidence against a code-verifiable contract.The supervisor never localizes and can accept with a signed certificate, reject with a request for more evidence, or abstain.
  • Implementation: A machine-checkable certificate provides numeric evidence for independent LLM auditing in a 32-package corruption stress test.The LLM auditor can tighten but never loosen the code-verifiable gate.

Results

The executor’s twin-based differential diagnosis and supervisor contract convert imperfect leak localization into selective, high-precision decisions. Across synthetic, benchmark, and real-event evaluations, the system improves precision by abstaining on unsupported or information-limited cases.

  • Synthetic evaluation: 31.7 ± 2.0% forced top-1 zone accuracy rises to 81.7 ± 5.0%, while the supervisor reaches 96.1 ± 3.3% decision precision.The supervisor acts on 40.5% of events, converting stronger twin-fit localization into selective decisions.
  • Differential diagnosis: 99 of 100 demand anomalies are correctly typed, sensor faults are never mis-classified as leaks, and 547 of 600 leaks are typed as leaks.Discrete confounders remain weakest: 33/75 valve and 39/75 sensor cases are correctly typed.
  • Risk–coverage control: 97.9% decision precision is achieved at 32.5% coverage, with 140/143 acted events correct and exact 95% CI 94.0–99.6%.The residual-derived confidence has expected calibration error 0.048 across four held-out test seeds and n = 440.
  • Independent benchmark: 15% forced top-1 accuracy on BattLeDIM L-Town becomes 100% decision precision on four acted leaks, with zero false dispatches and 88% abstention.The four acted events are strong abrupt leaks with onset |∆p| 0.40–1.15 m in the correct zone.
  • Independent benchmark: At matched four-event coverage, the contract reaches 100% precision versus 75% for a scalar existence threshold by rejecting poorly separated hypotheses.It rejects an existence 0.98, margin 0.02 event dispatched incorrectly by the scalar rule and admits a correct existence 0.52 event.

Discussion

The discussion frames the contribution as accountable decisions about when to act and how to prove them, rather than simply improving localization accuracy. It also identifies implementation failures and limits the broader generalization to a hypothesis.

  • Core contribution: The executor–supervisor architecture combines a physics twin, differential diagnosis of non-leak confounders, and a code-verifiable goal contract.The authors present this combination as new to the domain and as an answer to deployment under real noise.
  • Failure modes: Three failures exposed risks from over-flexible leak hypotheses, hypothesis-keying bugs, and inconsistent active-sensing evidence packages.The failures were detected through an Occam penalty, tool-traceable numbers, and checks involving posterior consistency and predicate G3.
  • Generalization: The architecture is proposed as domain-agnostic for networked physical-fault diagnosis when simulatable twins, structured signatures, confounders, and costly actuation coincide.Candidate domains include power systems, gas pipelines, contamination-source identification, structural-health monitoring, and water-quality prediction; water networks provide the demonstration.
  • Study boundaries: 100% precision on L-Town was achieved on only four events, while the study used no measured utility telemetry.L-Town used SCADA generated by competition organizers with an independent, deliberately perturbed copy of the network model.
  • Study boundaries: The confounder classes were simulated from literature priors, and the City D register contained real audited locations but modeled pressures and district inflows.The passage states that the City D register contributes real field events while its pressures and district inflows underpinning the analysis were not fully measured utility telemetry.

Methods · Problem formulation: from localization to accountable differential diagnosis · Hydraulic zoning and sensor placement

The method reformulates leak localization as differential diagnosis under sparse sensing, because anomaly type and location are uncertain and forced excavation guesses are costly. Hydraulic zoning and partition-aware sensor placement make the problem tractable and ensure each zone has a distinguishable pressure signature.

  • Problem formulation: from localization to accountable differential diagnosis: The network is modeled as G = (V, E), with N = |V| junction nodes and M = |E| pipe links governed by mass conservation and Hazen–Williams head loss.Leaks, demand surges, sensor faults, and mis-operated valves each induce pressure deviations relative to the no-anomaly baseline.
  • Problem formulation: from localization to accountable differential diagnosis: Anomalies are observed only through a sparse sensor set S ⊂V, where |S| = m ≪N.The method does not directly invert the hydraulic equation for leak location.
  • Problem formulation: from localization to accountable differential diagnosis: The inversion is ill-posed because candidate locations vastly outnumber m sensors, while demand surges, sensor faults, and valve mis-states can mimic leaks.A forced guess of 30 to 40% accuracy cannot justify costly excavation.
  • Hydraulic zoning and sensor placement: Nodes are partitioned into K ≪N hydraulically coherent zones by maximizing pressure-weighted, resolution-parameterized modularity.The zoning construction is reused unchanged from the benchmark configuration.
  • Hydraulic zoning and sensor placement: Weights average endpoint nodal pressures over T baseline time steps and are defined only on existing network links, including pipes, pumps, and valves.This restricts zones to hydraulically coupled nodes rather than merely similarly pressured, unconnected junctions.
  • Hydraulic zoning and sensor placement: Weights are rescaled linearly to [0, 1] with a floor of 10−3, placing networks with pressures differing by an order of magnitude on the same numerical footing.At each resolution, the objective is optimized by the Leiden algorithm.
  • Hydraulic zoning and sensor placement: Sensors are placed by a partition-aware greedy rule that maximizes minimum within-zone sensitivity, making every zone distinguishable and none unobservable.For EXA7, this yields K = 15 zones and m = 30 sensors, with two sensors per zone, computed offline and fixed throughout diagnosis.

Competing hypotheses and digital-twin falsification · Bayesian fusion and the executor agent

The method evaluates mutually exhaustive leak and non-leak hypotheses by falsifying them against calibrated digital-twin observations, then fuses evidence into an Occam-corrected posterior. A deterministic executor packages auditable residuals, posterior probabilities, fitted parameters, and falsification status for supervisory review.

  • Competing hypotheses and digital-twin falsification: The hypothesis space is mutually exhaustive, covering leaks, zone-wide demand anomalies, sensor faults, monitored-asset mis-states, and the null hypothesis of no actionable anomaly.Explicit non-leak explanations prevent diagnosis from assuming every event is a leak.
  • Competing hypotheses and digital-twin falsification: Each hypothesis is tested by injecting its corresponding perturbation into a calibrated WNTR/EPANET hydraulic digital twin and comparing predicted sensor responses with observations.Leaks, demand anomalies, and valve mis-states are represented as distinct physical perturbations.
  • Competing hypotheses and digital-twin falsification: A hypothesis that cannot reproduce observations accumulates residual and loses likelihood, with Mahalanobis distance ρh ≈1 indicating noise consistency and ρh ≫3 indicating physical incompatibility.This residual-based falsification distinguishes the approach from retrieval-based matching.
  • Competing hypotheses and digital-twin falsification: The null hypothesis is favored only when the observation is within noise, while sensor-fault hypotheses explain anomalous channels without requiring a hydraulic deviation elsewhere.These alternatives provide quantitative bases for “no actionable anomaly” and localized measurement faults.
  • Bayesian fusion and the executor agent: Bayesian fusion combines family priors and per-hypothesis likelihoods in log-space while applying an Occam penalty for differing hypothesis flexibility.The base rates are leak 0.50, demand 0.20, sensor 0.15, valve 0.10, and null 0.05.
  • Bayesian fusion and the executor agent: The deterministic, physics-grounded executor seeds competing hypotheses, invokes typed residual, digital-twin, Bayesian-update, and retrieval tools, and does not generate posterior inputs itself.Retrieval proposes candidate zones but is treated as evidence rather than a probability or final answer.
  • Bayesian fusion and the executor agent: When retrieval is weak under field noise, uncertain evidence leaves Πexist low, allowing the supervisor to abstain instead of forcing the retrieval top-one.This design prevents a weak retrieval result from becoming a mandatory localization.
  • Bayesian fusion and the executor agent: The executor’s structured evidence package records the posterior over H, fitted leak node and rate, residuals, Mahalanobis distances, candidate region, tool logs, and falsification statuses.The package is the sole interface to the supervisor and fixes the schema for auditable evidence.

The supervisor: goal contract, abstention, and active sensing · Independent large-language-model audit

The supervisor audits structured executor evidence against a goal contract, certifying only actionable leak hypotheses and otherwise requesting evidence or abstaining. An independent LLM auditor adds conservative rejection without overturning deterministic hard checks.

  • The supervisor: goal contract, abstention, and active sensing: The supervisor never localizes; it audits the executor’s evidence package against a conjunction of acceptance predicates in an explicit goal contract.The contract governs whether dispatch is authorized.
  • The supervisor: goal contract, abstention, and active sensing: G1 requires h(1) = hleak and Πexist ≥τe, preventing action while null, demand, or sensor explanations remain plausible.G2 limits |R| ≤Rmax, while G3 requires ∆≥δ.
  • The supervisor: goal contract, abstention, and active sensing: An acceptance certificate records the decision, contract version, thresholds, predicate results, accepted hypothesis, and region, cryptographically bound to the evidence package.The digest makes later tampering detectable and provides an auditable artifact for utilities or regulators.
  • The supervisor: goal contract, abstention, and active sensing: When predicates fail, the supervisor issues a structured request; if the contract remains unmet within the round budget, it abstains and defers to a human.The best-effort dossier preserves surviving hypotheses and reasons for predicate failure.
  • The supervisor: goal contract, abstention, and active sensing: For unresolved zone ambiguity, the executor ranks hidden nodes by the spread of surviving hypotheses’ twin-predicted responses and reveals the three highest-scoring nodes.This digital-twin discriminability criterion approximates maximal expected information gain.
  • Independent large-language-model audit: Because the executor is deterministic, the LLM receives only a structured numeric summary and serves as an independent supervisor auditor without access to internal computation or prose.The evidence summary includes the top hypothesis, residual, existence mass, margin, alternatives, residuals, and falsification status.
  • Independent large-language-model audit: The frontier open model returns strict-JSON accept-or-reject verdicts, may only add rejection, and cannot overturn a failed deterministic hard check.A conservative parse-failure default is specified in the reproduced audit materials.
  • Independent large-language-model audit: The audit layer is stress-tested with corrupted packages and genuine packages, measuring catch rate, false-alarm rate, and agreement across repeated temperature-zero audits.Corruption cases include unsupported assertions pairing probable leaks with incompatible residuals and fabricated exclusions of fitting alternatives.

Scenario generation, evaluation protocol, and baselines · Real-network transfer: L-Town, the City D field leg and standard-protocol legs

The study evaluates accountable leak localization through seeded, physics-based anomaly scenarios, action-level metrics, and forced or selective baselines. It then tests protocol transfer on independent networks, the L-Town benchmark, and a semisynthetic City D field leg while reporting design tradeoffs and explicit limitations.

  • Scenario generation, evaluation protocol, and baselines: Four anomaly classes are generated with one seeded WNTR/EPANET pipeline for bit-reproducible evaluation, including confusable demand anomalies and corrupted sensor channels.Leaks span 5 to 50 L s−1, while sensor faults include bias, stuck, or excess-noise corruption across one to three channels.
  • Scenario generation, evaluation protocol, and baselines: Action-level evaluation reports risk–coverage and precision–coverage curves, false-dispatch rate, four-class confusion, and residual-confidence calibration.Coverage is the fraction of acted events, and false-dispatch rate is defined as the complement of decision precision.
  • Scenario generation, evaluation protocol, and baselines: Forced baselines comprise cosine k-nearest-neighbour, random forest, multilayer perceptron, and graph convolutional network localizers, alongside calibrated-threshold and split-conformal selective predictors.The selective comparators are evaluated with calibration error, and the split-conformal predictor provides distribution-free risk control.
  • Real-network transfer: L-Town, the City D field leg and standard-protocol legs: KY4 provides an independent in-silico transfer test using an unmodified public model partitioned into 25 Leiden zones with 40 sensors.The sensors use degree-based topological observability, with one highest-degree junction selected per zone before remaining sensors are filled by global node degree.
  • Real-network transfer: L-Town, the City D field leg and standard-protocol legs: L-Town transfer diagnoses all 33 provided ground-truth leaks from SCADA series generated by a perturbed copy of the network model rather than field telemetry.The 782-node network is divided into 25 zones and uses 33 pressure gauges; the leak library is built only from the nominal model.
  • Real-network transfer: L-Town, the City D field leg and standard-protocol legs: City H and City D are additionally evaluated under the EXA7 standard in-silico protocol with σ = 0.05 m and five seeds, while the City D protocol includes explicit pressure and survey-observation calculations.The survey model varies nightly inflow uncertainty across 0.05/0.10/0.20 L s−1 and uses one seeded noise realization per event.
  • Real-network transfer: L-Town, the City D field leg and standard-protocol legs: A discriminability-optimized sensor placement increased the library classification proxy from 50.0% to 64.3% but left decision precision statistically unchanged at 83.3 ± 8.6% versus 83.1 ± 11.0%.Because library detection coverage at the 0.45 m floor fell from 34.7% to 30.0%, the detection-coverage placement was retained.
  • Real-network transfer: L-Town, the City D field leg and standard-protocol legs: The City D field leg uses an unmodified 541-junction utility model and a 2025 repair register, but its event signatures and flows are twin-simulated rather than based on real SCADA.Explicit limitations include 2016-vintage topology and equipment, a uniform demand update ratio of 1.168459, unknown coordinate datum, and work-order windows serving as reporting proxies.

Statistics and reproducibility

The diagnostic system is implemented with WNTR and EPANET, and its twin-grounded, code-checked decision path is replayable and verifiable. Statistical reporting uses predefined uncertainty and testing procedures, while a predecessor’s inflated City H result is explicitly excluded.

  • Implementation: The system uses Python with WNTR 1.4 and EPANET 2.2, memoized hydraulic solves, and precomputed per-zone leak-response libraries.A complete diagnostic episode uses cached lookups and targeted digital-twin evaluations rather than online optimization.
  • Reproducibility: The complete path from observation to authorised dispatch or abstention is replayable and verifiable because executor values and supervisor predicates are deterministic.Executor values originate from the digital twin and Eq. (3), while supervisor checks are implemented as hard code predicates rather than learned judgement.
  • Statistical analysis: All statistical tests are two-sided, uncertainty is reported as mean ± standard deviation over five independent random seeds, and small-sample proportions use exact 95% Clopper–Pearson intervals.No events were excluded; scenario generation is governed by a fixed seed registry, and blinding was not applicable because no human raters were involved.
  • Data provenance: 67% top-one localization on City H was identified as inflated from a real run of about 41%, and none of those predecessor numbers, code paths, or mappings enters this study.The predecessor claim is explicitly separated from the present study’s committed pipeline.

Data availability

The study is self-contained through vendored network, input, generated-corpus, and results files, while the KY4 network comes unmodified from a public Kentucky research database benchmark via WNTR.

  • Data availability: EXA7, its partition, sensor-placement, and scenario-fingerprint inputs are vendored under data/exa7/.This makes the study self-contained.
  • Data availability: Generated scenario corpora and all results JSON are committed as a provenance snapshot used to read every figure value.
  • Data availability: KY4 is a public Kentucky research database benchmark vendored unmodified from the pip-installed WNTR package under data/ky4/.

Extended Data

The extended-data schematics define a physics-grounded executor that tests exhaustive anomaly hypotheses in a hydraulic digital twin, then show how a supervisor audits the evidence against deterministic contract checks before authorizing dispatch. Passing checks produces an accountable certificate recording predicate values and outcomes.

  • Diagnostic pipeline: The executor confronts sparse, field-noisy sensor observations with mutually exhaustive hypotheses, including leaks, demand anomalies, sensor faults, valve mis-states, and no anomaly.Each hypothesis is realized in the hydraulic digital twin and must reproduce the observation.
  • Accountable action: The supervisor audits the executor’s numeric evidence package against the goal contract using deterministic checks for existence, region, margin, alternatives, residual, and safety.The deployment-reserved G5 field is not evaluated in the schematic.
  • Accountable action: Passing all evaluated checks authorizes dispatch and produces a certificate recording every predicate’s value and pass-or-fail outcome, bound to the evidence.The certificate makes the authorization traceable to the audited evidence package.

Supplementary Information · Supplementary Methods

The supplementary methods define a physics-grounded digital twin, structured competing hypotheses, and posterior summaries for leak diagnosis. They also specify active sensing and fail-safe, independently audited language-model decisions using explicit evidence thresholds.

  • S1. Governing hydraulics: The digital twin enforces steady-state mass conservation and the Hazen–Williams head-loss relationship, with WNTR/EPANET solving the governing equations.The model represents pipe flow, nominal demand, anomalous demand, hydraulic head, pipe resistance, and exponent n = 1.852; induced sensor deviations plus field noise form main-text Eq. (1).
  • S2. Competing-hypothesis space: Diagnosis uses a mutually exhaustive hypothesis space containing one dominant anomaly per event: zone leak, zone demand anomaly, sensor fault, valve mis-state, or no actionable anomaly.The null hypothesis h0 represents the absence of an actionable anomaly.
  • S3. Leak fit within a zone: Each zone-level leak hypothesis is fitted by selecting the best junction–rate pair across all zone junctions and the grid Q = {2, 5, 10, 20, 35, 50} L s−1.The resulting residual and Gaussian log-likelihood are defined in main-text Eq. (2).
  • S4. Posterior summary quantities: The supervisor summarizes the posterior using leak-existence mass Πexist, the top-hypothesis margin ∆, and posterior entropy.The margin compares the top hypothesis with the best alternative from any family.
  • S5. Active-sensing discriminability: When zone ambiguity remains, the executor ranks hidden nodes by the spread of surviving hypotheses’ twin-predicted responses and reveals three nodes per round.The first-ranked criterion approximates measuring maximal expected information gain.
  • S6. Language-model auditor and planner: prompts and strict-JSON templates: The independent auditor receives only structured numeric evidence and rejects dispatch when the claimed leak is unsupported, poorly separated, poorly fitting, or inconsistent.Residual Mahalanobis-per-dof near 1.0 indicates reproduction of observations, whereas >>3 indicates poor fit.
  • S6. Language-model auditor and planner: prompts and strict-JSON templates: The auditor rejects if leak_existence_probability < 0.7, margin_over_next_best < 0.15, fit_residual_mahalanobis > 3, or a supposedly falsified alternative has residual_mahalanobis < 1.5.It also rejects whenever the top hypothesis is not a leak, and otherwise returns reject=false.
  • S6. Language-model auditor and planner: prompts and strict-JSON templates: Missing or unparsable auditor output defaults conservatively to rejection with reason auditor_parse_failure_default_safe.The required response format is strict JSON containing reject and a short reason; transcripts are persisted for replay, and the planner selects non-leak explanations to test alongside leak hypotheses.

Supplementary Figures · Supplementary Tables

The supplementary results show that the system improves selective leak diagnosis through abstention, calibrated contracts, and differential cause typing, while exposing severe field-register detectability limits. Transfer benchmarks and sensitivity analyses support high precision on acted events, but pressure information is insufficient for most City D register leaks.

  • Supplementary Tables: 81.7 ± 5.0% twin-fit accuracy exceeds the strongest trained baseline while uniquely abstaining and typing non-leak causes.The executor–supervisor twin-fit is evaluated on EXA7 under the supplementary comparison.
  • Supplementary Tables: Distribution-free risk control holds, with realized risk below the target at both tested levels.The selective-prediction analysis uses split-conformal calibration on EXA7.
  • Supplementary Tables: 100% decision precision is achieved at 12% coverage on the 33-leak BattLeDIM L-Town benchmark, with 4 acted events.Forced top-1 zone accuracy is 15% on the same benchmark.
  • Supplementary Tables: At matched coverage of 4 acted events, the multi-predicate contract reaches 100% versus 75% for a scalar existence threshold.The contract vetoes leak candidate p523.
  • Supplementary Tables: Across 28 one-at-a-time settings, pooled decision precision spans 89.9–99.1%, compared with 93.6% nominal precision.Thresholds were varied around the operating values while other contract thresholds were held nominal.
  • Supplementary Tables: On the City D register, the transfer preset dispatches excavation on 5 of 194 events at 60% acted precision, while the stricter audit preset dispatches 0 of 194.The transfer preset raises 2% false alarms on 50 no-leak controls; the audit preset admits 0 of 50 controls, and forced top-1 accuracy is 11.9%.
  • Supplementary Tables: The City D field register remains below the network’s pressure-detectability threshold: signatures clear the noise floor only near 10–20 L s−1, while all register events are below 4 L s−1.This limits pressure-based excavation actionability on the stiff, well-pressurized network.
  • Supplementary Tables: Excavation-tier coverage stays at 2.6% under the transfer preset and zero under the audit preset, making the survey tier the principal recovery path.This result concerns the City D field-register analysis.
Loading 2608.18836v1…