PaperlayerView exact revision on arXiv

Source-linked AI summary

COMET: A coordinated multi-layer runtime monitoring framework for human-supervised AI-enabled safety systems

Niclas Flehmig, Mary Ann Lundteigen, Shen Yin

arXiv:2610.29816v1cs.AI

TL;DR

Safety-oriented AI monitoring needs to detect unreliable behavior while giving human supervisors information for fault diagnosis. COMET coordinates monitors grounded in different fault modes and communicates their diagnoses. In the evaluated leak-detection proxy, it reduced trusted-wrong-answer rates to zero on covariate-shifted no-leak data and three of four sensor faults, while human-operator evidence remains absent.

  • Problem

    Safety-oriented runtime monitoring needs to detect unreliable AI behavior and support supervisors who remain responsible for control adaptation.

    Source: §1.1 · p. 1

  • Method

    COMET coordinates monitors anchored in different fault modes and communicates their diagnoses through a human-oriented taxonomy and display.

    Source: §6 · p. 22

  • Results

    COMET reduced the trusted-wrong-answer rate to zero on covariate-shifted no-leak data and three of four injected sensor faults.

    Source: §6 · p. 22

  • Takeaways & Limitations

    Coordinating monitors with disjoint fault modes supports fault diagnosis by isolating and sometimes identifying faults, informing runtime human decision-making.

    Source: §6 · p. 22

    COMET’s greater informativeness for operators is supported by architecture, ablation, and theory, but has not been validated with human operators.

    Source: §5.9 · p. 22

Abstract

from arXiv · show

Incorporating AI systems into safety systems requires monitoring to avoid silent faults and ensure safe and reliable operation. Single monitors often do not provide actionable information for human supervisors, who act as the system's adaptive controller. We present COMET, a coordinated multi-layer runtime monitoring framework for human-supervised AI-enabled safety systems in which each monitor is anchored in a different fault mode. Monitor outputs are combined in a coordination layer and mapped by a communication layer to a traffic-light display. In this work, COMET consists of three monitoring layers: signal integrity, physics consistency, and the AI system's competence envelope. The framework is evaluated on an open dataset of compressed-air leaks, where an AI system detects leaks from acoustic emissions, with held-out conditions simulating covariate shift, semantic shift, and sensor faults, events expected during runtime but unknown in their exact appearance. All monitors are frozen after development and tested zero-shot, and the framework is compared against no-monitoring and single-monitor setups. The bare AI system fails silently and confidently under all three shift types, whereas COMET reduces the trusted-wrong-answer rate to zero on covariate-shifted no-leak data and on three of four sensor faults, at a nominal false-alarm rate of $\approx 1\%$. Single monitors come close in detection but cannot attribute the fault mode. COMET converts their anonymous detection flags into diagnoses. Residual silent misses under semantic shift define the boundary of passive monitoring. Runtime monitoring frameworks for AI-enabled safety systems must therefore combine quantitative fault detection and isolation with qualitative, human-oriented feedback.

1. Introduction

The paper identifies a gap between isolated AI monitors and safety-oriented monitoring that diagnoses faults and communicates actionable information to human supervisors. COMET combines monitors anchored in different fault modes with coordination and communication layers.

1.1. Motivation

  • The paper asks how runtime monitoring can detect potentially unreliable behavior, isolate faults, and support the human supervisor in the loop.The stated aim is to enhance safe and reliable operation.Source: §1.1 · p. 2
  • Safety-related monitoring needs a concrete runtime framework that supports a human supervisor who remains responsible for control adaptation.The paper contrasts this need with monitoring focused on operational performance or monitoring requirements that do not specify implementation.Source: §1.1 · p. 1, §1.2 · p. 2

1.2. Related work

  • Existing monitor combinations have emphasized detection rather than coordinating local diagnoses toward fault isolation or explicitly considering the human supervisor.The paper positions COMET as addressing both gaps.Source: §1.2 · p. 2

1.3. Contributions

  • COMET assigns monitors to different fault modes, fuses their outputs into a fixed taxonomy, and maps states to a traffic-light display for supervisors.The evaluation uses held-out covariate shifts, semantic shifts, and sensor faults, with monitors frozen for zero-shot testing against bare-AI and single-monitor setups.Source: §1.3 · p. 3

2. Methods

The methods pair a compressed-air leak-detection application with three monitors grounded in signal integrity, physics consistency, and learned competence. Their local diagnoses are coordinated for communication to a human supervisor.

2.1.1. Dataset

  • The evaluation uses an open compressed-air leak dataset as a proxy for safety-critical leak detection, with recordings spanning microphones, gains, backgrounds, leak types, and severity.Small leaks at knob settings 3.5–5.0 were excluded; the dataset description also identifies tube leaks and hydraulic backgrounds as unused training or calibration conditions.Source: §2.1 · p. 3, §2.1.1 · pp. 3–4, Figure 1 · p. 5

2.1.2. Primary AI system

  • A CNN classifies spectrogram patches as leak or no-leak; recordings are converted to STFT patches, noise-augmented in training, and normalized per frequency bin.The model uses convolution blocks and a dense output network; each 30-second recording yields 117 non-overlapping patches.Source: §2.1.2 · p. 4, §2.1.3 · p. 4

2.1.4. Training process

  • Training uses session-level splits, cross-entropy with AdamW, early stopping, and extracted activation taps; the procedure yields 12 models across three folds and four microphones.The training setup avoids session leakage, and the selected monitoring fold trains on sessions 1 and 2 and validates on session 3; a single fold took minutes on the stated GPU workstation.Source: §2.1.4 · p. 4, §2.1.5 · p. 4

2.2. Coordinated multi-layer runtime monitoring framework

  • The framework draws on fault-tolerant diagnosis and treats the human supervisor as responsible for adapting control based on diagnosis.Its coordination layer fuses decentralized diagnoses, while its communication layer maps taxonomy states to traffic-light feedback and recommended actions.Source: §2.2 · pp. 4–5
  • COMET combines local diagnosers anchored in distinct fault modes and coordinates their diagnoses into a taxonomy state communicated to the human supervisor.The supervisor retains control-adaptation responsibility; the framework’s coordination extends diagnosis toward isolation, not full fault identification.Source: §2.2 · p. 5, Figure 2 · p. 6

2.2.1. L1 - Signal-integrity monitor

  • L1 checks each raw waveform window with five predefined signal-integrity rules, including long zero runs, clipping, flat-line output, DC offset, and implausible RMS level.The rules use 12,288-sample windows; the zero-run rule fires when the longest run exceeds 20% of a window.Source: §2.2.1 · pp. 5–7

2.2.2. L2 - Physics-consistency monitor

  • L2 checks whether an AI-detected leak is physically consistent with a compressed-air leak’s broadband, high-frequency-weighted spectral pattern.A fixed spectral template yields consistent, quiet, inconsistent, or abstain outputs; the paper notes that acoustic background noise can mask leak signals.Source: §2.2.2 · p. 7

Offline construction and calibration

L2’s spectral template and thresholds are constructed from nominal data, calibrated before deployment, and frozen before held-out evaluation.

Offline construction and calibration

  • L2 estimates background energy with a per-condition median and fits a gain-normalized leak-spectrum template from nominal vent-leak files for each microphone and gain configuration.The runtime and offline procedures use the same spectral representation; the evaluation excludes the scored file from its own background estimate.Source: Offline construction and calibration · pp. 7–8
  • The abstention margin is calibrated at the 99th percentile of nominal no-leak gate statistics, while the absent and present verdict thresholds use nominal no-leak and leak score percentiles.The template, margin, and verdict thresholds are frozen after calibration.Source: Offline construction and calibration · p. 8
  • Only 1% of nominal no-leak files score below the absent threshold, and all three nominal-data checks of the template passed before its components were frozen.The checks covered monotonic severity scaling, spectral-shape similarity of at least 0.95 across severity, and leak/no-leak separability within each configuration.Source: Offline construction and calibration · p. 8

Runtime

At runtime, L2 compares a background-adjusted spectrum with a frozen leak template, while L3 checks whether inputs fall within the AI system’s learned competence envelope.

Runtime

  • L2 first abstains when excess energy in expected leak bands is insufficient, then normalizes the residual spectrum and compares its shape with the template by cosine distance.A low distance means the spectrum resembles a vent leak; a high distance means no known leak severity looks similar.Source: Runtime · p. 8
  • The frozen thresholds classify low-distance observations as consistent and high-distance observations as inconsistent, distinguishing missed leak physics from unsupported AI detections.The absent threshold asks whether leak physics is present despite an AI miss; the present threshold asks whether physics contradicts the AI detection.Source: Runtime · pp. 8–9

2.2.3. L3 - Deep k-NN feature-space monitor

  • L3 uses k-nearest neighbors on the CNN’s 64-dimensional embeddings to flag inputs outside the learned competence envelope as potentially unreliable.Unlike L2’s check against leak physics, L3 checks whether an input resembles data seen during training.Source: §2.2.3 · p. 9

Offline construction and calibration

The competence-envelope monitor is built from normalized embeddings of nominal patches and calibrated offline using nominal no-leak recordings. Its file-level threshold targets a 1% nominal false-alarm rate.

Offline construction and calibration

  • The reference store collects L2-normalized 64-dimensional penultimate-layer embeddings from all nominal patches, including both classes.The embeddings are extracted from the dense output after GELU and before dropout.Source: Offline construction and calibration · p. 9
  • The threshold is the 99th percentile of leave-one-file-out nominal no-leak recording scores, targeting a 1% nominal false-alarm rate.Each file’s patches are excluded from the reference store before scoring, and its patch scores are averaged per recording.Source: Offline construction and calibration · p. 9
  • Calibration uses recording-level mean scores, matching the runtime decision unit.Source: Offline construction and calibration · p. 9

Runtime

At runtime, the competence-envelope monitor scores normalized embeddings against nominal references and flags recordings whose mean patch score exceeds its calibrated threshold. A layer-profile alternative uses per-layer deviations to distinguish expected shift patterns.

Runtime

  • The dense k-NN monitor averages the 50 nearest-reference Euclidean distances per patch, then averages patch scores per recording for threshold comparison.On the unit sphere, the score is a monotone transform of cosine distance and acts as a local density estimate.Source: Runtime · p. 9
  • The dense-layer monitor flags inputs outside the competence envelope but does not identify what is wrong or where.Patch aggregation is used to keep the false-alarm rate low; a flag indicates extrapolation and potentially unreliable output.Source: Runtime · p. 10

2.2.4. L3 alternative - Deep k-NN-based layer-profile distance monitor

  • The layer-profile alternative expects noise shifts to affect early layers and decision-relevant semantic shifts to affect deeper layers, so profile shape can indicate shift type.It compares a four-dimensional per-layer distance profile with the profile of normality.Source: §2.2.4 · p. 10

Offline calibration and construction

The alternative monitor builds four per-layer reference clouds from normalized activations and standardizes their distances to form a profile. It scores block-averaged profiles against a nominal prototype using a threshold calibrated for a 1% false-alarm rate.

Offline calibration and construction

  • Per-layer k-NN distances use four reference clouds built from nominal training and validation patches, with leave-one-file-out scoring and k=50.Source: Offline calibration and construction · p. 10
  • Layer distances are standardized using the mean and standard deviation of nominal training-file distances because raw scales differ across layers.Source: Offline calibration and construction · p. 10
  • The normality prototype is the mean z-profile over nominal training patches from both classes; it formalizes the normalized-space origin because the same data define the z-scores.Source: Offline calibration and construction · p. 10
  • The monitor scores the Euclidean distance of profiles averaged over 20-patch blocks, with a threshold set to the 99th percentile of nominal no-leak block scores under leave-one-out calibration.The threshold targets a 1% false-alarm rate, and calibration uses the same 20-patch block unit as runtime decisions.Source: Offline calibration and construction · p. 10

Runtime

At runtime, the layer-profile monitor compares each 20-patch averaged profile with a calibrated threshold and reports the profile as a shift diagnostic. The coordination layer combines local diagnoses into fixed system states and a traffic-light display for supervisors.

Runtime

  • A profile-distance threshold exceedance raises a flag, while the z-profile shape is reported to distinguish expected early-layer covariate shifts from deeper-layer semantic shifts.Source: Runtime · p. 11

2.3. Coordination and communication layer

  • The fixed taxonomy combines local diagnoses into eight operational states and is not tuned on evaluation results.Source: §2.3 · p. 11
  • An L1 override blocks downstream judgments because learned or physics assessments are not meaningful on a corrupted signal.Source: §2.3 · p. 11
  • The four-level traffic light reserves red for a hidden fault in an unfamiliar context and is intended to make coordinated diagnoses comprehensible while allowing deeper inspection.Source: §2.3 · p. 11

3. Experiments

The experiments simulate deployment with held-out shifts while restricting monitor design and calibration to nominal data, then compare monitoring configurations on reliability and supervisor-oriented outcomes.

3. Experiments

  • The deployment premise treats background-noise and new-leak shifts as expected but unpredictable, while fixed integrity rules can target more specifiable sensor faults.Training, calibration, and design decisions are restricted to the nominal envelope; hydraulic noise and tube leaks represent the two held-out shift types.Source: §3 · p. 12

3.1. Held-out shift conditions

  • Held-out evaluation covers hydraulic-background covariate shift, novel tube-leak semantic shift, and synthetically injected sensor faults.Nominal session-3 files serve as a sanity anchor.Source: §3 · p. 12, §3.1 · p. 12

3.2. Freeze settings after development

  • All monitor design, calibration, and threshold choices use nominal data, are frozen, and face one post-freeze evaluation of held-out conditions.Pre-registered sensitivity variants are reported descriptively; approaches failing acceptance criteria are reported as negative results.Source: §3.2 · p. 12

3.3. Evaluation metrics

  • The evaluation compares the bare AI system with single-monitor arms and reports monitor outputs, taxonomy states, and traffic-light distributions.The configurations are S0 bare AI, S1a/b/c with MSP, L2, or L3, and human-supervisor-oriented metrics include trusted-wrong answers, missed-leak coverage, and nominal false alarms.Source: §3.3 · p. 13

4. Results

Across held-out conditions, COMET coordinates monitors to reduce trusted wrong answers and communicate fault states, while clipping and semantic-shift misses remain important boundaries.

4.1. Baseline performance and AI system behavior under shift

  • The bare AI system reaches 37.5% trusted-wrong answers on loud hydraulic no-leak files and 30.2% on tube leaks, while tube-leak recall falls to 69.8% from 93.1%.False alarms on loud hydraulic backgrounds and misses on tube leaks are confident and silent; faulted audio still yields leak detections on 25–69% of files.Source: §4.1 · p. 13

4.2.1. L1 - Signal-integrity monitor on injected faults

  • L1 catches 100% of dropout, flat-line, and offset injections but only 9.6% of clipping injections, allowing clipped files to reach trusted states.The clipping-rule defect also over-fires on microphone-1 and raises COMET’s nominal false-alarm cost.Source: §4.2.1 · p. 13, §4.3.3 · p. 15, §4.6.2 · p. 18

4.2.2. L2 - Physics-consistency monitor under covariate and semantic shift

  • L2 issues zero consistent verdicts on hydraulic no-leak files, abstaining on 83–88% of loud shifted-background cases; on tube leaks, it mostly abstains and inconsistency rarely fires.On nominal leaks it confirms 47.2% and abstains on 44.7%; the tube-leak inconsistency rate is 1.4%.Source: §4.2.2 · p. 14

4.2.3. L3 - Feature-space monitor k-NN vs layer-profile

  • The layer-profile monitor is more sensitive to hydraulic shifts than dense k-NN, but its nominal false positives make it a diagnostic complement rather than the L3 fusion vote.Hydraulic profiles peak in early convolutional layers and tube-leak profiles at the dense layer; dense k-NN flags 74.5% of tube leaks.Source: §4.2.3 · p. 14, Figure 3 · p. 15

4.3. Taxonomy states and traffic lights under shift

  • COMET reports hydraulic no-leak errors as safe states, with zero trusted false alarms, while 70.1% of tube-leak files are deferred as T4 and 8.3% remain all-clear.The hydraulic no-leak all-clear share is 67.7% on hydr-low and 36.1% on hydr; nominal no-leak operation is 95.6% all-clear.Source: §4.3 · p. 14, §4.3.1 · p. 15, §4.3.2 · p. 15

4.3.4. Traffic-light view

  • Dropout, flat-line, and offset faults appear as 100% yellow, while tube leaks are 70.3% orange; clipping remains the sensor-fault exception.The traffic-light mapping corresponds to taxonomy states pooled by condition.Source: §4.3.4 · p. 16, Figure 4 · p. 16

4.4. Ablation ladder

  • COMET reduces trusted-wrong-answer rates to 0% on hydraulic no-leak data and 0% on dropout, flat-line, and offset faults; nominal no-leak false alarms are 4.4%, or about 1% without the microphone-1 defect.The ablation comparison includes the bare AI system, three single-monitor arms, and COMET; the 4.4% cost is driven by the clipping-rule defect.Source: Figure 5 · p. 16, Table 5 · p. 17, §4.4 · p. 17

4.6.4. Residual worst case

  • COMET leaves 48 of 576 tube-leak files all-clear, including 22 high-confidence AI misses, and its clipping-rule blind spot lets 63.5% of clipped files reach trusted states.Rejected windowed aggregation produced 4–19% nominal false alarms, so the calibrated decision unit remains the 30-second recording.Source: §4.6.4 · p. 18, §4.6.2 · p. 18, §4.6.1 · p. 18

5. Discussion

COMET’s layers address largely disjoint fault modes, and coordination turns monitor flags into diagnoses, but the evaluation also exposes residual misses and limits to the evidence for human and system-level benefits.

5.8. Ablation

  • COMET reduced trusted-wrong-answer rates to zero on hydraulic no-leak data and three of four sensor faults at ≈1% nominal false alarms, while residual tube leaks still passed silently.On tube leaks, COMET and k-NN had similar TWA (8.3% vs. 8.5%) and silent-miss shares (27.6% vs. 28.2%).Source: §5 · p. 18, §5.8 · p. 21

5.2. Experimental design

  • All monitors were frozen after nominal-only design and tested zero-shot; the held-out covariate- and semantic-shift labels were assigned a priori, with z-profiles supporting the distinction.Hydraulic shift peaked in early convolutional layers, while tube-leak shift peaked in the dense task-oriented layer.Source: §5.2 · p. 19

5.3. Human supervisor

  • COMET’s coordination and communication layers turn detection into fault diagnosis, but more monitors risk information overflow and the taxonomy’s attributions can mislead supervisors.Taxonomy states are hypotheses; the interface should preserve access to individual monitor outputs, and an absent alarm is not evidence of safety.Source: §5.3 · p. 19

5.4. Signal-integrity monitor

  • Signal-integrity rules intercepted dropout, flat-line, and offset injections at 100%, but clipping detection caught only 9.6% of soft-clipping injections and over-fired on 15% of microphone-1 nominal no-leak files.Specified rules are auditable but not necessarily correct; revisions require nominal-data revalidation, and deterministic guarantees cover only specified faults.Source: §5.4 · pp. 19–20

5.5. Physics-consistency monitor

  • The physics monitor abstained on 83–88% of files under covariate shift, yet had zero false inconsistencies on nominal leaks and zero physics-supported false alarms under covariate shift.Its fixed template can confirm known leaks but cannot falsify leaks; the implementation is deliberately simple, and not every task offers readily modeled physics.Source: §5.5 · p. 20

5.6. Feature-space monitor

  • Feature-space OOD detection was the strongest single monitor, but scalar k-NN separation is imperfect and residual tube leaks passed all layers; its single-monitor output says only that inputs are unfamiliar.The penultimate representation was collapsed to a participation ratio of 1.4–2.5 of 64 dimensions; supervised-contrastive retraining failed its preregistered criterion.Source: §5.6 · p. 20

5.7. Coordination and communication layer

  • The traffic-light taxonomy trades fast, intuitive decisions against pre-structuring judgment, while raw monitor outputs risk information overflow; its design should be co-designed with operators.The current coordination layer is a static, rule-based taxonomy frozen before evaluation; runtime updating would require its own governance and validation.Source: §5.7 · p. 21

5.9. Limitations

  • The evidence is limited to one dataset, and COMET does not itself make a socio-technical system safe; operator studies are still needed to test decision benefits and co-design presentation.The physics layer assumes modelable task physics, and how the coordinated principle generalizes to other domains remains to be tested.Source: §5.9 · pp. 21–22

6. Conclusion

The conclusion presents COMET as a coordinated monitoring framework that diagnoses faults for human supervisors. In its compressed-air leak evaluation, it reduced trusted-wrong answers in selected conditions, while undetected tube leaks remained a boundary of passive monitoring.

6. Conclusion

  • COMET combines signal-integrity, physics-consistency, and feature-space monitors through coordination into taxonomy states communicated with a traffic-light display to a human supervisor.The layers check sensor faults, agreement with a spectral leak template, and whether inputs lie within the AI system’s learned competence envelope.Source: §6 · p. 22
  • COMET reduced trusted-wrong answers to zero on covariate-shifted no-leak data and three of four injected sensor faults, at ≈1% nominal false alarms, but 27.6% of AI-undetected tube leaks passed all monitors silently.The zero-shot evaluation used an open compressed-air leak dataset with held-out background noises, a novel leak type, and injected sensor faults.Source: §6 · p. 22
  • The authors conclude that coordinated monitors with disjoint fault modes can support fault diagnosis, while further evaluation, human-operator validation, and system-level safety research remain future work.The framework’s diagnosis can isolate and sometimes identify faults, informing runtime decisions and later maintenance.Source: §6 · p. 22

CRediT authorship contribution statement

The contribution statement assigns original drafting, methodology, investigation, analysis, data curation, validation, visualization, and conceptualization to Niclas Flehmig; Mary Ann Lundteigen and Shen Yin are credited with review and editing and supervision.

CRediT authorship contribution statement

Loading 2610.29816v1…