PaperlayerView exact revision on arXiv

Source-linked AI summary

Uncertainty-Guided Counterfactual Sensor Selection and Safety-Gated Recognition for Autonomous Sensing Agents in IoT Systems

Shirong Guo, Hongming Zhou, Muhammad Attique Khan, Jinfeng Xu, Yonis Gulzar, Manar Mousa Altamimi, Kunfu Wang

arXiv:2610.35344v1cs.NI

TL;DR

Sensor degradation can make multichannel evidence unreliable, yet clean-input recognition does not identify which sensor could correct a decision. The paper pairs controlled sensor restorations with calibrated risk and cost-sensitive action selection; on LARa, PGIK-Fusion and the counterfactual policy improve recognition and automation outcomes against their stated baselines.

  • Problem

    Degraded observations can change predicted classes and evidence reliability, while clean-input accuracy neither establishes confidence after degradation nor identifies a sensor that could correct an error.

    Source: §I · p. 1

  • Method

    The method pairs factual degraded inputs with controlled single-sensor restorations, then uses pre-request calibrated risk, predicted utility, and cost to select acceptance, one request, or handoff.

    Source: §I · pp. 1–2

  • Results

    On LARa, PGIK-Fusion reached 0.696 Macro-F1, 2.00 percentage points above raw XGBoost–1D-CNN fusion; at matched action rates, automation accuracy rose 18.51 points and expected cost fell 24.7% versus an action-rate-matched random policy.

    Source: §V · p. 13

  • Takeaways & Limitations

    The task-bounded evidence manager distinguishes recognition quality, recoverability, and evidence sufficiency while linking candidate restorations to acceptance, request, or handoff.

    Source: §V · p. 13

    Evidence covers subject-held-out logistics data, one recorded wind-turbine event, predefined simulated faults, and single-step actions; cross-event behavior requires separate validation.

    Source: §V · p. 13

Abstract

from arXiv · show

Autonomous IoT systems rely on distributed sensing, but sensor faults and communication loss can leave multichannel evidence incomplete or unreliable. Their agents must therefore classify measurements, assess evidence sufficiency, select a sensor to reacquire, and hand off unsafe decisions. We formulate counterfactual sensing: a degraded window defines the factual state, each candidate restoration defines a controlled alternative, and a pre-request utility model estimates loss reduction. Physics-Guided Invariant-Kinematic Fusion (PGIK-Fusion) constructs class evidence, while calibrated risk and cost-sensitive utility select acceptance, sensor request, or handoff from factual information. On the Logistic Activity Recognition Challenge dataset, PGIK-Fusion achieves 0.768 accuracy and a macro-averaged F1 score (Macro-F1) of 0.696, exceeding extreme gradient boosting (XGBoost)--one-dimensional convolutional neural network (1D-CNN) fusion by 2.49 and 2.00 percentage points, respectively. Across seven predefined degradation conditions on held-out subjects, the policy reduces unsafe acceptance from 33.0% to 13.8%; relative to an action-rate-matched random policy, it raises automation accuracy by 18.5 percentage points and lowers expected cost by 24.7%. In the auxiliary within-event wind-turbine study, PGIK-Fusion reaches 0.975 Macro-F1, improves on XGBoost--1D-CNN by 1.17 percentage points, and reduces unsafe acceptance from 20.1% to 8.0%. The evaluation characterizes the controlled responses of a frozen recognition model to specified sensor restorations under predefined fault families and severities. Within this scope, the results demonstrate that calibrated recognition, targeted evidence acquisition, and conservative deferral can form a unified perception--action loop that reduces unsafe automation under sensor degradation.

I. INTRODUCTION

Degraded multichannel observations require more than a class prediction: an agent must judge evidence and decide whether to accept, reacquire one sensor, or hand off. The paper frames this as counterfactual evidence control, combining physics-guided recognition with pre-request risk and utility.

I. INTRODUCTION

  • The proposed agent maps networked observations to accept, request-one-sensor, or handoff actions; its autonomy is task-bounded rather than tied to a particular application domain.Source: §I · p. 1
  • Missingness, noise, packet loss, and delay can change both predicted class and evidence reliability, while clean-input accuracy neither establishes confidence under degradation nor identifies a corrective sensor.Source: §I · p. 1
  • PGIK-Fusion pairs degraded factual inputs with controlled single-sensor restorations, then uses calibrated risk, predicted benefit, and acquisition cost to select acceptance, a request, or handoff.Source: §I · pp. 1–2

A. Autonomous Sensing Agents and Multichannel Perception

B. Temporal and Physically Structured Representations

  • Tree ensembles and raw-sequence encoders provide broad time-series task coverage, while other representations add temporal geometry, periodicity, sensor relations, or domain structure; these approaches do not estimate restoration value.Source: B. Temporal and Physically Structured… · p. 2

C. Uncertainty and Intervention-Aware Sensing

Calibration and selective prediction assess current evidence, but they do not specify which sensor restoration could improve a decision. The paper defines a single-window action loop that estimates candidate recovery and weighs it against risk and cost.

C. Uncertainty and Intervention-Aware Sensing

  • Calibration assesses whether confidence tracks correctness, and selective prediction controls risk and coverage through rejection; neither identifies which sensor restoration could improve a decision.Source: C. Uncertainty and Intervention-Aware… · p. 2

D. Counterfactual Reasoning and Cost-Aware Acquisition

  • The counterfactual estimand is the frozen recognition model’s controlled response to a specified restoration, not the natural causal effect of sensor failure.Source: D. Counterfactual Reasoning and… · p. 3
  • Candidate-specific loss reduction is supervised for restoration of one degraded physical sensor, while handoff remains available when neither current nor restored evidence supports automation.Source: D. Counterfactual Reasoning and… · p. 3

III. METHODOLOGY

  • At validation, frozen factual and restored-window responses define calibration, risk, and benefit targets; deployment uses only pre-request evidence, with fitted components fixed during testing.Source: §III · p. 3

A. Task Formulation and Requestable Sensors

  • For each synchronized window, the finite action set includes accepting the factual label, requesting one sensor for one recomputation before release, or handing off without automated release.Source: A. Task Formulation and Requestable… · p. 3
  • The request set is domain-specific—three IMUs for LARa and load, strain, and vibration for MDWTBM-LS—and the pre-request state retains class evidence, uncertainty summaries, and sensor quality.Source: A. Task Formulation and Requestable… · p. 3

B. Physics-Guided Invariant-Kinematic Fusion

PGIK-Fusion combines independent temporal and physics-guided probability estimators, enriching raw sequence evidence with descriptors adapted to each sensing system. Its fusion weight is selected on validation data and fixed before testing.

B. Physics-Guided Invariant-Kinematic Fusion

  • PGIK-Fusion independently trains a 1D-CNN temporal branch and an XGBoost branch over statistical-spectral and system-specific physical descriptors, combining their probabilities rather than latent features.Source: B. Physics-Guided Invariant-Kinematic… · p. 3
  • The statistical-spectral summary captures location, dispersion, range, distribution shape, signal energy, and coarse frequency allocation; applied to LARa’s 27 channels, it produces 324 base descriptors.Source: B. Physics-Guided Invariant-Kinematic… · p. 4
  • For wearable signals, physical augmentation summarizes magnitude, first-order change, and directional consistency, adding 216 descriptors that encode axis-robust magnitude, transition strength, and relative direction.Source: B. Physics-Guided Invariant-Kinematic… · p. 4
  • The magnitude and directional-consistency operators are invariant to a shared orthogonal rotation or reflection of the triaxial frame, but not to independent sensor displacement or time-varying misalignment.Source: B. Physics-Guided Invariant-Kinematic… · p. 4
  • For MDWTBM-LS, load, strain, and vibration vectors yield magnitude, first-difference, and within-modality cross-position similarity sequences, expanding the statistical-spectral representation from 144 to 324 dimensions.Source: B. Physics-Guided Invariant-Kinematic… · p. 5
  • Validation Macro-F1 selects the structured-evidence weight from 0 to 1 in 0.05 increments, yielding 0.90 on LARa; features, branches, and weight are fixed before calibration, interventions, and testing.Source: B. Physics-Guided Invariant-Kinematic… · p. 5

C. Controlled Interventions and Sensor Utility

The section defines controlled sensor restorations as counterfactual alternatives to degraded factual inputs, then estimates their candidate-specific recovery value against acquisition costs. The intervention results are scoped to predefined fault families and severities.

Controlled interventions

  • The interventions model complete IMU loss, two noise levels, 20% packet loss, and a 10-sample delay, covering source removal, amplitude corruption, outage, and misalignment.Both branches and derived features are recomputed; the same intervention families and severities are used for policy development and held-out-subject evaluation.Source: C. Controlled Interventions and Sensor… · p. 5

Intervention inputs

  • Sensor-quality indicators encode supplied degradation conditions, not outputs from a field fault detector.For example, packet loss and delay use retained-sample fractions, while noise quality is one minus normalized noise amplitude.Source: C. Controlled Interventions and Sensor… · p. 5

Counterfactual targets

  • Each candidate restoration changes exactly one requestable source while holding other observed channels fixed; deployment selection uses only the factual state and candidate identity.Validation supervision can use restorations and labels, unlike deployment selection.Source: C. Controlled Interventions and Sensor… · p. 5, Figure 3 · p. 5

Sensor utility

  • Candidate benefit is the signed change in 0–1 decision correctness: +1 for error correction, 0 for unchanged correctness, and −1 for a newly introduced error.The label and restored input are unavailable before a request.Source: C. Controlled Interventions and Sensor… · pp. 5–6
  • The policy separates factual error risk from candidate-specific recovery, using a logistic risk model and a ridge predictor for expected candidate benefit.The ridge predictor uses standardized state, candidate identity, and their interactions; its predictions are clipped to [−1, 1].Source: C. Controlled Interventions and Sensor… · p. 6

Acquisition cost

  • Request cost combines normalized energy, latency, and bandwidth burden; LARa assigns each request cost 0.10 and tests sensitivity from 0.05 to 0.25.The equal primary cost reflects the three IMUs’ equal channel widths and lack of platform telemetry; deployed costs can use device telemetry and application priorities.Source: C. Controlled Interventions and Sensor… · p. 6

D. Uncertainty and Finite Safety Actions

The section combines probability calibration and set-valued uncertainty with a cost-based choice among acceptance, sensor request, and handoff. Its deployment policy permits one request and releases the updated prediction without a second gate.

Uncertainty calibration

  • Temperature scaling adjusts probability sharpness without changing the predicted class, and calibration is assessed with ECE, NLL, Brier score, and risk–coverage levels.LARa selects temperature 1.45 on validation data.Source: D. Uncertainty and Finite Safety Actions · p. 6

Set-valued uncertainty

  • Split conformal uncertainty targets marginal coverage under exchangeability, but shared temperature-selection and quantile-estimation subjects mean test coverage is empirical rather than backed by the standard finite-sample guarantee.The reported target miscoverage is 0.10, with threshold 0.933 on LARa; a disjoint conformal-calibration partition would be required for that guarantee.Source: D. Uncertainty and Finite Safety Actions · p. 6

Finite safety actions

  • The finite safety gate places acceptance, acquisition, and deferral on one cost scale, suppressing high-risk automated outputs through a sensor request or handoff.With request cost 0.10, handoff cost 0.30, and unit error cost, the policy can accept low-risk cases, request for recoverable risk, or hand off high residual risk.Source: D. Uncertainty and Finite Safety Actions · pp. 6–7
  • A request is preferred only when predicted recovery offsets acquisition cost and post-request loss falls below handoff cost; candidate-specific net utility selects the sensor.The release decision also depends on factual risk and the handoff cost.Source: D. Uncertainty and Finite Safety Actions · p. 7

Deployment policy

  • The deployed policy allows at most one sensor request, then releases the recomputed prediction without another accept/request/handoff decision.After reacquisition, affected features are reconstructed and PGIK-Fusion is rerun.Source: D. Uncertainty and Finite Safety Actions · p. 7

Policy evaluation

  • The action-rate-matched random comparator preserves aggregate action counts while randomizing allocation and sensor choice; unsafe acceptance is the error fraction among accepted windows.The Oracle uses realized restoration outcomes only as a retrospective lower bound.Source: D. Uncertainty and Finite Safety Actions · p. 7

E. Datasets and Evaluation Protocols

The primary evaluation uses person-held-out activity recognition on the MotionMiners IMU subset of LARa, with a separate within-event wind-turbine forecasting study as an auxiliary protocol.

Primary protocol

  • The LARa protocol uses 200-sample windows from three 100-Hz IMUs and splits eight available subjects into model-fitting, validation, and final test groups.S07–S10 fit the model, S11–S12 support validation and policy fitting, and S13–S14 provide final evaluation; estimates apply to this subset, not the full LARa subject population.Source: E. Datasets and Evaluation Protocols · p. 7

Auxiliary protocol

  • The auxiliary MDWTBM-LS study forecasts the maximum Normal, Transition, or Anomaly state in the hour after a 30-minute history around one recorded turbine lightning event.It uses three seeds; validation mean true-class NLL reduction selects the candidate modality for each degradation, and validation-trained gates allocate actions.Source: E. Datasets and Evaluation Protocols · p. 7

F. Baselines and Implementation Details

The evaluation compares PGIK-Fusion with tree, sequence, transformed-domain, graph, and probability-fusion baselines. Neural baselines share an optimization budget, and latency is measured on CPU under a specified single-window protocol.

Baselines

  • The comparison set includes feature-based trees, recurrent, convolutional, and self-attention encoders, transformed-domain models, Sensor-GNN, TimesNet, ModernTCN, TimeMIL, and probability fusion.Transformed-domain and graph models are evaluated without XGBoost fusion.Source: F. Baselines and Implementation Details · pp. 7–8

Training protocol

  • LARa neural models share class-weighted cross-entropy, AdamW settings, batch size 128, a 50-epoch maximum, and early stopping on validation Macro-F1.The shared optimization budget controls training differences across neural baselines.Source: F. Baselines and Implementation Details · p. 8

Efficiency measurement

  • Single-window latency is measured on CPU with one thread and batch size one after warm-up; PGIK-Fusion timing includes physical feature construction, XGBoost, TorchScript 1D-CNN inference, and probability fusion.Reported latency is the median of seven trials.Source: F. Baselines and Implementation Details · p. 8

A. Perception Comparison and Backbone Selection

On held-out S13–S14, PGIK-Fusion led the tested models in accuracy and Macro-F1, while comparisons also expose latency and compact-backbone trade-offs.

A. Perception Comparison and Backbone Selection

B. Physical Evidence, Class Effects, and Efficiency

Physical descriptors improved aggregate Macro-F1 but affected classes and metrics unevenly. Calibrated selective prediction showed a risk–coverage trade-off, while the evaluated perception-plus-policy path remained submillisecond.

B. Physical Evidence, Class Effects, and Efficiency

  • Directional consistency increased Macro-F1 by 1.73 percentage points over raw fusion and recovered approximately 86% of the full 2.00-point gain.The joint descriptor combination added only 0.27 points beyond that variant, while the kinematic-only variant attained the highest balanced accuracy (0.698), 3.43 points above the full model.Source: B. Physical Evidence, Class Effects,… · p. 9
  • The physics-guided projection formed locally tighter Cart and Synchronisation regions, while handling activities remained substantially overlapped.The t-SNE geometry is descriptive because neighborhood projections do not quantify classification error.Source: B. Physical Evidence, Class Effects,… · p. 9, Figure 5 · p. 9
  • Four class-level recall gains were 9.09, 3.57, 3.61, and 2.60 percentage points; recall fell 3.06 points for Handling up, 7.53 for Handling down, and 8.33 for Synchronisation.Relative to releasing every factual prediction, erroneous accepted outputs fell from 76.3% to 16.1% for Standing and from 65.23% to 11.79% for Handling down; protection arose mainly from routing rather than reliable direct acceptance.Source: B. Physical Evidence, Class Effects,… · p. 10
  • The perception-plus-policy path measured 0.698 ms per window, including 0.520 ms for perception and 0.179 ms for the gate.The gate latency was 34.4% of perception latency under the stated batch-one, single-thread CPU protocol.Source: B. Physical Evidence, Class Effects,… · p. 10

C. Calibrated Evidence and Selective Risk

D. Counterfactual Sensor Recovery and Safety Actions

The section separates fault severity from restoration value, then evaluates whether candidate-specific sensing improves safety-policy decisions. Across seven degraded conditions, the policy reduces unsafe acceptance and outperforms action-rate-matched random allocation; the analysis also reports restoration-ranking limits and illustrative single-step decisions.

D. Counterfactual Sensor Recovery and Safety Actions

  • Restoring the affected sensor yielded positive mean net utility for trunk and right-IMU loss (0.021 and 0.062), while unaffected-sensor restorations had negative values.The selector chose the matching sensor for 98.5% of trunk-loss windows and all left- and right-loss windows; other fault types had negative mean utility after request cost.Source: D. Counterfactual Sensor Recovery and… · pp. 10–11, Figure 10 · p. 11
  • 85.22% of corrective restoration opportunities were captured by the selected candidate, while predicted benefit had a 0.219 Pearson correlation with realized loss reduction.Only 12.53% of degraded window-condition instances contained at least one corrective restoration; mean regret versus the retrospectively best candidate was 0.048 loss units per window.Source: D. Counterfactual Sensor Recovery and… · p. 11

E. Counterfactual Policy Comparison and Agent Decisions

  • 13.84% unsafe acceptance versus 33.03% for always accepting; against action-rate-matched random allocation, automation accuracy rose 18.51 percentage points and expected cost fell 24.7%.Candidate-specific allocation reduced unsafe acceptance by 19.02 percentage points; gains arose from where the policy acted, not how often.Source: Figure 11 · p. 11, E. Counterfactual Policy Comparison… · p. 11
  • Illustrative decisions requested right-IMU restoration when predicted risk reduction exceeded request cost, handed off when residual risk remained high, and accepted when risk was low without a cost-effective request.The examples use pre-request evidence and do not supply post-request outcomes to the policy.Source: Figure 12 · p. 12, E. Counterfactual Policy Comparison… · p. 12
  • At request costs of 0.05–0.25, request rates fell from 22.37% to zero at costs of at least 0.20, while expected cost ranged from 0.236 to 0.242.Confidence-only gating had 14.68% unsafe acceptance, 0.853 automation accuracy, and 0.250 expected cost; quality-only gating had 26.6% unsafe acceptance.Source: E. Counterfactual Policy Comparison… · p. 12
  • The counterfactual policy remained 0.166 above the Oracle cost of 0.074, with candidate-selection regret and risk- and benefit-estimation errors among the observed gap sources.Its performance fell between action-rate-matched random allocation and the retrospective Oracle lower bound.Source: E. Counterfactual Policy Comparison… · p. 12

F. Auxiliary Study in Wind-Turbine Blade Monitoring

In the auxiliary wind-turbine study, PGIK-Fusion improves perception performance and the counterfactual policy reduces unsafe acceptance. These results are from partitions of one recorded lightning event, so cross-event generalization remains untested.

F. Auxiliary Study in Wind-Turbine Blade Monitoring

  • The auxiliary evaluation forecasts the maximum Normal, Transition, or Anomaly state in the next 60 min from blade load, strain, and vibration histories.Models were retrained under a block-partition protocol with seeds 2026, 2027, and 2028.Source: F. Auxiliary Study in Wind-Turbine… · p. 12
  • 0.975 ± 0.022 Macro-F1 for PGIK-Fusion versus 0.963 ± 0.020 for XGBoost–1D-CNN; the mean paired increase over raw fusion was 1.17 percentage points.Two seeds improved and one tied, so the average gain was not uniform across seeds.Source: F. Auxiliary Study in Wind-Turbine… · p. 12
  • Removing load, strain, and vibration reduced Macro-F1 by 32.25, 38.50, and 37.70 percentage points, respectively, despite high clean performance.Temperature scaling reduced mean ECE and Brier score but increased test NLL; conformal coverage at α=0.10 was 0.909 ± 0.110.Source: F. Auxiliary Study in Wind-Turbine… · p. 12
  • Unsafe acceptance fell from 0.201±0.030 under always accepting to 0.080±0.057, a 60.2% relative reduction.Against action-rate-matched random allocation, expected cost fell 44.5%, from 0.194 ± 0.024 to 0.108 ± 0.033; automation accuracy rose 9.57 percentage points.Source: F. Auxiliary Study in Wind-Turbine… · p. 12, Figure 13 · p. 13
  • All training, validation, and test blocks came from one recorded lightning event; cross-event generalization requires evaluation on independent events.Overlapping histories and forecast horizons leave residual temporal dependence between nearby blocks.Source: F. Auxiliary Study in Wind-Turbine… · p. 13

V. CONCLUSION

  • The conclusion frames the evidence as a perception–action loop across distinct IoT sensing configurations, with deployment requiring sensor-quality metadata, reacquisition or retransmission, and costs tied to operational constraints.The evaluated scope uses predefined simulated fault families and severities and single-step actions; unseen faults and cross-event behavior need separate validation.Source: §V · p. 13

Related papers

Loading 2610.35344v1…