PaperlayerView exact revision on arXiv

Source-linked AI summary

Estimating Inconsistency Response Surfaces under Uncertainty in Cyber-Physical System Development

Johannes Mäkelburg, Tim Schwabe, Maribel Acosta

arXiv:2609.11331v1cs.LGcs.SEeess.SY

TL;DR

CPS uncertainty can make interacting models inconsistent, while existing methods provide limited support for exploring large uncertainty spaces. The paper models inconsistency as an intervention response, trains a geometry-based surrogate, and reports accurate approximation with microsecond-scale evaluation. The resulting surfaces support sensitivity analysis and minimal uncertainty interventions for restoring consistency.

  • Problem

    Existing approaches can determine consistency for particular uncertainty configurations but provide limited support for systematically learning and analyzing inconsistency across large uncertainty spaces.

  • Method

    The paper combines Saltelli sampling, multi-fidelity Monte Carlo estimation, and a surrogate that predicts inconsistency from propagated uncertainty geometry.

  • Results

    Across 48 scenarios, the surrogate accurately approximates Monte Carlo estimates while reducing evaluation time from milliseconds to microseconds.

  • Takeaways & Limitations

    Learned inconsistency response surfaces support sensitivity analysis and recourse interventions that restore consistency.

Abstract

from arXiv · show

Cyber-Physical Systems (CPS) are commonly represented through multiple interconnected models. During development, CPS consistency requires that shared model elements remain compatible across these models. Uncertainty, for example, due to sensor noise or model abstraction, changes the admissible values of model elements and can introduce inconsistencies, i.e., situations in which models can no longer be jointly satisfied. While existing approaches can determine consistency for a given uncertainty configuration, they provide limited support for systematically exploring, analyzing, and explaining inconsistency across large uncertainty spaces. We address this challenge by reformulating inconsistency as an intervention response modeling problem. Using Saltelli sampling and multi-fidelity Monte Carlo estimation, we generate intervention-response datasets and train a surrogate model that directly predicts inconsistency from the propagated uncertainty geometry. Experiments on 48 scenarios and 10 CPS domains show that the surrogate matches Monte Carlo estimates while reducing evaluation time from milliseconds to microseconds, enabling orders-of-magnitude more response-surface evaluations within fixed computational budgets. Building on the learned response surfaces, we perform sensitivity analysis to identify dominant uncertainty drivers and introduce a gradient-based consistency recourse method to determine minimal uncertainty interventions that restore consistency. The results show that inconsistency under uncertainty can be effectively learned, analyzed, and repaired through response-surface modeling, providing a scalable foundation for uncertainty-aware consistency management in CPS development.

I. INTRODUCTION

CPS consistency requires compatible shared model elements, but propagated uncertainty can make interacting models jointly infeasible. The paper reframes this challenge as learning and analyzing inconsistency response surfaces over uncertainty interventions.

  • Uncertainty from parameter variation, sensor noise, and abstraction gaps can propagate across interacting models and induce inconsistencies absent at nominal conditions.
  • Under uncertainty, consistency concerns whether admissible model configurations remain jointly feasible across interacting uncertainty sets.
  • Existing methods can test a particular configuration but offer limited support for systematically learning and analyzing global inconsistency behavior across uncertainty spaces.
  • The paper reformulates inconsistency as an intervention-response function I(θ) and combines surrogate modeling with systematic intervention-response data generation.
  • Consistency is evaluated by propagating mapped uncertainty sets and checking whether their intersections contain jointly admissible realizations.

III. METHODOLOGY

The methodology propagates uncertain model states through mappings, constructs a global feasible region, and estimates inconsistency by aggregating sampled consistency outcomes. This response is then represented as I(θ) over intervention configurations.

  • Intervention configurations induce parameterized uncertainty sets and transform inconsistency into a response function over the intervention space.
  • The consistency pipeline propagates uncertainty sets through mappings and intersects them to obtain the global feasible region Zsys.
  • Sampled realizations are evaluated against Zsys to produce binary consistency scores, which are aggregated over stochastic realizations.
  • The resulting inconsistency function represents the probability that the system becomes inconsistent and is estimated numerically because exact feasibility reasoning is generally intractable.

B. Estimating the Inconsistency Response

The study compares Monte Carlo, axis-aligned bounding-box, and multi-fidelity estimators for inconsistency. MFMC improves computational efficiency while retaining close agreement with the Monte Carlo reference.

  • MC is unbiased and principled but computationally expensive, requiring many samples for accurate inconsistency estimates.
  • AABB is cheaper because it replaces zonotopes with axis-aligned boxes, but ignoring orientations causes systematic inconsistency overestimation.
  • MFMC combines a small-sample MC estimate with a correlated AABB control variate to reduce estimator variance and computational cost.
  • An observed correlation of approximately 0.94 yields an 8–11x variance reduction for MFMC.

C. Dataset Construction

The paper constructs intervention-response datasets over uncertainty geometry and uses them to train a geometry-aware Product-Set Transformer surrogate. Relative per-dimension and global features support efficient prediction and downstream response-surface analysis.

  • Dataset Construction: The intervention space parameterizes uncertainty geometry through scale, center displacement, and correlation changes.
  • Dataset Construction: Saltelli-sampled intervention configurations are labeled with MC estimates to approximate the inconsistency response landscape and support sensitivity analysis.
  • Learning Inconsistency Response Surfaces: The surrogate uses relative propagated-frame features, making predictions invariant to common translation, rotation, and scaling of the zonotopes.
  • Learning Inconsistency Response Surfaces: Per-dimension features encode offsets, magnitudes, alignment, and width ratios, while global features capture volume, distance, dimension, and support-function relationships.
  • Learning Inconsistency Response Surfaces: The Product-Set Transformer applies self-attention across dimensions and combines per-dimension logits into a joint non-containment estimate.
  • Learning Inconsistency Response Surfaces: The surrogate is trained on real and synthetic zonotope scenarios using full-MC inconsistency labels, with separate models for 2D–3D and 4D cases.

E. Analyzing Inconsistency Response Surfaces

The analysis parameterizes uncertainty modifications as intervention vectors and uses variance-based sensitivity analysis to quantify how individual factors and their interactions shape inconsistency.

  • Intervention Parameterization: Figure 4 illustrates scaling, center shifts, and correlation interventions applied to a zonotope.
  • Intervention Parameterization: Intervention vectors parameterize scale, center-shift, and correlation changes to uncertainty representations.For each uncertainty, scale modifies the generator matrix, while center-shift modifies the center and correlation modifies the uncertainty geometry.
  • Sensitivity Analysis: Sobol variance decomposition quantifies the relative importance of uncertainty factors and their interactions in the inconsistency response.
  • Sensitivity Analysis: The first-order Sobol index measures the contribution of parameter θj alone to inconsistency-function variance.

F. Consistency Recourse

Consistency recourse searches for small interventions that move an inconsistent uncertainty configuration below an operational consistency threshold, using optimization with MFMC or a differentiable surrogate.

  • Recourse Formulation: Recourse seeks a small change to an inconsistent configuration that restores consistency below the operational threshold.The starting configuration has I > l, where l denotes the operational consistency threshold.
  • Recourse Formulation: The recourse objective minimizes distance from the original intervention subject to predicted inconsistency remaining at or below l.
  • Optimization: The constrained recourse problem is converted into an unconstrained objective with a penalty term.
  • Optimization: MFMC supports evolutionary and finite-difference searches, whereas the differentiable surrogate enables direct gradient-based optimization with respect to θ.
  • Optimization: The repair vector δθ identifies which uncertainty parameters must change to restore consistency.

A. Experimental Case Studies

The study evaluates intervention-response estimation across 48 scenarios spanning ten CPS domains and compares geometric, multi-fidelity, and learned estimators against Monte Carlo references.

  • Case Studies: 48 scenarios across ten CPS domains form the structured intervention-response dataset used for evaluation.The domains include automotive systems, HVAC, industrial robotics, medical devices, railway systems, satellites, smart grids, water and chemical processes, and wind turbines.
  • Case Studies: Interventions vary uncertainty scale, center displacement, and correlation structure over zonotope-based representations.The scale factor ranges from 0.1 to 5.0, while normalized center shifts range from −1.0 to 1.0.
  • Estimator Evaluation: AABB is faster than Monte Carlo but exhibits larger deviations and systematic overestimation, whereas MFMC offers near-reference accuracy at higher computational cost.Figure 5 compares agreement with the Monte Carlo reference, and Figure 6 summarizes the accuracy-efficiency tradeoff.
  • Estimator Evaluation: The learned surrogate closely matches the Monte Carlo reference, achieving ρ = 0.993 and R2 = 0.983 for overall inconsistency structure.MFMC achieves ρ = 0.997 and R2 = 0.997, while the surrogate is on par with MFMC in rank correlation for 4D scenarios (+0.015).
  • Estimator Evaluation: 935×: the learned surrogate reduces MFMC evaluation time to 3.54 µs per sample while maintaining competitive accuracy across CPS domains.MFMC achieves near-reference accuracy at approximately 3306 µs, while the surrogate’s MAE is below the standard error of the Monte Carlo labels.
  • Estimator Evaluation: The resulting hierarchy is coarse inexpensive geometric approximation, higher-cost near-reference MFMC estimation, and accelerated learned-surrogate estimation.

C. RQ2: Learning Inconsistency Response Surfaces

The learned surrogate accurately approximates inconsistency response surfaces and enables substantially faster exploration than Monte Carlo estimators. It also preserves sensitivity structure while revealing interaction-dependent landscape differences across CPS domains.

  • Learning inconsistency response surfaces: The surrogate maintains ρ values between 0.989 and 0.994 with MAE below 0.015 across evaluated zonotope dimensions.Its near-MFMC performance persists as dimensions increase, including 4D scenarios with only three training scenarios.
  • Learning inconsistency response surfaces: 4 µs per evaluation enables approximately 2.8 × 10^5 evaluations per second in 2D/3D, compared with roughly 300 for MFMC and 540 for AABB.In 4D, inference remains below 40 µs per sample and exceeds 27,000 evaluations per second.
  • Learning inconsistency response surfaces: The surrogate preserves both sensitivity structure and response-landscape geometry, supporting scalable sensitivity and intervention analysis.This enables exploration budgets several orders of magnitude larger than explicit MC or MFMC estimation.
  • Sensitivity and structural drivers: Center uncertainty has the strongest first-order influence, while correlation uncertainty is weak in isolation but interaction-driven through larger total effects.MC S1 for center uncertainty ranges from 0.36 ± 0.06 to 0.45 ± 0.06, whereas correlation uncertainty has S1 ≤0.08 and ST ≤0.20.
  • Sensitivity and structural drivers: Surrogate–MC differences remain within 0.02 for first-order and 0.04 for total Sobol indices across nine CPS domains.The largest reported deviation is ΔST(su) = +0.05 in Engineering, where confidence intervals are widest.
  • Intervention-response landscapes: Similar sensitivity rankings can produce different response geometries, from broad low-inconsistency regions in HVAC to no admissible region in Satellite.The landscapes therefore complement global Sobol measures by exposing domain-specific interaction structure.

E. RQ4: Consistency Recourse

The recourse experiments compare MFMC-guided and surrogate-based searches for low-inconsistency interventions, including repairs close to the initial configuration. Surrogate-based methods trade slightly worse repair distances for substantially faster runtime, while the hybrid method offers a strong compromise for unconstrained consistency.

  • Minimal-distance repair: The surrogate converges directly near the feasible boundary, whereas finite differences take suboptimal steps and CMA-ES reaches a slightly closer point with significantly more steps.CMA-ES requires 25 costly MFMC evaluations in the illustrated Engineering scenario.
  • Unconstrained search: For unconstrained low-inconsistency search, MFMC-based methods reach median inconsistency 0, while the surrogate alone reaches a significantly higher median.The hybrid method finds fully consistent configurations in almost half the runtime of CMA-ES.
  • Consistency recourse: MFMC-based search offers slightly better repairs, whereas surrogate-based gradient search provides significantly faster runtime.The comparison supports using the surrogate for rapid exploration and MFMC when repair quality is prioritized.

V. RELATED WORK

Prior work addresses consistency verification, expensive stochastic response estimation, geometric surrogates, and sensitivity analysis, but this paper unifies these ideas around inconsistency response surfaces for interacting CPS models.

  • Consistency Analysis under Uncertainty: Existing consistency approaches primarily verify deterministic model states or local uncertainty configurations rather than learning global inconsistency behavior.The paper positions its approach against deterministic consistency relations and limited support for systematic analysis across uncertainty spaces.
  • Response-Function Approximation: Prior response-function methods combine multi-fidelity approximations, variance reduction, surrogate modeling, and sensitivity analysis to explore expensive stochastic systems.These methods motivate the paper’s use of scalable estimation and learned response functions.
  • Response-Function Approximation: Unlike prior pairwise overlap or geometric-property predictors, this paper models inconsistency as a stochastic response induced by uncertainty propagation across interacting multi-model systems.The surrogate operates directly on zonotope-based uncertainty representations and estimates response surfaces over interventions.
  • Sensitivity Analysis and Feature Importance: The framework combines global sensitivity analysis with intervention-based reasoning and algorithmic recourse to identify dominant uncertainty drivers and restore consistency.Its stated scope includes systematic data generation, surrogate exploration, and minimal uncertainty interventions.
  • Results and Scope: Across 48 scenarios, the surrogate approximated Monte Carlo estimates and Sobol indices while reducing evaluation time from milliseconds to microseconds.The results support scalable exploration of inconsistency landscapes, although larger structures and higher-dimensional representations remain future work.

APPENDIX A SURROGATE ARCHITECTURE ABLATION

The ablation study shows that the Product-Set Transformer’s factorized architecture is especially valuable for the harder, data-scarce 3D holdout.

  • Component Ablations: Removing the product head or both the product head and attention degraded accuracy, while removing attention alone retained comparable cost with only 850 parameters.The ablation attributes performance to the noisy-AND factorization and attention together.
  • Component Ablations: The full model and the variant without the global head were indistinguishable on aggregate metrics, indicating that per-dimension factors carried most of the signal.Reported metrics use the best validation checkpoint, while some simpler variants degraded after early training.
Loading 2609.11331v1…