Source-linked AI summary
Guaranteed Conditional Performance of Control Charts via Bootstrap Methods
Axel Gandy, Jan Terje Kvaløy
TL;DR
Estimating the in-control state can impair control-chart performance, particularly the conditional performance after parameters are estimated. The paper uses bootstrap-based adjustments to guarantee desired conditional performance with high probability, reporting successful control of low in-control ARLs and broad applicability, subject to theoretical conditions.
Problem
Estimating the in-control state can substantially distort control-chart performance, while conditional ARL is often more relevant than unconditional ARL when charts run without re-estimation.
Method
The paper bootstraps data used to estimate the in-control state, constructing confidence intervals and tuning monitoring schemes for conditional performance given the estimated distribution.
Results
With 90% probability, bootstrap adjustment guarantees an in-control ARL of at least 100 in the illustrated CUSUM and avoids overly low ARLs at that prescribed probability.
Takeaways & Limitations
The approach applies across multiple control-chart types and can be robust to model specification error when a nonparametric bootstrap is used.
Takeaways & Limitations
The general applicability is stated under the conditions of Theorem 1, with extensions to settings such as regression, autocorrelation, and multivariate data presented as conjectures.
Abstract
from arXiv · showhide
To use control charts in practice, the in-control state usually has to be estimated. This estimation has a detrimental effect on the performance of control charts, which is often measured for example by the false alarm probability or the average run length. We suggest an adjustment of the monitoring schemes to overcome these problems. It guarantees, with a certain probability, a conditional performance given the estimated in-control state. The suggested method is based on bootstrapping the data used to estimate the in-control state. The method applies to different types of control charts, and also works with charts based on regression models, survival models, etc. If a nonparametric bootstrap is used, the method is robust to model errors. We show large sample properties of the adjustment. The usefulness of our approach is demonstrated through simulation studies.
1 Introduction
Estimating the in-control state can substantially distort control-chart performance, especially the conditional ARL relevant when estimated charts are run without re-estimation. The paper proposes bootstrap-based adjustments that guarantee desired conditional performance with high probability and applies broadly across chart settings.
- Motivation: Estimation error substantially affects control-chart performance, motivating methods that account for uncertainty in the estimated in-control state.The issue has been documented across control-chart applications and is especially relevant when the in-control distribution is unknown.
- Motivation: For a CUSUM with an estimated in-control mean, the conditional in-control ARL remains substantially affected by estimation error even with n = 1000.The ARL depends on the estimated mean and is therefore random; the illustration uses threshold c = 2.84 and ∆ = 1.
- Motivation: Averaging performance over parameter estimation can leave the conditional ARL considerably below 100, increasing the probability of shortened times to false alarms.This affects both the unadjusted threshold and the threshold chosen to correct the unconditional ARL.
- Proposed approach: The proposed method evaluates performance conditional on the estimated in-control distribution and constructs charts whose desired properties hold with high probability.The approach uses bootstrap procedures to adjust monitoring schemes and can target quantities such as ARL or false alarm probability.
- Results: For the illustrated CUSUM, bootstrap adjustment guarantees an in-control ARL of at least 100 with probability 90% and avoids overly low ARLs at that prescribed probability.The paper reports that the resulting increase in out-of-control ARL is modest and that hitting-probability criteria give similar results.
- Scope and novelty: Compared with prior exceedance-probability methods, the approach covers more chart types without deriving setting-specific approximation formulas and supports confidence intervals for fixed-threshold performance.A nonparametric bootstrap also provides robustness to model misspecification and can be used for bias correction.
2 Monitoring homogeneous observations
The paper defines conditional control-chart performance using estimated in-control parameters and constructs bootstrap-based one-sided confidence bounds to select thresholds with desired guarantees. The framework covers Shewhart and CUSUM charts, with parametric-bootstrap exactness in certain settings and a tail-behavior limitation for nonparametric Shewhart calibration.
- Conditional performance: Conditional performance q(P; ˆξ) evaluates a chart under the true in-control distribution while holding the estimated parameters used to run it fixed.Relevant measures include ARL, false-alarm probability hit, and threshold quantities cARL and chit.
- Bootstrap adjustment: The bootstrap method estimates one-sided confidence bounds for q(P; ˆξ) from bootstrap replicates of the estimated in-control distribution and parameters.The generic algorithm estimates ˆP and ˆξ, generates bootstrap samples, and uses an empirical quantile of bootstrap performance values.
- Bootstrap adjustment: Adjusted thresholds are chosen using cARL or chit, or their log transforms, to provide the desired in-control properties in approximately 1 −α of applications.The guarantee concerns the conditional performance given the estimated in-control state.
- Shewhart charts: For Shewhart charts, the method can use a parametric model and is exact in certain cases when the bootstrap sample size B tends to infinity.Exactness occurs when the distribution of the relevant performance-estimation error does not depend on unknown parameters.
- Shewhart charts: Shewhart calibration depends heavily on tail behavior, making small-sample nonparametric methods problematic; the paper primarily recommends a parametric bootstrap there.This is a scope limitation of the suggested implementation for Shewhart charts.
- CUSUM charts: The method also applies to CUSUM charts, including scaled normal-observation versions and likelihood-ratio forms for general in-control and out-of-control distributions.For some parametric CUSUM settings, the parametric bootstrap is exact under the same parameter-free error-distribution condition.
3 General theory
The paper develops a bootstrap framework for conditional control-chart performance when the in-control distribution and chart parameters are estimated. Under differentiability and bootstrap regularity conditions, the framework supports asymptotic guarantees and applies to nonparametric and parametric examples.
- Framework: The framework evaluates q(P; ξ̂), the chart property under the true in-control distribution conditional on estimated parameters, rather than averaging over parameter estimation.The setup allows P and its estimator to be distributions in a normed space, while ξ contains the parameters used to run the chart.
- Framework: Bootstrap inference uses an estimate P̂ and an independent random vector W_n to construct a bootstrapped version P̂* of the in-control distribution.With resampling, W_n is a multinomial weight vector; with a parametric bootstrap, it generates observations from the estimated parametric distribution.
- Theory: The main theorem extends the functional delta method by requiring Hadamard differentiability of q in its distribution argument while the parameter component converges.Its assumptions also include asymptotic convergence of √n(P̂−P), bootstrap validity, continuity of the limiting distribution, and measurability conditions.
- Examples: For nonparametric CUSUM examples, the framework covers hitting probabilities and thresholds chosen to achieve a target hitting probability.The threshold functional is Hadamard differentiable when P has a continuous, bounded, positive density that tends to zero in both tails.
- Scope: The examples are illustrative rather than exhaustive, and other performance measures such as log(chit) or logit(hit) require additional chain-rule arguments.The paper also notes that verifying one theorem condition in full is outside its scope.
- Examples: For normally distributed observations, the parametric CUSUM example estimates mean and variance and uses a parametric bootstrap; the relevant hit and threshold functions satisfy the differentiability condition.The normal setup identifies the distribution with its estimated parameters, which are also the parameters needed to run the chart.
4 Simulations for homogeneous observations
Simulations show that estimating the in-control state can badly distort conditional ARL, while bootstrap-adjusted thresholds achieve the targeted in-control guarantee with only a modest out-of-control cost. Nonparametric bootstrap performance is close to parametric performance under correct specification and is more reliable under misspecification.
- 4.1 Coverage probabilities: For n = 50, untransformed confidence-interval coverage is notably inaccurate, especially for ARL; log and logit transformations improve coverage.
- 4.2 The benefit of an adjusted threshold: For n = 50, the probability of an in-control ARL below 50 exceeds 20% with the unadjusted threshold.
- 4.2 The benefit of an adjusted threshold: The adjusted threshold achieves an in-control ARL of at least 100 in 90% of cases, while increasing the out-of-control ARL only slightly.
- 4.3 Nonparametric bootstrap - advantages and disadvantages: Under correct normal specification, parametric and nonparametric bootstrap charts perform almost identically, apart from slightly worse nonparametric in-control performance for n = 50.
- 4.3 Nonparametric bootstrap - advantages and disadvantages: Under exponential misspecification, the parametric chart misses the desired in-control probabilities, whereas the nonparametric chart performs well, particularly for n = 500.
5 Regression models
The paper extends bootstrap-based guaranteed-performance adjustments to risk-adjusted control charts built from linear, logistic, and survival models. Simulations indicate that the correction improves conditional in-control performance, including in survival analysis.
- Linear models: Risk-adjusted charts account for explainable heterogeneity through regression models, whose estimation error must also be incorporated.The paper considers linear, logistic, and survival models.
- Linear models: The linear-model procedure uses a nonparametric bootstrap of past in-control observations to estimate regression parameters and calibrate chart performance.The approach is intended to remain useful under linear-model misspecification.
- Logistic regression: The method is also applied to logistic-regression CUSUMs and can be extended to other generalized linear models by replacing the likelihood-ratio increment.Poisson regression monitoring is given as an example of the broader extension.
- Survival analysis models: Without adjustment, the desired false alarm probability of 0.1 is reached in roughly 60% of cases, versus roughly 90% after bootstrap correction.Increasing fitting and deployment lengths from n = 100 to n = 500 raises out-of-control hitting probabilities.
6 Conclusions and discussion
The paper proposes bootstrap calibration to guarantee, with high probability, conditional in-control performance after parameter estimation. It argues that this conditional focus is more relevant for charts operated without independent reestimation, while acknowledging that broader applicability remains conjectural in some settings.
- Conclusions and discussion: Bootstrap methods tune monitoring schemes to guarantee, with high probability, a specified conditional in-control performance given the estimated distribution.With a nonparametric bootstrap, the approach is described as robust against model specification error.
- Conclusions and discussion: Conditional performance is emphasized because an estimated chart is usually run for some time without independent reestimation.The paper contrasts this focus with average performance over estimation and chart operation.
- Conclusions and discussion: The approach is demonstrated for variants of Shewhart and CUSUM charts, while applicability to other charts and settings is conjectured when Theorem 1 holds.Examples mentioned include regression, autocorrelated, and multivariate data.
A Proof of the main theorem
The proof establishes bootstrap validity for performance measures involving both an estimated distribution and estimated chart parameters. It combines an extended functional delta method with conditional weak-convergence and quantile arguments.
- Main theorem: Lemma 3 handles q evaluated at an estimated distribution and parameter by combining continuity of ξ with Hadamard differentiability of q.The proof uses the extended continuous mapping theorem.
- Main theorem: The theorem proof applies Lemma 3 to joint mappings involving q and ξ, then follows subsequence arguments to establish the required convergence.Independent copies of the limiting random element are introduced in the proof.
- Main theorem: The bootstrap coverage argument uses conditional convergence to a limiting distribution and convergence of its quantile function.The argument extends from continuity points to all β using monotonicity and continuity properties.
B Proofs for Hadamard differentiability
The differentiability proofs supply the technical ingredients needed for the main theorem, including chain rules, inverse-map differentiability, and differentiability of CUSUM hitting probabilities.
- Hadamard differentiability: The appendix proves the lemmas required for the main theorem after establishing a chain rule and uniform Hadamard differentiability of inverse maps.It also proves differentiability of CUSUM hitting probabilities with respect to the updating distribution.
- Hadamard differentiability: These differentiability results are presented as potentially useful beyond the specific applications considered in the paper.
B.1 Chain rule
This section establishes a chain rule for families of functions that are Hadamard differentiable with respect to a parameter. Uniform Hadamard differentiability of the outer function and continuity in the parameter ensure differentiability of the compositions.
- The section introduces a stronger, uniform form of Hadamard differentiability for the chain-rule argument.
- Lemma 4 states that composing a parameterized Hadamard-differentiable family with a uniformly Hadamard-differentiable outer function preserves Hadamard differentiability.
- The chain rule requires continuity of the inner function in the parameter under the metric d.
B.2 Uniform Hadamard differentiability of the inverse map
The inverse map is defined as the first crossing point of a threshold and is shown to be uniformly Hadamard differentiable under smooth, strictly increasing baseline functions. The proof establishes convergence of perturbed crossing points using local threshold behavior and shrinking perturbations.
- The inverse map returns the first point where a non-decreasing function crosses the threshold β.
- The metric compares derivatives uniformly and is set to infinity when either function is not differentiable on the interval.
- The map is uniformly Hadamard differentiable when the baseline function has a continuous, bounded, positive derivative.
- The proof first shows that perturbed crossing points converge to the baseline crossing point.
- Choosing ε_n=o(t_n), together with bounded derivatives and convergence of intermediate points, controls the perturbation of the crossing point.
B.3 Differentiability of the hitting probability with respect to the updating distribution
This section proves uniform Hadamard differentiability of finite-horizon CUSUM hitting probabilities with respect to the distribution of adjusted updates. The argument decomposes the derivative and controls remainder terms uniformly in the chart threshold.
- The updates Y_i are adjusted observations, and the section studies hitting probabilities within the first T monitoring steps.
- The functional is uniformly Hadamard differentiable tangentially to continuous perturbations vanishing at both tails.
- The derivative decomposition separates a principal term from remainder terms that converge uniformly in the threshold c.
- The first remainder is controlled by |C_n| ≤ T∥H_n−H∥ and therefore converges to 0.
- A density regularity lemma establishes differentiability of the finite-horizon maximum distribution and uniform convergence of its density derivatives on compact threshold sets.
B.4 Hadamard differentiability of hitting probability in simple examples
The section applies the chain rule to show Hadamard differentiability of hitting-probability functionals in simple location-scale examples. It also treats a normal-model version through an analogous composition.
- For the basic example, the hitting functional is written as φ composed with a map that transforms the distribution using location and scale parameters.
- The transformation map is Hadamard differentiable because it is linear in the distribution and the perturbation limit is uniformly continuous.
- Uniform continuity of the density ensures convergence of the transformed derivatives as the parameter varies.
- The normal-model hitting functional is represented by the same inverse-map composition, with the proof following analogous steps.