Fault Detection and Identification using Bayesian Recurrent Neural Networks

Introduction

• Effective fault detection and identification supports maintenance decisions by helping operators detect faults and identify relevant variables for diagnosis.Faults can reduce efficiency and reliability, while accurate and prompt detection and identification can reduce system recovery effort and time.Source: §1 · pp. 1–2
• PCA and DPCA are limited by linear model structure, while static nonlinear extensions do not address process dynamics, motivating methods that capture nonlinear dynamics.Source: §1 · p. 2
• Existing neural-network FDI methods often leave fault identification unresolved, lack uncertainty estimates for decisions, and risk overfitting that weakens generalization across operating conditions.Identifying relevant input variables can be difficult even after detection, and a directly classified fault may have multiple possible causes.Source: §1 · p. 3
• The proposed end-to-end FDI framework uses BRNNs to model joint distributions and dynamics across process variables and estimate uncertainty from both model uncertainty and inherent data noise.The BRNN is implemented with variational dropout (Gal and Ghahramani, 2016a,b).Source: §1 · p. 4
• The authors present this as the first successful application of Bayesian spatio-temporal models, particularly BRNNs, to chemical-manufacturing FDI, addressing nonlinearity, non-Gaussian variables, and strong spatio-temporal correlations.Source: §1 · p. 4

• The BRNN probabilistic framework enables more sensitive and robust fault detection and identification.Source: §1 · p. 4
• Interpretable visualizations help operators categorize fault types, analyze possible propagation paths, and determine root causes using engineering judgment.Source: §1 · p. 4

2.1. Recurrent Neural Networks

• RNNs use recurrent hidden states to capture temporal dependencies and nonlinear system dynamics, but simple BPTT-trained networks can struggle with long-range dependencies because of vanishing gradients.LSTM and GRU gating mechanisms allow information and gradients to flow through the unrolled network with less attenuation.Source: Figure 1 · p. 5, §2.1 · pp. 5–6

2.2. BRNNs

• BRNNs treat model parameters as random variables to produce probabilistic output distributions, but exact posterior inference is infeasible for complex neural networks.The described likelihood assumes a normal distribution for simplicity, with its precision parameter reflecting intrinsic data noise.Source: §2.2 · p. 7

2.3. Variational Dropout as Bayesian Approximation

• Variational dropout approximates the intractable posterior with a simpler parameterized distribution, enabling Monte Carlo prediction without substantial model-architecture changes.Its uncertainty estimates require multiple stochastic forward passes; the paper adopts it for simplicity and generalization capability, while cautioning that RNN recurrence complicates dropout application.Source: §2.3 · p. 8
• Timestep-varying dropout can destabilize RNNs and disrupt sequence modeling; variational dropout addresses these issues and can reduce overfitting by preventing co-adaptations (Gal and Ghahramani, 2016b).Source: §2.3 · p. 8
• Variational dropout uses the same sampled mask on input, recurrent, and output connections throughout a sequence during training and testing, unlike standard dropout.The mask is sampled and frozen for each sequence; the approach follows Gal and Ghahramani (2016b).Source: §2.3 · pp. 8–9, Figure 2 · p. 9
• Repeated test-time forward passes approximate samples from the posterior predictive distribution, enabling estimates of predictive mean, standard deviation, and higher-order statistics.Higher-order statistics can be estimated by moment-matching.Source: §2.3 · p. 9
• Because the forward passes use independent, fixed realizations of the RNN model distribution, they can run concurrently, making variational dropout a candidate for online monitoring.The paper frames the method in chemical process monitoring and notes that it can be extended to other manufacturing industries.Source: §2.3 · pp. 9–10

3. Methodology for FDI

• The framework uses historical NOC data to build a model, assess new observations for departures from NOC, and identify affected variables when a fault is detected.Model training uses NOC data and is unsupervised.Source: §3 · p. 10
• The proposed probabilistic model is a BRNN with variational dropout, intended to characterize NOC and capture nonlinear spatial and temporal signatures in process data.The model produces probabilistic estimates of observation likelihood relative to its inferred posterior distribution.Source: §3 · p. 10
• Observations are treated as faults when their deviation from the model's NOC region exceeds a threshold set to keep false alarms under NOC within a predefined level.The model's likelihood estimates are used to assess deviation from NOC.Source: §3 · pp. 10–11
• At each time step, repeated variational-dropout passes produce Monte Carlo samples that approximate the BRNN's posterior predictive distribution for the next observation.The sampled prediction distributions support assessment of new observations against expected process behavior.Source: Figure 4 · p. 11, §3 · p. 11

3.1. Fault Detection

• The BRNN is trained on historical NOC data to predict the next observation from its current observation and state, then repeated dropout passes approximate that prediction's distribution.At each time step, the true observation is compared with the predictive distribution and fed into the model before the procedure repeats.Source: §3.1 · p. 11
• The evolving predictive distribution creates an adaptive decision boundary that uses past system information, including spatial and temporal correlations.Modeling nonlinear correlations is associated with increased detection sensitivity and robustness through more accurate NOC-pattern modeling.Source: §3.1 · p. 12
• The Gaussian method is faster and simpler but limited to Gaussian predictive distributions, whereas the flexible nonparametric method requires tuning an additional density-estimation parameter.The number of nearest neighbors also affects LDR behavior: small k can cause fluctuations, while very large k can reduce detection sensitivity.Source: §3.1 · p. 12, §3.1.2 · p. 13

3.1.1. Method 1: Squared Mahalanobis distance for Gaussian predictive distributions

• For Gaussian or approximately Gaussian predictions, squared Mahalanobis distance measures deviation using the Monte Carlo sample mean and covariance, with its threshold chosen for a maximum false-alarm rate on validation data.Larger distances indicate observations farther from the predicted mean and a higher likelihood of fault.Source: §3.1.1 · p. 12

3.1.2. Method 2: Local density ratio (LDR) for non-Gaussian predictive distributions

• For non-Gaussian predictive distributions, the local density ratio compares the observation's local density with the averaged local density of its nearest prediction samples.A larger ratio indicates greater distance from the predictive samples; selecting k trades off fluctuations at small values against reduced sensitivity at large values.Source: §3.1.2 · p. 13

3.2. Fault Identification

• Fault identification applies the detection approach independently to each variable, comparing its observation with the corresponding predicted marginal posterior distribution to find abnormal deviations without labeled fault examples.The marginal distributions retain spatial and temporal correlations from past observations, but sacrifice information from the complete joint distribution to obtain variable specificity.Source: §3.2 · pp. 13–14

3.2.1. Method 1: Standard deviation for Gaussian predictive distributions

• Under a Gaussian predictive-distribution assumption, each variable's deviation is measured in standard deviations from its predictive mean, with significant deviations determined using NOC validation thresholds.The deviation's sign indicates whether a variable is higher or lower than expected.Source: §3.2.1 · p. 14

3.2.2. Method 2: LDR for non-Gaussian predictive distributions

• For non-Gaussian predictions, the variable-wise LDR identifies variables with large deviations, which can be selected using a threshold or ranked by deviation magnitude.The statistic uses the corresponding variable's predictive samples and follows the same nearest-neighbor selection considerations as fault detection.Source: §3.2.2 · pp. 14–15

3.2.3. Fault identification plots

• Identification plots track each variable's statistic over time and interpret its status directly, unlike contribution plots that express a variable as a relative component of a global statistic.The plots are visually similar to contribution plots, but their interpretation and analysis are fundamentally different.Source: §3.2.3 · p. 15

3.3. FDI Scheme

• Variational dropout supplies uncertainty-characterizing samples that support both fault detection and identification.M^2 or LDR statistics detect faults, while D_l or LDR_l statistics identify impacted variables for fault location and possible root-cause analysis.Source: §3.3 · p. 15
• Fault identification requires only additional statistics calculated from the same samples, so it adds minimal computation.Source: §3.3 · p. 15
• The scheme separates offline training from online monitoring and selects the detection and identification statistics at design time rather than during operation.The offline-trained model is used during online monitoring.Source: §3.3 · p. 15, Figure 5 · p. 16

4. Case Studies

• The case studies evaluate the BRNN-based FDI method on the benchmark Tennessee Eastman process and a real chemical-plant dataset.The TEP flowsheet uses the second control structure.Source: §4 · p. 16, Figure 6 · p. 17
• PCA and DPCA are comparison methods, each tested with and without dimension reduction; parallel analysis selects the retained PCs.The paper describes these methods as commonly accepted FDI benchmarks.Source: §4 · p. 16
• DPCA is described as limited to linear dynamics and correlations and as scaling poorly with increased temporal memory depth; the authors state BRNN avoids these limitations.DPCA is presented as a contrast because it models spatial and temporal correlations in limited form.Source: §4 · p. 16
• The authors tested recurrent-cell types, activations, network sizes, regularization, dropout, and training settings, selecting the final configuration by validation-set likelihood.Only results for the final BRNN configuration are shown.Source: §4 · pp. 16–17

4.1. Tennessee Eastman Process

• The TEP monitoring system uses 52 variables—41 sensor measurements and 11 manipulated variables—with 480 training and 960 validation NOC samples collected at a 3-minute sampling period.Source: §4.1 · p. 17
• The final BRNN uses one linear RNN layer and a linear output layer; its predictive distribution is approximated as Gaussian, with M² and D_l statistics used for FDI.Despite its simpler structure, it was easier to train and performed better than more complex structures; the authors suggest TEP normal-operation dynamics are well modeled as linear.Source: §4.1 · p. 18
• The study distinguishes controllable, back-to-control, and uncontrollable faults: detection should remain low for controllable faults and be high for the latter two, with this prior-knowledge classification excluded from training.Source: §4.1 · pp. 18–19

4.1.1. Training and validation results on the NOC data

• The BRNN captured the TEP normal-operation pattern: measurements lay within its predictive distribution on training and separate validation data, indicating generalization without overfitting.Prediction results did not differ significantly for more than 100 model samples.Source: §4.1.1 · p. 19

4.1.2. Fault detection results

• At a 5% FAR threshold, BRNN yields close to 5% FDR on controllable faults, while (D)PCA—especially full PCA and DPCA—over-detects them and fails to distinguish them from other cases.The authors attribute this difference to (D)PCA’s inadequate characterization of normal-operation dynamics.Source: §4.1.2 · p. 20
• BRNN reliably detected back-to-control and uncontrollable faults while better resisting controllable faults than (D)PCA, whose sensitivity increased the risk of unwarranted alerts.Reduced-dimensionality PCA variants were more reasonable on controllable faults but missed several back-to-control and uncontrollable faults.Source: §4.1.2 · pp. 20–21, Table 1 · p. 22
• Because BRNN models NOC dynamics, oscillatory faults can include moments that resemble normal operation and evade detection; (D)PCA avoids this ambiguity but cannot assess controllability.This explains BRNN’s slightly lower FDR in those cases.Source: §4.1.2 · pp. 21–22

4.1.3. Controllable fault: Fault 3

• For controllable Fault 3, BRNN correctly characterized the disturbance as NOC, while full PCA and DPCA incorrectly flagged variables, including before the disturbance began.Reduced PCA and DPCA produced similar identification to BRNN but had lower detection rates; full models were oversensitive and had high FARs.Source: §4.1.3 · pp. 23, 25

4.1.4. Back-to-control fault: Fault 5

• For back-to-control Fault 5, BRNN showed process measurements returning to the NOC region while the controller-adjusted XMV(11) remained deviated.After about 200 data points, measurements XMEAS(1)–XMEAS(41) were back within the predictive NOC region, distinguishing recovery from an uncontrollable fault.Source: §4.1.4 · p. 25
• For Fault 5, the BRNN identification plot showed XMEAS(22) deviating first, helping locate the likely root cause as increased condenser cooling-water temperature.The outlet stream’s higher separator cooling-water temperature and increased condenser cooling-water flow supported that inference.Source: §4.1.4 · p. 27
• For Fault 5, r-PCA and r-DPCA miss the consistent XMV(11) deviation, while f-PCA and f-DPCA identify several variables but are oversensitive and nonspecific for direct operator diagnosis.The r-PCA and r-DPCA results indicate lower detection and identification sensitivity than BRNN; the f-PCA and f-DPCA results also flag variables unrelated to the fault.Source: §4.1.4 · pp. 27–28
• For the back-to-control fault, BRNN FDI is highly accurate and robust, and its identification plot helps operators assess affected variables, fault type, and system stage.The method provides more specific information about system state than the comparison methods described.Source: §4.1.4 · p. 28

4.1.5. Uncontrollable fault: Fault 1

• BRNN identifies the persistent negative deviation in XMV(4), a central manipulated variable for total feed flow in Stream 4, that r-PCA and r-DPCA miss.The BRNN plot shows XMV(4) deviating from the NOC region.Source: §4.1.5 · p. 29, Figure 16 · p. 29
• For Fault 1, f-PCA and f-DPCA identify involved variables but also flag unrelated variables, making their results require substantial processing before operators can diagnose the fault.They highlight variables that remain within their normal pattern, such as XMV(11).Source: §4.1.5 · p. 29
• BRNNs accurately and robustly detect and identify uncontrollable faults, with identification plots that support root-cause analysis more precisely than the (D)PCA results.Several (D)PCA methods also detect these faults, but their identification results are less accurate and precise than BRNN's.Source: §4.1.5 · p. 29

4.1.6. Fault propagation path analysis: Fault 6

• BRNN identification statistics support fault-propagation analysis by locating NOC deviations over time with high temporal precision, which can be combined with process expertise.The chronological deviations help reveal the disturbance's start and evolution through the process.Source: §4.1.6 · p. 30
• For Fault 6, sorting variables by when their D_l statistic first exceeds a threshold estimated from the NOC validation set yields a time-ordered identification plot.The plot orders deviations to show the fault's temporal sequence through the process.Source: §4.1.6 · pp. 30–31
• For Fault 6, XMEAS(1) and XMV(3) deviate first, followed by reactor pressure and cooling-system variables; within an hour, the fault affects final-product concentrations of A and C.The propagation sequence is inferred by combining BRNN deviation timing with process knowledge.Source: §4.1.6 · p. 31

4.2. Real Industrial Dataset

• The amine-tower case study uses two months of data from 20 sensor measurements sampled every minute, divided into 35-day training, 17-day validation, and 7- and 14-day tests containing Faults 1 and 2, respectively.Source: §4.2 · pp. 32–33
• The final real-data BRNN uses a sigmoid RNN with one 40-unit recurrent layer; LSTM and GRU variants performed similarly despite higher complexity.Results were also similar across a reasonable range of hyperparameter configurations.Source: §4.2 · p. 32

4.2.1. Fault 1 results

• For Fault 1, BRNN detects the documented foaming event and identifies X1.PV and X6.PV, while later X1.MV deviations are also flagged and verified as unexplained fluctuations.The BRNN results are shown in Figure 20.Source: §4.2.1 · p. 33

4.2.2. Fault 2 results

• For Fault 2, BRNN detects an earlier disturbance around time point 6000 and identifies X14.PV, X15.PV, and X9.PV; inspection of recorded measurements supported the assessment.The deviation was less significant than during the documented fault period; analysis around time point 15000 yielded similar results.Source: §4.2.2 · p. 34, Figure 22 · p. 35
• A real-data chemical-process case study reports higher accuracy, specificity, and robustness for BRNN-based FDI than for (D)PCA-based methods; Figures 22 and 23 present Fault 2 results.Source: Figure 22 · p. 35, Figure 23 · p. 35, §4.2.2 · p. 36

5. Conclusion

• The probabilistic RNN framework uses variational dropout to estimate uncertainty over time, supporting adaptive confidence intervals for fault detection and identification.The conclusion also describes direct identification plots and fault-propagation paths.Source: §5 · p. 36
• The conclusion identifies nonlinear-dynamics fault detection and direct fault identification with interpretable plots and propagation paths as supported BRNN capabilities.The RNN characterizes system dynamics using current and past information.Source: §5 · p. 36
• Across the benchmark TEP and real manufacturing case studies, BRNN provides accurate results that are more specific and directly relevant for fault identification than the compared PCA and DPCA methods.The reported results also support distinguishing controllable, back-to-control, and uncontrollable faults.Source: §5 · p. 36
• The proposed Bayesian spatio-temporal strategy could be adapted to other models, including dynamic process models; variational dropout can be applied without modifying model architecture.The paper presents this as potential for broader industrial time-series applications.Source: §5 · p. 37
• Training data may not cover evolving normal operating-condition modes, so the conclusion identifies online BRNN adaptation as important for reducing false alarms and maintenance costs.Online adaptation is listed as future work.Source: §5 · p. 37