Source-linked AI summary
Deep Vision in Smart Manufacturing: MODERN Framework for Intelligent Quality Monitoring and Diagnosis
Yicheng Kang, Yuling Jiao, Xin Geng, Mahesh Nagarajan
TL;DR
Applying the framework can be challenging when there are relatively few training images and when charting-statistic distributions are nonstandard. The paper proposes transfer monitoring and EWMA-based monitoring, reporting faster convergence than retraining and guidance for monitoring and control decisions.
Problem
Deep learning application is challenged by relatively small sets of training images and nonstandard distributions of charting statistics.
Method
The paper proposes transfer monitoring for limited training data and uses an EWMA to capture accumulated out-of-control evidence.
Results
Transfer monitoring achieves a much faster convergence rate than retraining.
Takeaways & Limitations
The framework provides guidance for decisions that may improve monitoring and control in smart manufacturing.
Takeaways & Limitations
In a new setting, the original control limit may become invalid, requiring it to be recomputed.
Abstract
from arXiv · showhide
Smart manufacturing processes are often installed with a large number of sensors, imaging devices and computers, which not only enable instant communication across various modules of a production system but also aid in intelligent manufacturing management. In this paper, we introduce MODERN, a deep learning framework for quality monitoring and fault isolation, which integrates these enhanced capabilities into the practice of industrial quality control. Using the architecture of an inception residual neural network, we develop a control chart that monitors the likelihood of a product containing defects. We also propose a faulty region estimator that identifies the defective area using transfer learning. To extend our framework to cases where there are not sufficient training data, we suggest a transfer monitoring technique that requires only a small sample size and a hypothesis testing approach for quantitatively assessing the applicability of our method. Theoretically, we establish the minimax optimal convergence rate for both our defect likelihood estimation and fault diagnosis. Our results lead to a seemingly counter-intuitive managerial implication - it may not always be in a manufacturer's best interests to keep upgrading its monitoring equipment regardless of the cost. Empirically, we demonstrate the superior performance of our method in comparison with a state-of-the-art approach using both simulated experiments and real data.
Introduction
MODERN is a general deep-learning framework for image-based statistical process control that combines real-time quality monitoring with post-signal fault diagnosis. It also extends monitoring to limited-data settings and establishes theoretical and managerial guidance for deployment.
- Problem and scope: The framework monitors persistent process shifts rather than inspecting individual products for defects.Its output is an image-based SPC control chart, not a product-quality inspection technique.
- Practical contribution: MODERN provides an implementable framework for image-based SPC quality monitoring and fault diagnosis in smart manufacturing.It targets both quality monitoring and accurate post-signal isolation of faulty regions.
- Quality monitoring: MODERN-Chart uses an EWMA control chart to signal abnormality in real time, detecting small but persistent manufacturing-process shifts.The chart is designed for timely signal delivery and is reported to perform better than the MF approach in that respect.
- Fault diagnosis: MODERN-Diagnosis locates defective areas after an out-of-control signal and can be calibrated using a relatively small defective-image dataset.The faulty-region estimator is reported to perform well in locating defects.
- Transfer monitoring: Transfer monitoring adapts the control limit to a new manufacturing setting with significantly fewer samples than retraining the neural network.Its convergence rate is reported to be much faster than retraining, addressing the sample-size challenge in image-based quality control.
- Theoretical contribution: Theoretical results establish minimax asymptotic optimality for defect-likelihood estimation and post-signal fault diagnosis.The estimation error is decomposed into CNN convergence and image-mislabeling terms, with effectiveness reduced when the mislabeling bound is exceeded.
- Managerial implications: Equipment upgrades are not universally preferable: manufacturers should weigh image resolution against labeling quality and the total training-sample regime.Higher-resolution images are warranted when the total sample size exceeds the relevant asymptotic bound because the method otherwise becomes less effective.
2. Model Descriptions
The model uses deep convolutional neural networks to learn representative image features for quality-related prediction. Its architecture combines convolutional, activation, pooling, and fully connected layers, with inception-style processing supporting multiple feature scales.
- Deep CNNs learn representative image features automatically, avoiding manually defined features that are time-consuming, domain-specific, and often non-generalizable.
- MODERN uses convolutional, activation, pooling, and fully connected layers as its four transformation-layer types.
- Convolutional Layer: Convolutional layers apply shared kernels locally, making feature detection translation invariant while reducing parameter counts through parameter sharing.
- Activation Layer: Activation layers introduce nonlinearity, and MODERN uses ReLU to suppress locations without the corresponding feature and retain signal strength and location.
- Pooling Layer: Pooling uses maximum or average values within neighborhoods to downsize feature maps while preserving detected features.
- Fully Connected Layer: Fully connected layers map flattened image features to target variables using units connected to the entire subsequent layer.
3. The Proposed MODERN Framework
MODERN trains an inception-residual CNN to estimate defect probabilities, monitor upward shifts with MODERN-Chart, and diagnose faulty regions. It also includes augmentation-based control-limit selection and transfer procedures for broader industrial applicability.
- Training of MODERN-Net: MODERN-Net is trained on normal and defective images by minimizing cross-entropy loss with stochastic gradient descent.
- Training of MODERN-Net: The trained network estimates the probability that an input product image contains defects, producing bp as the key output for MODERN-Chart.
- Quality Monitoring by MODERN-Chart: MODERN-Chart applies a recursive EWMA statistic to accumulate upward deviations of defect probabilities from the in-control mean.
- Quality Monitoring by MODERN-Chart: The chart resets its statistic to zero after negative updates, which the authors state helps it respond quickly to quality degradation.
- Control-Limit Selection: Augmented Bootstrap selects the control limit for a prespecified ARL0 by expanding in-control images through rotations and flips.
- Fault Diagnosis and Transfer Monitoring: MODERN-Diagnosis estimates faulty regions, while transfer monitoring and an applicability test extend the framework to settings with limited training data.
4. Asymptotic Properties
MODERN has theoretical guarantees for defect-likelihood estimation, faulty-region diagnosis, and transfer monitoring. The results also characterize how sample size, image resolution, and label quality affect performance and equipment decisions.
- Theoretical guarantees: MODERN establishes minimax asymptotics for defect-likelihood estimation and fault diagnosis, while the AB transfer-monitoring algorithm achieves a √n convergence rate.The framework also establishes consistency for both neural networks and identifies convergence rates for their estimators.
- Defect-likelihood estimation: The defect-likelihood network estimates out-of-control likelihood increasingly accurately as training data grow and achieves the minimax optimal rate up to a logarithmic factor.The network approximates the logistic probability of an image being out of control and uses a bounded probability output.
- Fault diagnosis: The faulty-region estimator is asymptotically consistent, with convergence results applying to its weighted L1 loss and region estimates.The estimator approximates the functional relationship between an out-of-control image and its defective-region representation.
- Resolution and rates: Both theorem-derived convergence rates slow as image dimension, equivalent to resolution, increases.The theoretical discussion links image dimension to the number of training-image features that the model must process.
- Transfer monitoring: Transfer monitoring can reduce the required new-setting sample size because its AB algorithm converges at n^-1/2, faster than the Theorem 1 rate for high-dimensional images.The method first uses a hypothesis test to assess applicability, then uses in-control samples in the new setting.
5. Numerical Studies
Numerical studies evaluate MODERN across simulated resolution settings, the DAGM benchmark, simulated comparisons with Markov Fields, and two manufacturing applications. Results show resolution-dependent performance, accurate fault highlighting, timely signaling, and advantages over the MF approach in several settings.
- Study design: The studies evaluate MODERN-Net, MODERN-Chart, and MODERN-Diagnosis across simulated data, DAGM, and real manufacturing settings.The real applications concern electric commutator and leather manufacturing.
- Impact of image resolution: Higher image resolution can reduce classification performance in finite-sample settings, especially when the out-of-control signal is weak.Low-resolution networks detect stronger shifts well, whereas high-resolution networks do not show the same behavior, consistent with the theoretical findings.
- DAGM benchmark: DAGM provides a challenging benchmark because some defects are difficult to discern and several textures violate Markov Field locality and stationarity assumptions.The dataset is used to evaluate quality monitoring and diagnosis.
- DAGM benchmark: Validation performance stabilizes after 120 epochs, so training is stopped at the 150-th epoch.Validation tracks sensitivity, specificity, and their average while monitoring underfitting and overfitting.
- DAGM benchmark: MODERN-Chart usually signals at the change point, while MODERN-Diagnosis identifies defective areas sufficiently for practical use.Estimated fault ellipses may not fully overlap true regions, but the defective areas are accurately highlighted.
- Manufacturing applications: In manufacturing applications, MODERN signals at the exact commutator change point and correctly indicates persistent shifts in the leather setting.The MF chart produces false signals and can return to the in-control state after a shift.
6. Concluding Remarks and Possible Generalizations
The paper concludes that MODERN combines deep-learning monitoring, faulty-region diagnosis, transfer monitoring, and applicability testing. Its results support fast signaling and effective defect localization, while suggesting that equipment upgrades should depend on image-label fidelity and that real-world randomized evaluation remains needed.
- Contributions: MODERN combines an inception residual neural-network chart for fast abnormality signaling with a faulty-region estimator for post-signal diagnosis.The estimator identifies defective areas in post-signal analysis.
- Transfer monitoring: Transfer monitoring extends MODERN to settings with insufficient image data by requiring only a small sample size and adding an applicability hypothesis test.The test quantitatively assesses whether the pretrained method can be used in a new setting.
- Theory: Theoretical analysis establishes optimal convergence rates for defect-likelihood estimation and fault isolation.These results concern both principal estimation and diagnosis tasks.
- Managerial implication: Monitoring-equipment upgrades may not improve quality control when image labels lack sufficient fidelity, so upgrade decisions should consider label quality.The paper states that this implication applies to both monitoring and diagnosis.
- Applications: The framework is demonstrated in electric commutator and leather manufacturing and is proposed as potentially useful in food, farming, and environmental monitoring.The paper describes possible applications including automated food inspection, crop surveillance, and monitoring environmental changes.
- Future evaluation: Randomized controlled experiments with industry partners are identified as an important next step for evaluating MODERN in real settings.The proposed studies aim to evaluate efficacy and gain additional evidence with industry partners.
Appendix A: Proofs of Theorems 1, 2, 3 and 4
The appendix proves the theoretical results by characterizing the population cross-entropy minimizer and bounding neural-network estimation error. It uses convexity, first-order optimality, and complexity-based arguments to establish the stated results.
- Population minimizer: The population cross-entropy criterion is shown to be convex, and setting its first variation to zero gives D∗ = f∗ almost surely.This identifies the population minimizer used in the theoretical analysis.
- Neural-network estimation: Theorem 1 estimates f∗ nonparametrically with deep ReLU neural networks by minimizing empirical cross-entropy over a bounded network class.The network class is indexed by depth, width, and a bound on the function norm.
- Error analysis: The proof bounds the nonparametric estimation error in L2(ν) using the population and empirical loss structure around the neural-network estimator.The appendix introduces the relevant error quantity and analyzes the loss landscape near the estimator.
Proof of (6).
The proof decomposes the estimation-error bound into empirical-process and approximation components. It then controls the stochastic term using Lipschitzness, independent copies, Rademacher variables, and Gaussian complexity.
- Error decomposition: The proof decomposes the excess-loss expression into differences involving the empirical estimator, a comparison network, and the population minimizer.The decomposition uses the definitions of the empirical and comparison functions.
- Empirical-process control: The sample loss is 1-Lipschitz in the network function, enabling concentration and complexity-based control of empirical deviations.This property is stated for b(D,Z) on D.
- Empirical-process control: The argument introduces independent copies and Rademacher variables, then invokes Gaussian complexity of the network class to upper-bound the stochastic term.These tools provide the empirical-process bounds used in the proof.
Proof of (9).
The proof bounds the relevant network complexity by relating Rademacher and Gaussian complexities. It uses Jensen’s inequality, distributional symmetry, Lipschitz contraction, and standard complexity relationships.
- The argument first bounds Rademacher complexity using Jensen’s inequality and symmetry of the compared terms.
- Lipschitz contraction then controls the transformed class complexity.
- The proof next bounds Gaussian complexity and converts between Gaussian and Rademacher complexity using standard relationships.
- The resulting bound contains network-size, logarithmic, and exponential terms involving N and D W.
Proof of (10).
The proof of (10) decomposes estimation error into empirical-process and approximation components, then controls both with ReLU-network complexity and approximation results. It also derives a noisy-label bound and discusses a slower convergence rate caused by the L1 loss.
- ReLU approximation lemmas provide networks with depth O(12D + C1,d) and width O(C2,dW) for Lipschitz target functions.
- The estimator’s prediction error is bounded by empirical-process variation over the network class plus approximation bias.
- Choosing D W = O(N d/(2d+4)) yields the displayed rate after combining the empirical-process and approximation bounds.
- The convergence rate for G is slower than the minimax-optimal rate because of the technical difficulty introduced by the L1 loss.
- The monitoring proof uses EWMA transformations, a run-length regularity condition, and bootstrap sampling from the in-control empirical distribution.
- The Dvoretzky–Kiefer–Wolfowitz theorem gives √nIC ∥FnIC − F0∥∞ = Op(1), completing the empirical-distribution approximation argument.
B.1. Details for Section 5.2
The benchmark examples show that some DAGM surface defects are difficult for human observers and that background textures may violate assumptions required by the Markov Field method. Training curves determine separate stopping epochs for the two MODERN-Net networks.
- Some DAGM surface defects are difficult for human eyes to discern.
- Several background textures contain random patterns and do not satisfy the Markov Field method’s locality and stationarity assumptions.
- G(·;U) stabilizes from epoch 250, so training stops at epoch 275.
B.2. Details for Section 5.3
The AR simulations evaluate MODERN-Net’s applicability to images outside its training distribution and compare its fault isolation with the MF method. MODERN separates in-control and out-of-control images and localizes both simulated fault types more accurately than MF in the reported example.
- The simulated image set includes an in-control image and two out-of-control image types generated by the AR model.
- The applicability test uses 500 in-control images and 500 images of each of two out-of-control fault types.
- 415.13 and 464.59 are the t-test statistics for type-1 and type-2 faults, respectively, with both p-values effectively 0.
- MODERN-Net distinguishes in-control and out-of-control AR images well despite not being trained on those images.
- The fault-isolation comparison reports results for the median SDSC value across repeated simulations.
- MODERN-Net accurately locates both fault regions, whereas MF detects only the type-2 fault and misses defective pixels for type-1.
B.3. Details for Section 5.4
The section illustrates electric commutator images and tests whether a benchmark-trained network applies to the manufacturing setting. The test and visualization indicate that the network separates the single OC image from IC images.
- Figure 13 presents one IC image and the only OC image from electric commutator manufacturing.
- The trained network can separate the OC image from the other images quite well.Figure 14 provides a direct visualization of this separation.
B.4. Comparison with AlexNet
The section benchmarks MODERN-Net against AlexNet using the same DAGM datasets and aligned training procedures. MODERN-Net performs stably across 10 image classes, while AlexNet varies substantially and performs poorly on OC classification.
- MODERN-Net uses an Inception ResNet architecture, while the benchmark comparison uses AlexNet.Other CNN architectures can also be used in the MODERN framework without much difficulty.
- AlexNet comprises convolution layers, ReLU activations, and pooling operations.
- AlexNet has 33.4 million parameters compared with 23.4 million for MODERN-Net.
- MODERN-Net performed stably across the 10 image classes, whereas AlexNet's performance varied substantially.The comparison uses the same DAGM training and validation datasets.
- AlexNet did not perform well in classifying OC.