PaperlayerView exact revision on arXiv

Source-linked AI summary

Detection and Segmentation of Manufacturing Defects with Convolutional Neural Networks and Transfer Learning

Max Ferguson, Ronay Ak, Yung-Tsun Tina Lee, Kincho H. Law

arXiv:1808.02518v2cs.CV

TL;DR

Manufacturing inspection needs faster, less error-prone ways to identify casting and welding defects. The paper develops a transfer-learned Mask R-CNN that jointly detects and segments X-ray defects, achieving state-of-the-art casting detection and strong welding segmentation results.

  • Problem

    Manual quality control is time-consuming and error-prone, while manufacturing requires stringent quality control for casting and welding products.

  • Method

    The system uses Mask R-CNN for joint object detection and instance segmentation, with transfer learning from large datasets before finetuning on a small casting X-ray dataset.

  • Results

    Joint detection and segmentation achieved mAPbbox 0.957 on GDXray Castings, mAPmask 0.850 on GDXray Welds, and higher box accuracy than detection-only training.

  • Takeaways & Limitations

    The system is accurate and performant enough for real manufacturing use and supports defect detection across casting and welding X-ray datasets.

Abstract

from arXiv · show

Quality control is a fundamental component of many manufacturing processes, especially those involving casting or welding. However, manual quality control procedures are often time-consuming and error-prone. In order to meet the growing demand for high-quality products, the use of intelligent visual inspection systems is becoming essential in production lines. Recently, Convolutional Neural Networks (CNNs) have shown outstanding performance in both image classification and localization tasks. In this article, a system is proposed for the identification of casting defects in X-ray images, based on the Mask Region-based CNN architecture. The proposed defect detection system simultaneously performs defect detection and segmentation on input images, making it suitable for a range of defect detection tasks. It is shown that training the network to simultaneously perform defect detection and defect instance segmentation, results in a higher defect detection accuracy than training on defect detection alone. Transfer learning is leveraged to reduce the training data demands and increase the prediction accuracy of the trained model. More specifically, the model is first trained with two large openly-available image datasets before finetuning on a relatively small metal casting X-ray dataset. The accuracy of the trained model exceeds state-of-the art performance on the GRIMA database of X-ray images (GDXray) Castings dataset and is fast enough to be used in a production setting. The system also performs well on the GDXray Welds dataset. A number of in-depth studies are conducted to explore how transfer learning, multi-task learning, and multi-class learning influence the performance of the trained system.

I. INTRODUCTION

Manufacturing defects can threaten product quality and safety, motivating automated visual inspection. The paper proposes a Mask R-CNN system that jointly detects and segments defects in X-ray images.

  • Intelligent visual inspection is becoming essential as manufacturers seek higher production rates while maintaining stringent quality limits.
  • Casting and welding can introduce defects that may cause catastrophic failure in critical components if they remain undetected.
  • Automated inspection supports consistent, cost-effective quality control through faster rates and quantitative evaluation less affected by human fatigue.
  • Defect detection can be formulated as object detection, semantic segmentation, or the more difficult instance-segmentation task.
  • The proposed Mask R-CNN system simultaneously performs object detection and instance segmentation on the GDXray X-ray dataset.

II. RELATED WORKS

Prior defect-inspection methods include handcrafted, filtering, wavelet, and neural-network approaches. The paper builds on object-detection research by using Mask R-CNN for simultaneous detection and instance segmentation.

  • Traditional methods include background subtraction, matched filtering, wavelet techniques, and manually engineered pixel features.
  • Background subtraction is sensitive to image positioning and random noise, while many frequency-based methods struggle to combine local and global image information.
  • GDXray provides archived experimental data, including Castings and Welds series used to compare defect classifiers and segmentation methods.
  • Deep-learning studies on GDXray Castings reported up to 86.4 % patch classification accuracy after resizing small patches for pretrained networks.
  • Mask R-CNN extends Faster R-CNN by simultaneously performing object detection and instance segmentation, which this work applies to defect detection.

A. Residual Networks

Residual networks address training difficulties in deep neural networks through residual connections. Their intermediate representations can also serve as feature extractors for broader image-processing tasks.

  • Increasing network depth enables more complex features but can make training difficult because of the vanishing-gradient problem.
  • ResNet uses residual connections to overcome many training issues associated with very deep neural networks.
  • Although designed for image classification, ResNet intermediate-layer outputs can provide high-level image representations for other image-processing tasks.

IV. DEFECT DETECTION SYSTEM

The proposed Mask R-CNN-based system combines feature extraction, region proposals, region-based detection, and instance mask prediction for casting defects in X-ray images. Transfer learning and anchor designs address limited data and small, varied defects.

  • The system has four modules: a ResNet-101 feature extractor, region proposal network, region-based detector, and mask prediction network.The modules respectively generate features, propose regions, classify and refine detections, and produce binary masks.
  • A. Feature Extraction: ResNet-101 is selected as the feature-extraction backbone because it previously produced higher GDXray Castings bounding-box accuracy than VGG-16.The backbone has 101 trainable layers and approximately 27 million parameters, motivating feature transfer from ImageNet.
  • B. Region Proposal Network: The RPN takes a feature map and generates rectangular region proposals with scores indicating the likelihood that each region contains an object.A small CNN slides over the feature map and predicts bounding-box coordinates and object likelihoods.
  • B. Region Proposal Network: Anchor boxes vary in scale and aspect ratio so the RPN can evaluate casting defects with different shapes and sizes.Because many GDXray defects are about 20 × 20 pixels, the smallest anchor is 16×16 pixels, with aspect ratios 1:1, 1:2, and 2:1.
  • B. Region Proposal Network: For each anchor, training combines localization and classification losses as a weighted sum averaged over anchors.The localization term uses smooth L1 loss, while the classification term uses cross-entropy; α and β balance them.
  • B. Region Proposal Network: The RPN transfers across domains because it identifies likely object-containing regions without discriminating among object classes.After COCO pretraining, the RPN immediately identifies casting defects among other interesting regions in X-ray images.

C. Region-Based Detector

The region-based detector classifies and refines each proposed region, while the instance segmentation network predicts a class-specific mask for every region of interest.

  • C. Region-Based Detector: The region-based detector classifies casting defects in each proposal and fine-tunes the predicted bounding-box coordinates.It is based on the Faster R-CNN object detection network.
  • C. Region-Based Detector: RoIAlign converts variable-sized cropped regions into fixed-length feature vectors using bilinear interpolation over an H × W grid.The resulting feature representation has spatial dimensions H × W regardless of the input region size.
  • C. Region-Based Detector: The detector outputs class probabilities, including background, and refined bounding-box positions for the predicted object classes.Its architecture contains two convolutional layers and two fully connected layers.
  • Defect Segmentation: The instance segmentation network produces a 28 × 28 × K output containing one binary mask for each of K classes per region of interest.At test time, only the mask corresponding to the detector’s predicted class is used.
  • Defect Segmentation: The predicted floating-point mask is resized to the region-of-interest dimensions and binarized at a threshold of 0.5.During training, the mask loss applies per-pixel sigmoid outputs and binary cross-entropy only to the ground-truth class mask.

V. IMPLEMENTANTION DETAILS AND EXPERIMENTAL RESULTS

The system is primarily trained and evaluated on the GDXray dataset, whose Castings series contains labeled X-ray images of automotive components.

  • The GDXray Castings series contains 2727 X-ray images mainly depicting automotive parts, including aluminum wheels and knuckles.Casting defects are labeled with tight-fitting bounding boxes, and image dimensions range from 256 × 256 pixels to 768 × 572 pixels.

A. Training

Training uses image preprocessing, ImageNet and COCO transfer learning, and staged fine-tuning on GDXray. The output layers are adapted from 80 COCO classes to casting-defect and background classes.

  • Training: Images are scaled so their longest edge is at most 768 pixels, padded to 768 × 768 pixels, and randomly flipped horizontally and vertically.These adjustments account for small casting defects and the limited number of GDXray images.
  • Transfer Learning: The ResNet-101 feature extractor is initialized from ImageNet weights, and the full defect detection system is then trained on COCO.COCO pretraining reduces total training time and improves trained-model accuracy.
  • Fine-Tuning: The COCO-trained output layers are resized to support two classes: Casting Defect and Background.The resized layers are initialized randomly with a zero-mean Gaussian distribution having standard deviation 0.01.
  • Inference: The instance segmentation module predicts masks that are resized to each predicted bounding box during inference.The masks are produced at 28 × 28 resolution before resizing.

B. Inference

The system evaluates region proposals individually using mAP for both bounding-box and segmentation-mask predictions, with masks generated only for the highest-ranked detections.

  • For each image, the system evaluates the top 600 region proposals selected by RPN objectness score.Masks are predicted only for the top 100 bounding boxes from the region-based detector.
  • The models are evaluated with GPU enabled and disabled on an 8-core Intel Xeon E5 desktop with 32 GB RAM and an NVIDIA GTX 1080 Ti GPU.
  • Performance is measured using mean average precision for both bounding-box prediction (mAPbbox) and segmentation-mask prediction (mAPmask).
  • A detection is correct when the overlap between predicted and ground-truth bounding boxes exceeds 0.5 under the IoU criterion.The overlap is computed from the intersection divided by the union of the two boxes.

C. Main Results

The proposed Mask R-CNN system achieves strong casting-defect detection accuracy and outperforms the cited Faster R-CNN baseline, while joint segmentation improves detection but adds computational cost.

  • 0.957 mAPbbox is achieved on casting-defect detection, exceeding previous state-of-the-art performance and outperforming the cited Faster R-CNN system in accuracy.The proposed system is also faster than that Faster R-CNN implementation when evaluated on a GPU, although SSD ResNet-101 remains faster.
  • 0.931 mAPbbox is obtained when the proposed system is trained without its segmentation module.The authors attribute the improvement from joint training to shared image representations between the bounding-box and segmentation modules.
  • Joint object detection and instance segmentation are approximately 12 % slower than object detection alone.Simultaneous inference requires 9.72 Gigabytes of GPU memory, 9 % higher than detection alone.
  • The system makes very few misclassifications on the GDXray Castings test dataset.Reported examples include a false positive on the X-ray machine itself and a bounding-box regression error affecting IoU-based labeling.

VI. Discussion

The discussion examines speed–accuracy tradeoffs, training-data requirements, and augmentation in the proposed defect-detection system, alongside its evaluation context and reported examples.

  • A. Speed / Accuracy Tradeoff: 600 region proposals provide a reported balance between detection speed and accuracy.More proposals reduce missed objects but increase computational demand.
  • B. Data Requirements: Increasing training data from ∼1100 to 2308 images significantly improves both mAPbbox and mAPmask.The discussion indicates that instance segmentation requires more training data than defect detection and may improve with larger datasets.
  • C. Data Augmentation: The study evaluates common image-augmentation techniques for their effects on testing accuracy.Augmentation is applied during training, while original images are used at test time; the supplied passage does not report the resulting values.
  • D. Model Comparison: The reported comparison table evaluates model accuracy and performance against previous state-of-the-art defect-detection results.
  • C. Data Augmentation: Table III reports mAPbbox and mAPmask for augmentation techniques on the GDXray Castings test set.

D. Transfer Learning

The transfer-learning study compares training without pretraining, ImageNet initialization, and ImageNet-plus-COCO pretraining, while also considering representative detection outputs and evaluation relationships.

  • D. Transfer Learning: Transfer learning initializes the ResNet feature extractor with ImageNet weights before subsequent COCO pretraining and fine-tuning for casting defects.The study tests training schemes that vary whether ImageNet and COCO pretraining are used.
  • Error Analysis: Example figures include a casting-machine false positive and a misclassification caused by bounding-box regression error.
  • A. Speed / Accuracy Tradeoff: The proposed system is evaluated by varying region-proposal counts to examine their relationship with detection accuracy and evaluation speed.
  • B. Data Requirements: Training-set size is evaluated through both object-detection mAPbbox and segmentation mAPmask on the test set.
  • D. Transfer Learning: 0.651 mAPbbox is obtained without transfer learning, compared with 0.874 using ImageNet initialization and 0.957 with full ImageNet-plus-COCO pretraining.Across schemes, the models fit the training data closely; transfer learning therefore affects generalization to unseen images rather than training-data fit.

E. Weld defect segmentation with multi-class learning

Multi-class learning extends the system to casting and welding defects, while testing also probes generalization to unseen X-ray objects. Weld segmentation is accurate despite limited data, but joint training does not improve casting detection accuracy.

  • Weld defect segmentation with multi-class learning: 0.850 mAPmask is achieved for welding-defect instance segmentation on the GDXray Welds test dataset.The dataset contains only 88 annotated high-resolution weld images, divided into 704 smaller training and testing images.
  • Weld defect segmentation with multi-class learning: Joint training enables simultaneous identification of casting and welding defects.The system is retrained on both GDXray Castings and GDXray Welds datasets.
  • Weld defect segmentation with multi-class learning: Joint training produces no measurable improvement in casting-defect detection accuracy.
  • Zero-shot generalization: Four of five casting defects are correctly identified in an unseen jet turbine blade X-ray image.The training set contained no turbine blade images, and one detected defect was incorrectly classified as both a casting and welding defect.
  • Overall performance: The system exceeds state-of-the-art GDXray Castings detection performance, reaching 0.957 mAPbbox, and establishes a new instance-segmentation benchmark.The work evaluates transfer learning, dataset augmentation, and multi-task learning through quantitative ablation testing.
  • Practical deployment: The system is accurate and performant enough for a real manufacturing setting, although its training process is complex and computationally expensive.
Loading 1808.02518v2…