Source-linked AI summary
Segmentation-Based Deep-Learning Approach for Surface-Defect Detection
Domen Tabernik, Samo Šela, Jure Skvarč, Danijel Skočaj
TL;DR
Surface-defect inspection requires flexible methods that can work with limited defective samples and practical annotation effort. This paper proposes a segmentation-based two-stage deep-learning architecture and evaluates it on real industrial surface cracks. The method outperforms related approaches while achieving strong performance with approximately 25–33 defective samples.
Problem
Surface inspection needs flexible deep-learning methods, but practical deployment must address limited defective samples and annotation requirements.
Method
The paper uses a two-stage architecture combining a pixel-wise segmentation network with an image-level decision network.
Results
The proposed method outperformed related methods on KolektorSDD, with one misclassification versus five or more for related methods.
Takeaways & Limitations
The approach achieved strong performance with 33 defective samples and still good performance with 25 samples, supporting its suitability for the studied industrial application.
Takeaways & Limitations
The architecture is limited to tasks that can be framed as surface segmentation with pixel-wise annotations.
Abstract
from arXiv · showhide
Automated surface-anomaly detection using machine learning has become an interesting and promising area of research, with a very high and direct impact on the application domain of visual inspection. Deep-learning methods have become the most suitable approaches for this task. They allow the inspection system to learn to detect the surface anomaly by simply showing it a number of exemplar images. This paper presents a segmentation-based deep-learning architecture that is designed for the detection and segmentation of surface anomalies and is demonstrated on a specific domain of surface-crack detection. The design of the architecture enables the model to be trained using a small number of samples, which is an important requirement for practical applications. The proposed model is compared with the related deep-learning methods, including the state-of-the-art commercial software, showing that the proposed approach outperforms the related methods on the specific domain of surface-crack detection. The large number of experiments also shed light on the required precision of the annotation, the number of required training samples and on the required computational cost. Experiments are performed on a newly created dataset based on a real-world quality control case and demonstrates that the proposed approach is able to learn on a small number of defected surfaces, using only approximately 25-30 defective training samples, instead of hundreds or thousands, which is usually the case in deep-learning applications. This makes the deep-learning method practical for use in industry where the number of available defective samples is limited. The dataset is also made publicly available to encourage the development and evaluation of new methods for surface-defect detection.
Introduction
Surface inspection is labor-intensive, while classical vision methods lack flexibility for rapidly changing products. The paper develops a two-stage deep-learning approach and evaluates its data, annotation, computational, and comparative requirements on real industrial defects.
- Manual surface-quality inspection is time consuming, inefficient, and can limit production capacity.
- Classical machine-vision systems require hand-crafted features and manual adaptation, limiting flexibility across products and defects.
- Deep learning can learn features directly from low-level data and adapt to different products, but its annotation and data requirements remain open questions.
- The proposed two-stage segmentation-and-decision network targets surface-crack detection with small numbers of defective training samples.
- Evaluation on the real-world KolektorSDD compares the approach with related methods, including commercial software and standard segmentation networks.
Proposed approach
The proposed approach frames defect detection as binary image classification using a two-stage network. Its segmentation stage localizes defects while being designed to capture small details in high-resolution images.
- The first stage performs pixel-wise defect segmentation, and the second stage predicts whether an anomaly is present in the whole image.
- The segmentation network uses 11 convolutional layers and three max-pooling layers, with normalization and ReLU following each convolution.
- The architecture targets small defects by combining a large receptive field with preservation of small feature details.
Decision network
The decision network combines segmentation features with the segmentation map to predict image-level anomalies. Separate two-stage training is used, and the network produces both localization and anomaly-probability outputs.
- The decision network receives the final segmentation features concatenated with the single-channel segmentation output map.
- Its layered convolution and down-sampling design captures local and global defect shapes while using segmentation shortcuts to limit unnecessary feature maps.
- The segmentation network was evaluated with mean squared error and cross-entropy losses, while models were randomly initialized rather than pretrained.
- Separate training was preferred because simultaneous training introduced no performance gain and was more difficult to implement.
- The model accepts grayscale images and outputs an 8 × 8-block segmentation probability map plus an image-level anomaly probability.
Segmentation and decision network evaluation
The evaluation uses the real-world KolektorSDD dataset and varies annotation precision, loss, image resolution, and rotation. Detection is evaluated as binary image classification rather than pixel-wise segmentation accuracy.
- Dataset: KolektorSDD contains real images of cracked electrical commutators collected for surface-defect detection.
- Dataset: The dataset includes 399 images, 52 positive images, 347 negative images, and pixel-wise annotation masks.
- Dataset: Four additional annotation types were generated by dilating the original masks to assess the effect of annotation accuracy and labor.
- Experimental design: The experiments vary five annotation types, two segmentation losses, two image sizes, and whether 90° rotation is used.
- Evaluation target: The evaluation measures per-image defect classification because accurate anomaly presence is more important than pixel-wise localization in the industrial setting.
Performance metrics
The evaluation uses product-disjoint 3-fold cross-validation and compares networks with AP, false negatives, and false positives. Training balances defective and non-defective images while using single-image stochastic-gradient updates.
- 3-fold cross-validation keeps images from each physical product in only one fold, preventing product overlap between training and testing.
- Average precision, false negatives, and false positives are the three classification metrics used for network comparison.AP is preferred to AUC because it better captures performance when datasets contain many non-defective samples.
- Single-image stochastic-gradient updates use batch size one because the images are large and GPU memory is limited.The networks were implemented in TensorFlow and trained without momentum.
- Training alternates defective and non-defective images across iterations to maintain balanced exposure during learning.
- 6600 training steps correspond to 100 epochs with 33 defective images per training fold.An epoch ends after every defective image has been observed, while some non-defective images may remain unseen.
Segmentation and decision network
The proposed two-stage network combines segmentation and image-level decision making, with performance depending on loss, resolution, rotation, and annotation design. Cross-entropy, full resolution, no rotation, and medium-to-large dilation perform best overall.
- 99.9% AP, zero FP, and one FN are achieved with cross-entropy, full resolution, no rotation, and annotations dilated with a 5 × 5 kernel.
- Cross-entropy improves average precision by 7 percentage points over mean squared error across settings.
- Reduced image resolution lowers average AP by 5 percentage points overall, although cross-entropy is comparatively less sensitive.For some cross-entropy configurations, reduced resolution performs marginally better by approximately one percent in AP.
- Random 90° image rotations provide no significant performance gain and can reduce performance substantially in some configurations.The largest observed gain is at most one percentage point.
- Medium-to-large annotation dilation rates produce the best overall results, while smaller annotations have only a slight negative impact under cross-entropy.
Contribution of the decision network
The decision network improves image-level defect classification beyond segmentation alone, while coarse annotations retain nearly the performance of finer annotations with less annotation effort.
- Contribution of the decision network: 98.2% AP, zero FP, and four FN are achieved by the best segmentation-only cross-entropy configuration.The dilate=9 annotation gives this best segmentation-only result.
- Contribution of the decision network: The decision network improves cross-entropy AP by 3.6 percentage points to above 98% on average.With MSE, average AP rises from below 90% without the decision network to above 95% with it.
- Contribution of the decision network: The decision network reduces average miss-classifications from 4 to 2 for cross-entropy models.
- Contribution of the decision network: Global defect shape supports image-level classification, whereas pixel-wise segmentation does not require that information.The decision network uses later segmentation features and additional layers to separate noise from relevant features.
- Required precision of the annotation: Finer annotations reach 99.9% AP with one miss-classification, but coarse annotations remain feasible with minimal or no performance loss.Coarse annotation is less time consuming for human annotators.
- Required precision of the annotation: Coarse rotated-box annotations achieve 99.7% AP and two miss-classifications, compared with 98.7% AP and three miss-classifications for big annotations.
Comparison with the state of the art
The study evaluates Cognex ViDi Red and varies annotation, feature size, resolution, and rotation under matched cross-validation conditions. The best commercial configuration reaches 99.0% AP, while smaller features and original resolution are favored.
- Cognex ViDi Red is evaluated in supervised mode as the commercial comparison system for anomaly detection, visual inspection, and segmentation.
- The commercial evaluation varies five annotation types, three feature sizes, two image sizes, and 90° input rotation.
- 99.0% AP with five miss-classifications is obtained using dilate=5 annotations, 20-pixel features, original resolution, and no rotation.The five miss-classifications comprise five false negatives and zero false positives.
- Dilated annotations outperform non-dilated annotations, but dilate=5 and dilate=17 differ by only 0.1 percentage points.
- Small feature sizes consistently outperform larger features, especially on the high-resolution dataset containing many small defects.
- Half-resolution images and random 90° rotations both slightly decrease performance rather than improving it.
Using state-of-the-art segmentation networks
The proposed segmentation-and-decision network is compared with DeepLabv3+, U-Net, commercial software, and other state-of-the-art methods for surface-crack detection. It achieves the strongest reported detection performance, including under a zero-miss requirement.
- Standard segmentation networks: DeepLabv3+ with dilate=9 annotations achieved 98.0% AP among the standard segmentation networks.It produced two false positives and four false negatives at the ideal F-measure.
- Annotation settings: Slightly dilated annotations performed best for the standard networks, while larger dilation kernels produced worse results.The proposed approach and commercial software selected dilate=5, whereas DeepLabv3+ and U-Net selected dilate=9.
- Comparison with state-of-the-art methods: The proposed approach outperformed all state-of-the-art methods in every reported metric.The commercial product ranked second, while DeepLabv3+ slightly outperformed U-Net among the standard segmentation networks.
- Comparison with state-of-the-art methods: One false negative was the proposed network’s only miss-classification at the ideal F-measure, whereas every other method produced at least five.The missed defect was small and was also missed by the other methods.
- Industrial operating conditions: At a 100% recall requirement, the proposed model produced three false positives, or 0.75% of 399 images.The commercial product required seven manual verifications, compared with 68 for DeepLabv3+ and 108 for U-Net.
Sensitivity to the number of training samples
The study evaluates how reducing defective training samples affects the proposed network and comparison methods. The proposed model retains high, stable performance with substantially fewer samples than the alternatives.
- Performance with fewer samples: Over 99% AP and one miss-classification were retained by the proposed network with only 25 defective training samples.With five defective samples, its AP remained around 96%.
- Comparison with commercial software: At 25 defective samples, Cognex ViDi Suite dropped to 97.4% AP, while at five samples it fell slightly below 90%.The proposed network remained above 99% AP at 25 samples and around 96% AP at five.
- Comparison with segmentation networks: U-Net’s average precision varied from 75% to slightly above 90% as training samples decreased.DeepLabv3+ retained fairly good results at 15 samples but fell to 46% AP at 10 and 16% at five.
- Comparison with segmentation networks: DeepLabv3+ performance at 20 and 15 defective samples slightly exceeded its result with all training samples.This indicates sensitivity to particular training examples in that experiment.
- Overall sensitivity: The proposed method retained superior and stable performance as the number of defective training samples decreased.The comparison used the same train/test procedure and 3-fold cross-validation as the preceding experiments.
Computational cost
The proposed approach is evaluated for computational cost against standard segmentation networks and commercial software. It combines better accuracy with faster forward-pass performance than DeepLabv3+ and U-Net.
- Forward-pass efficiency: The proposed method was significantly faster than DeepLabv3+ and U-Net while also achieving better accuracy.Forward-pass time and average precision were measured on a single NVIDIA TITAN X (Pascal) GPU.
- Forward-pass efficiency: The proposed approach was competitive with the commercial software in computational cost.Its efficiency was attributed to a smaller number of parameters.
Discussion and conclusion
The proposed two-stage segmentation-and-decision architecture outperformed related methods on the KolektorSDD crack-detection task while using few defective samples and limited manual inspection. Its scope is tied to segmentation-based quality-control tasks, and comparisons with commercial software are constrained by undisclosed implementation details.
- Approach and evaluation: The two-stage architecture combines pixel-wise defect segmentation with an image-level decision network and was compared with proprietary software and standard deep-learning segmentation methods.The evaluation used the publicly available KolektorSDD benchmark for fractures on an industrial electrical commutator.
- Comparative performance: One miss-classification was recorded for the proposed model, compared with five or more for related methods on KolektorSDD.The authors attribute the result to the two-stage design, larger receptive field, and improved capture of fine defect details.
- Scope and limitations: Commercial-software differences are difficult to assess because its method is undisclosed, although it performed worse at lower resolution and did not match the proposed method at high resolution.The experiments suggest that the commercial software struggles more with fine defect details.
- Data and deployment requirements: 33 defective samples were sufficient for the reported performance, while 25 samples still produced good performance and outperformed related methods in that setting.The study also evaluated detection rate, annotation precision, and computational cost for industrial use.
- Data and deployment requirements: Only three of 399 images required manual inspection to reach a 100% detection rate, corresponding to a 0.75% inspection rate.Large, coarse annotations achieved performance similar to finer annotations, and sometimes performed better.
- Scope and limitations: The approach is limited to tasks that can be framed as surface segmentation with pixel-wise annotations, making it less suitable for some complex 3D quality-control problems.The paper identifies broken- or missing-part detection as an example better addressed by detection methods such as Mask R-CNN.