PaperlayerView exact revision on arXiv

Source-linked AI summary

Mixed supervision for surface-defect detection: from weakly to fully supervised learning

Jakob Božič, Domen Tabernik, Danijel Skočaj

arXiv:2104.06064v3cs.CV

TL;DR

Industrial surface-defect detection requires substantial annotated data, yet defective samples are scarce and pixel-level labels are costly. The paper introduces an end-to-end mixed-supervision architecture with segmentation and classification sub-networks, and evaluates it on four industrial datasets including the new KolektorSDD2. The model outperforms related methods across weakly and fully supervised settings, while adding only 5–10% fully labeled data can approach the performance of using all fully labeled data.

  • Problem

    Industrial defect detection is constrained by scarce defective samples and costly, difficult-to-produce pixel-level annotations.

  • Method

    The paper proposes an end-to-end architecture combining pixel-label-driven segmentation with image-label-driven classification under mixed supervision.

  • Results

    The model outperforms related methods in weakly and fully supervised settings, and mixed supervision with 5–10% fully labeled data often achieves almost the performance of using all fully labeled data.

  • Takeaways & Limitations

    Large collections of weakly labeled data, supplemented by a small fully labeled subset, can provide strong defect-detection performance with reduced annotation effort.

  • Takeaways & Limitations

    The mixed-supervision evaluation includes pixel-level labels only for a subset of anomalous images rather than all training images.

Abstract

from arXiv · show

Deep-learning methods have recently started being employed for addressing surface-defect detection problems in industrial quality control. However, with a large amount of data needed for learning, often requiring high-precision labels, many industrial problems cannot be easily solved, or the cost of the solutions would significantly increase due to the annotation requirements. In this work, we relax heavy requirements of fully supervised learning methods and reduce the need for highly detailed annotations. By proposing a deep-learning architecture, we explore the use of annotations of different details ranging from weak (image-level) labels through mixed supervision to full (pixel-level) annotations on the task of surface-defect detection. The proposed end-to-end architecture is composed of two sub-networks yielding defect segmentation and classification results. The proposed method is evaluated on several datasets for industrial quality inspection: KolektorSDD, DAGM and Severstal Steel Defect. We also present a new dataset termed KolektorSDD2 with over 3000 images containing several types of defects, obtained while addressing a real-world industrial problem. We demonstrate state-of-the-art results on all four datasets. The proposed method outperforms all related approaches in fully supervised settings and also outperforms weakly-supervised methods when only image-level labels are available. We also show that mixed supervision with only a handful of fully annotated samples added to weakly labelled training images can result in performance comparable to the fully supervised model's performance but at a significantly lower annotation cost.

1. Introduction

Industrial surface-defect detection needs deep-learning models but is constrained by scarce defective samples and costly, difficult pixel-level annotation. The paper proposes mixed supervision to combine weak image-level and available pixel-level labels, evaluating the approach across industrial datasets and introducing KolektorSDD2.

  • Motivation: Industrial deep-learning inspection is limited by scarce defective samples and the difficulty, cost, and imprecision of pixel-level labeling.Defective items can be extremely rare, while defect boundaries may be difficult to define accurately.
  • Motivation: Unsupervised and weakly supervised methods reduce annotation costs but generally underperform fully supervised methods for defect detection.Mixed supervision had been used in other computer-vision tasks but not yet considered for industrial surface-defect detection.
  • Contribution: The proposed end-to-end model combines a segmentation sub-network trained with pixel-level labels and a classification sub-network trained with image-level labels.The architecture supports varying label detail and accounts for spatial uncertainty in coarse region-based annotations.
  • Evaluation: The approach was evaluated on DAGM, KolektorSDD, Severstal Steel, and the newly compiled KolektorSDD2 dataset containing over 3000 images.KolektorSDD2 was created from a practical real-world industrial problem and made publicly available.
  • Results: The model outperformed related approaches with weak labels and fully supervised methods, while a few fully annotated images yielded performance comparable to fully supervised models at lower annotation cost.This result is reported for mixed supervision using only a handful of fully annotated samples alongside weakly labeled training images.

2. Related work

Prior work spans fully supervised, unsupervised, weakly supervised, and mixed-supervision approaches for industrial anomaly detection and related vision tasks. The paper distinguishes its contribution by applying mixed supervision to industrial surface-defect detection and analyzing mixtures of weak and full labels.

  • Fully supervised learning: Fully supervised industrial defect-detection research includes classification, segmentation, patch-based processing, fully convolutional networks, and lightweight architectures.Prior systems include VGG16, custom convolutional networks, two-stage segmentation-classification models, U-Net, DeepLabv3, and MobileNet-v2dense.
  • Unsupervised learning: Unsupervised methods learn from defect-free images without annotations using reconstruction, adversarial, self-supervised, or out-of-distribution objectives.Examples include AnoGAN, f-AnoGAN, deep metric learning, and knowledge-transfer approaches.
  • Weakly supervised learning: Weakly supervised methods use image tags, point labels, class activation maps, or other coarse information for localization and detection.Prior industrial examples applied CAM to LED-chip defects and extended it with bounding-box prediction and knowledge distillation.
  • Mixed supervision: Mixed supervision had been explored for semantic segmentation and brain-tumor segmentation by combining fully labeled images with weaker or unlabeled data.These studies targeted segmentation, whereas this paper focuses on image-level industrial anomaly detection and analyzes mixtures of weak and fully supervised learning.

3. Anomaly detection with mixed supervision

The proposed end-to-end model combines segmentation and classification sub-networks to learn from pixel-level and image-level annotations in mixed supervision. It uses joint loss balancing, selective gradient flow, and spatially weighted segmentation losses to accommodate incomplete or uncertain labels.

  • 3. Anomaly detection with mixed supervision: The architecture combines segmentation and classification sub-networks, using pixel-level information for segmentation and image-level information for classification.The network produces a segmentation map and a classification prediction.
  • 3. Anomaly detection with mixed supervision: Mixed supervision requires simultaneous training of both sub-networks so weakly and fully labeled samples can contribute to learning.Segmentation learning is disabled for positive images lacking pixel-level labels but remains possible for defect-free images, whose segmentation targets are all zero.
  • 3. Anomaly detection with mixed supervision: The model combines segmentation and classification cross-entropy losses into one unified loss with weights controlling annotation availability and each sub-network’s contribution.The balancing factor λ and weights γ and δ complement the SGD learning rate and account for differing loss scales.
  • 3. Anomaly detection with mixed supervision: Dynamic loss balancing shifts training from segmentation at the beginning toward classification at the end, because unbalanced learning can produce exploding gradients.The balancing factors are computed as a linear function of the current epoch and total training epochs.
  • 3. Anomaly detection with mixed supervision: Gradient flow from classification layers into the segmentation sub-network is stopped to prevent unstable early segmentation features from negatively affecting segmentation learning.The adjustment is applied at the segmentation-feature connection and the max/average-pooling shortcut.
  • 3. Anomaly detection with mixed supervision: For approximate region labels, the segmentation loss weights central defect pixels more than boundary pixels using normalized distance and a scaling function.The power-function exponent p controls the decrease in pixel importance; p = 1 or p = 2 often performs best, while negative-label weights remain 1.

4. Experimental setup

The evaluation uses four industrial surface-defect datasets, per-image classification metrics, and implementation settings spanning weak, mixed, and fully supervised training.

  • Datasets: Four benchmark datasets are evaluated: DAGM, KolektorSDD, KolektorSDD2, and Severstal Steel defect.KolektorSDD2 is a newly compiled real-world dataset with fine-grained defect masks and over 3000 images.
  • Datasets: DAGM contains ten grayscale surface classes with defects such as scratches and spots, while related methods sometimes report only six classes.Each class is treated as a binary-classification problem.
  • Datasets: KolektorSDD2 contains controlled color images split into 2085 negative and 246 positive training samples, plus 894 negative and 110 positive test samples.Its defects vary from small scratches and spots to larger surface imperfections.
  • Datasets: Severstal Steel contains 12,568 grayscale images across four defect classes, but evaluation uses all negatives and only positive images from the most common defect class.The experiments therefore use a subset of the dataset.
  • Metrics: Performance is evaluated mainly with per-image average precision, while DAGM additionally reports metrics used by related work with dataset-specific class averaging.AP, AUC, F1-measure, and CA are averaged over all ten DAGM classes; another CA and mAcc are averaged over the first six.
  • Implementation: The method disables classification-to-segmentation gradient flow, weights positive pixels, and enables dynamically balanced loss for mixed and fully supervised settings but not weak supervision.The architecture is implemented in PyTorch and trained with stochastic gradient descent without momentum or weight decay.

5. Evaluation results

The proposed method is evaluated across supervision levels by varying pixel-wise annotations while retaining image-level labels, and achieves strong results with few fully annotated samples.

  • Supervision settings: The experiments vary the number of anomalous training images with pixel-wise masks, while image-level labels remain available for all images.Weak supervision uses N = 0, mixed supervision uses 0 < N < Nall, and full supervision uses N = Nall.
  • DAGM: Five pixel-level labels raise average AP to 91.5% and yield a 100% detection rate on eight DAGM classes.The result demonstrates a substantial gain from adding only a small number of fully annotated samples.
  • DAGM: Fifteen pixel-level labels produce a 100% detection rate on all ten DAGM classes while using fewer than one quarter of positive samples with pixel-level labels.The method outperforms other fully supervised approaches in this mixed-supervision setting.
  • Annotation efficiency: On the evaluated dataset in Fig. 7, AP rises from 93.4% with no segmented samples to 99.1% with five, approaching fully supervised performance with fewer than 15% of images annotated.Using all fully labeled samples achieves a 100% detection rate.
  • Ablation study: The ablation study finds best performance when dynamically balanced loss, gradient-flow adjustment, and distance transform are all enabled, reaching 98.74% AP on Severstal Steel.The combined configuration completely solves the DAGM and KolektorSDD datasets in the reported fully supervised experiments.

6. Conclusion

The proposed mixed-supervision model combines weak and full annotations across industrial defect-detection settings, achieving performance competitive with fully supervised learning while reducing annotation effort. Results across three datasets support using abundant weak labels with a small fraction of pixel-level annotations.

  • The unified architecture combines segmentation learned from pixel-level labels with classification learned from image-level labels, enabling weak, full, or mixed supervision.The network supports end-to-end learning and coarse pixel-level labels.
  • The method outperforms existing state-of-the-art anomaly-detection models in both fully supervised and weakly supervised modes.The approach was demonstrated on DAGM, KolektorSDD, and Severstal Steel Defect.
  • A significant performance boost often comes from adding just 5–10% fully labeled data, yielding almost the same performance as using all fully labeled data.The authors report that this can considerably reduce annotation time and cost.
  • Large quantities of weakly labeled data can be more valuable than a 10-fold smaller set of fully labeled samples, without sacrificing performance in some industrial settings.Weak labels can be obtained at no cost in existing production-control contexts.
  • Performance is reported using average precision, false positives, and false negatives across three datasets and fully, mixed, and weakly supervised learning modes.
Loading 2104.06064v3…