PaperlayerView exact revision on arXiv

Source-linked AI summary

Defect-GAN: High-Fidelity Defect Synthesis for Automated Defect Inspection

Gongjie Zhang, Kaiwen Cui, Tzu-Yi Hung, Shijian Lu

arXiv:2103.15158v1cs.CV

TL;DR

Automated defect inspection is limited by scarce and costly defect samples, especially for deep learning. Defect-GAN synthesizes defects from normal samples through defacement and restoration, using controllable, stochastic, layer-based generation; experiments report superior diversity and fidelity and improved inspection-network training.

  • Problem

    Defect inspection lacks sufficient defect samples because they are costly to label, rare, poorly reusable, and difficult to collect across varied backgrounds.

  • Method

    Defect-GAN mimics defacement and restoration on normal samples, combining category-and-location control, stochastic variation, and layer-based composition.

  • Results

    Defect-GAN generates defects with superior fidelity and diversity, and its synthesized samples help train more accurate defect inspection networks.

  • Takeaways & Limitations

    The synthesized defects provide a data-augmentation route for training inspection networks when normal samples are more readily available than defect samples.

  • Takeaways & Limitations

    The generated samples still exhibit a domain gap from original samples, so naively training recognition models on augmented data can cause them to learn distinctions between domains.

Abstract

from arXiv · show

Automated defect inspection is critical for effective and efficient maintenance, repair, and operations in advanced manufacturing. On the other hand, automated defect inspection is often constrained by the lack of defect samples, especially when we adopt deep neural networks for this task. This paper presents Defect-GAN, an automated defect synthesis network that generates realistic and diverse defect samples for training accurate and robust defect inspection networks. Defect-GAN learns through defacement and restoration processes, where the defacement generates defects on normal surface images while the restoration removes defects to generate normal images. It employs a novel compositional layer-based architecture for generating realistic defects within various image backgrounds with different textures and appearances. It can also mimic the stochastic variations of defects and offer flexible control over the locations and categories of the generated defects within the image background. Extensive experiments show that Defect-GAN is capable of synthesizing various defects with superior diversity and fidelity. In addition, the synthesized defect samples demonstrate their effectiveness in training better defect inspection networks.

1. Introduction

Automated defect inspection is important across industrial applications but is constrained by scarce, costly, and poorly reusable defect samples. Defect-GAN addresses this gap by synthesizing controllable, diverse, high-fidelity defects from readily available normal samples.

  • 1. Introduction: Defect inspection remains difficult because labeled defect samples are expensive, time-consuming, rare, and often non-transferable across tasks.These limitations create a bottleneck for designing effective deep neural network inspection models.
  • 1. Introduction: Existing GANs face data, structural, and background challenges when synthesizing complex, irregular defects across varied surfaces.They commonly require large defect datasets, simplify stochastic defect patterns, and reproduce backgrounds similar to their references.
  • 1. Introduction: Defect-GAN simulates defacement and restoration to leverage readily available normal samples for automated defect generation.The approach is designed to mitigate defect insufficiency without relying on large collections of real defect samples.
  • 1. Introduction: Its control mechanisms specify defect categories and locations, while stochastic variations and layer-based composition improve diversity, fidelity, and background preservation.The architecture generates defects over different normal samples while minimally changing their background appearance.
  • 1. Introduction: Extensive experiments show that Defect-GAN samples help train more accurate defect inspection networks.The reported benefit concerns inspection-network accuracy, alongside the synthesized samples’ diversity and fidelity.

2. Related Works

Prior work spans general image synthesis, learning from limited data, and machine-vision defect inspection. Defect-GAN combines GAN-based augmentation with defect-specific synthesis to address limited defect samples and varied inspection backgrounds.

  • Image Synthesis: Image-synthesis research progressed from unconditional GAN generation toward conditional image-to-image applications such as translation, editing, super-resolution, and inpainting.These applications condition synthesis on input images rather than generating images without input constraints.
  • Learning From Limited Data: Limited-data research includes few-shot learning and data augmentation, with GAN-based augmentation synthesizing realistic training samples.The paper positions Defect-GAN as a data-augmentation method that targets both quantity and diversity.
  • Learning From Limited Data: Few-shot learning can use extremely limited examples, but its performance is described as limited and it still requires substantial same-domain samples.The cited example considers one or three samples for an unseen class.
  • Defect Inspection: Machine-vision defect inspection identifies and localizes surface defects across manufacturing, safety inspection, and construction applications.Deep CNN-based methods have achieved remarkable performance, while traditional handcrafted pipelines require specialized expertise and are not robust.
  • Defect Inspection: Prior defect-synthesis methods include manual damage, CAD-based synthesis, and GAN augmentation, but earlier approaches are described as limited to simple cases.Defect-GAN extends this line of work by synthesizing varied defects with greater diversity and fidelity.

3. Methodology

Defect-GAN synthesizes controllable, diverse defects by translating between normal and defective images through defacement and restoration. Its layer-wise composition, stochastic variation, and spatial-categorical controls preserve backgrounds while generating realistic defects for inspection training.

  • Methodology: Defect-GAN assumes abundant normal samples but limited defect samples and uses unpaired image-to-image translation to address this imbalance.The method is designed around the rarity and collection difficulty of defect samples.
  • Methodology: The generator performs two translation cycles, n → d → n̂ and d → n → d̂, while training uses only the n → d → n̂ cycle description because both cycles are identical.n and d denote normal and defect samples; n̂ and d̂ denote restored normal and defect samples.
  • Generator Design: The generator combines encoder-decoder processing with spatial-categorical control, stochastic noise injection, and layer-based composition.An attribute map controls defect category and location through SPADE normalization and decoder blocks, while adaptive Gaussian noise models stochastic defect patterns.
  • Layer-Wise Composition: Defect-GAN composes generated defect or repaint foregrounds with normal backgrounds instead of synthesizing complete images directly.Spatial distribution maps guide both defacement and restoration compositions.
  • Layer-Wise Composition: Layer-wise composition improves realism, diversity, and transferability by preserving background identities and modeling defects within varied contexts.Using normal samples as backgrounds lets the model focus on defects and simulate their interaction with specific appearances and styles.
  • Training Objective: Training combines adversarial, category-classification, reconstruction, spatial cycle-consistency, and region-constraint objectives.The reconstruction term preserves input content, spatial cycle consistency aligns defect and repaint maps, and the region constraint discourages maps covering the whole image.
  • Boosting Defect Inspection Performance: Generated defects are mixed with original data to train ResNet and DenseNet inspection models, while a source classifier with a Gradient Reversal Layer addresses the synthetic–real domain gap.The domain classifier discourages the backbone from learning undesirable distinctions between synthesized and real samples.

4. Experiments

Experiments on CODEBRIM evaluate Defect-GAN for defect synthesis and inspection-data augmentation. The results indicate strong synthesis fidelity, controllability, transferability, and improved inspection performance.

  • Dataset: CODEBRIM provides six mutually non-exclusive defect classes and is presented as a challenging dataset reflecting practical defect-inspection scenarios.The classes are crack, spallation, efflorescence, exposed bars, corrosion, and normal samples.
  • Implementation: Defect-GAN is trained with classification images plus 50,000 additional normal patches, then synthesizes 50,000 defect samples for augmentation.The generated defects are mixed with original training data, while restored normal samples are included to avoid imbalance.
  • Defect Synthesis: Lower FID indicates better synthesis realism, and Defect-GAN achieves significantly better synthesis FID than the compared image-to-image translation methods.Direct synthesis methods generally perform poorly because defect data are scarce and defect patterns are complex and irregular.
  • Defect Synthesis: Ablation results show that SCC enables convergence, while LWC, ANI, and SC further improve synthesis and jointly provide complementary benefits.Without the designed components, the model degrades to StarGAN and fails to converge on this task.
  • Defect Synthesis: Qualitatively, Defect-GAN preserves normal-image appearance and style, transfers learned defects to unseen backgrounds, and produces realistic and diverse samples.StarGAN with SPADE tends to alter background identity and fails to transfer learned defect patterns to novel backgrounds.
  • Defect Inspection: Defect-GAN supports categorical and spatial control, generating different defect types on the same normal image and defects at specified locations while maintaining natural context.Figure 5 illustrates single and multiple target-category conditioning, while Figure 6 varies spatial controls across normal samples.

5. Conclusion

Defect-GAN generates diverse, high-fidelity defects while preserving background appearance and controlling defect variation, category, and location. The synthesized samples can improve defect inspection performance.

  • Defect-GAN mimics defacement and restoration to generate defect samples with superior fidelity and diversity.Its compositional layer-based architecture preserves the style and appearance of provided backgrounds.
  • The method captures stochastic defect variations and flexibly controls generated defect categories and locations.
  • Defect-GAN-generated samples can significantly boost defect inspection network performance.
  • The work was conducted within the Delta-NTU Corporate Lab with funding from Delta Electronics and Singapore’s National Research Foundation.

Normal Sample

The figures illustrate categorical defect control, using normal samples in Row 1 as backgrounds and showing generated defects conditioned on target categories in Rows 2–6.

  • Rows 2–6 show defect samples generated for target categories, while Row 1 contains the corresponding normal samples.
  • The same categorical-control illustration is presented for normal samples paired with generated defect categories.
Loading 2103.15158v1…