PaperlayerView exact revision on arXiv

Source-linked AI summary

MINet: Multi-scale Interactive Network for Real-time Salient Object Detection of Strip Steel Surface Defects

Kunye Shen, Xiaofei Zhou, Zhi Liu

arXiv:2405.16096v1cs.CV

TL;DR

Strip-steel defect detection needs accurate predictions without the large models and slow inference that limit practical use. MINet combines a multi-scale interactive module with a lightweight network, and the authors report competitive accuracy alongside lower computational cost and faster inference than cutting-edge methods.

  • Problem

    Existing strip-steel defect methods face a trade-off between multi-scale feature representation and the model size, computational cost, and inference speed relevant to practical use.

    Source: §I · pp. 1–2

  • Method

    MINet uses an MI module that extracts multi-scale features with DWConv and aggregates them with PWConv in a lightweight encoder-decoder network.

    Source: §I · p. 2

  • Results

    On SD-Saliency-900, MINet has smaller computational cost, better real-time performance, and competitive detection accuracy than cutting-edge methods.

    Source: §V · p. 10

  • Takeaways & Limitations

    The MI-based real-time backbone balances computational efficiency and detection accuracy on the evaluated strip-steel defect task.

    Source: §V · p. 10

Abstract

from arXiv · show

The automated surface defect detection is a fundamental task in industrial production, and the existing saliencybased works overcome the challenging scenes and give promising detection results. However, the cutting-edge efforts often suffer from large parameter size, heavy computational cost, and slow inference speed, which heavily limits the practical applications. To this end, we devise a multi-scale interactive (MI) module, which employs depthwise convolution (DWConv) and pointwise convolution (PWConv) to independently extract and interactively fuse features of different scales, respectively. Particularly, the MI module can provide satisfactory characterization for defect regions with fewer parameters. Embarking on this module, we propose a lightweight Multi-scale Interactive Network (MINet) to conduct real-time salient object detection of strip steel surface defects. Comprehensive experimental results on SD-Saliency-900 dataset, which contains three kinds of strip steel surface defect detection images (i.e., inclusion, patches, and scratches), demonstrate that the proposed MINet presents comparable detection accuracy with the state-of-the-art methods while running at a GPU speed of 721FPS and a CPU speed of 6.3FPS for 368*368 images with only 0.28M parameters. The code is available at https://github.com/Kunye-Shen/MINet.

I. INTRODUCTION

Strip-steel defect detection must balance accuracy with model size and inference speed, while multi-scale features can improve defect representation at added computational cost. MINet embeds multi-scale feature extraction and fusion in a lightweight network for real-time detection.

I. INTRODUCTION

  • Steel defects such as inclusions, patches, and scratches affect product aesthetics and reliability, and detection practicality depends jointly on accuracy, model size, and inference speed.Low contrast, varied defect shapes and scales, and low illumination are among the stated detection challenges.Source: §I · p. 1
  • The design targets the trade-off between preserving multi-scale information and controlling cost: summation may dilute features, while direct concatenation raises computation.The authors propose embedding multi-scale processing into DSConv as a lower-cost alternative to directly concatenating its outputs.Source: §I · pp. 1–2
  • MINet combines a multi-scale interactive module, a five-block real-time backbone, and an encoder-decoder network for strip-steel defect detection.The module uses DWConv to extract features and PWConv to aggregate them; the network reports comparable accuracy with 0.28M parameters and 721 FPS on a GPU.Source: §I · p. 2

A. Surface Defect Detection

  • Surface-defect research includes bounding-box, pixel-wise, and salient-object detection approaches, with prior saliency methods addressing strip-steel defects through multi-resolution, edge-aware, and refinement designs.The authors position their work as real-time salient-object detection for surface defects.Source: A. Surface Defect Detection · pp. 2–3

B. Real-time Neural Network

The section reviews lightweight neural-network designs and motivates DSConv as a basis for efficient models. It then frames multi-scale fusion as a trade-off between retaining information and limiting computation, before outlining the paper’s methodology.

B. Real-time Neural Network

  • Real-time network designs use techniques such as depthwise separable convolution, channel splitting, and context-guided blocks to balance efficiency with task performance.The section identifies DSConv as a building block for the proposed MI module.Source: B. Real-time Neural Network · p. 3

III. METHODOLOGY

  • The methodology section proceeds from DSConv and multi-scale fusion to the MI module, its backbone and MINet, and finally the loss function.The overview also flags the computational cost of DSConv and the fusion trade-offs as preliminary material.Source: §III · p. 3

A. Preliminary

  • DSConv applies DWConv followed by PWConv, while multi-scale fusion faces a cost–information trade-off: concatenation preserves features but is costly, and summation may dilute them.The paper presents its MI module as an approach to that fusion problem.Source: A. Preliminary · p. 3

B. Multi-scale interactive module

The MI module embeds multi-scale feature extraction and fusion within depthwise-separable convolution, then combines the result with the input through a residual structure.

B. Multi-scale interactive module

  • The feature-extraction stage uses four DWConvs with dilation rates 1, 2, 4, and 8 to produce multi-scale features with different receptive fields at the same spatial resolution.PWConv fuses these features.Source: 1) Architecture · p. 4
  • Pointwise convolution first combines features across receptive fields for each channel, while concatenation avoids the dilution associated with summation.A second pointwise convolution fuses the enhanced features across channels.Source: 1) Architecture · p. 4, Figure 3 · p. 4
  • The module adds the fused features to the original input and applies ReLU to produce its output.Depthwise convolution lacks inter-channel interaction, motivating the subsequent pointwise-convolution fusion.Source: 1) Architecture · p. 4

2) Computational cost analysis:

The cost analysis compares the MI module with a variant using four depthwise-separable convolutions and feature concatenation or summation, and concludes MI is more efficient when c is greater than 1.

2) Computational cost analysis:

  • Equation (5) is stated as 4·k2·c·w·h + c·4·w·h + c2·w·h.Source: 2) Computational cost analysis · p. 4
  • 4·(k^2·c·w·h + c^2·w·h) is the stated cost of the variant using four DSConvs with different dilation rates.The variant combines the resulting features by concatenation or element-wise summation.Source: 2) Computational cost analysis · pp. 4–5
  • For c greater than 1, the authors state that the MI module's cost is lower than the variant's, theoretically supporting its more efficient lightweight structure.The passage identifies ablation results as a subsequent evaluation of this analysis.Source: 2) Computational cost analysis · p. 5

C. Network Architecture

The authors target a real-time backbone for surface defect detection because existing efforts use ResNet, whose parameters and floating-point computations increase processing time.

C. Network Architecture

  • Existing surface-defect detection efforts – use a ResNet backbone, whose parameters and floating-point computations increase processing time.The authors aim to build a real-time backbone with their MI module and deploy it as the encoder in an encoder-decoder detection network.Source: C. Network Architecture · p. 5

1) MI-based real-time backbone:

The MI-based backbone uses five encoder stages, with MI modules arranged linearly in its last four stages. MINet then decodes multi-level features into defect predictions at multiple resolutions.

1) MI-based real-time backbone:

  • The five-stage backbone uses a 3×3 convolution in stage 1, followed by four stages containing stride-2 DSConvs and varying numbers of MI modules.The DSConvs downsample spatial resolution and expand feature channels; the MI modules extract features.Source: 1) MI-based real-time backbone · p. 5
  • MI modules are arranged linearly within stages 2–5, with 3, 4, 6, and 3 units, respectively, to support low latency.The backbone produces five levels of contextual features for decoding.Source: 1) MI-based real-time backbone · p. 5

2) MINet architecture:

  • The encoder supplies five levels of multi-scale contextual features to a decoder composed of five decoder blocks.Each block contains two 3×3 DSConvs, with dilation rates 2 and 1 and stride 1.Source: 2) MINet architecture · p. 5
  • Each decoder feature produces a prediction map through dropout, a 1×1 convolution, and sigmoid; bilinear upsampling aligns maps to input resolution.The highest-resolution prediction, S1, is selected as the final defect detection result.Source: 2) MINet architecture · p. 5

D. Loss function

MINet applies deep supervision across decoder stages and combines BCE and SSIM losses. SSIM loss is intended to direct attention toward defect-region boundaries.

D. Loss function

  • Deep supervision evaluates prediction results at each decoding stage.The method adopts this strategy alongside BCE and SSIM losses.Source: D. Loss function · pp. 5–6
  • SSIM loss steers the network to attend more to defect-region boundaries, which is intended to benefit detection accuracy.Source: D. Loss function · pp. 5–6
  • The hybrid loss combines BCE and SSIM losses for the decoder predictions against the ground truth.The text identifies the predictions as the outputs of each decoding stage.Source: D. Loss function · p. 6

IV. EXPERIMENTS

The experiments evaluate MINet on SD-Saliency-900, which covers three strip-steel defect types, and compare it with 19 state-of-the-art methods using accuracy and computational-efficiency criteria.

1) Dataset:

  • SD-Saliency-900 contains 810 training and 900 test images across inclusion, patches, and scratches, with horizontal flipping expanding training to 1,620 images.Each defect type contains 300 images, and the dataset provides pixel-wise annotations.Source: 1) Dataset · p. 6

3) Evaluation Criteria:

  • Accuracy is evaluated with MAE, WF, OR, SM, PFOM, IoU, PR curves, and F-measure curves; efficiency uses parameters, FLOPs, and FPS.The paper describes parameters as a storage-resource indicator, FLOPs as computational complexity, and FPS as inference speed.Source: 3) Evaluation Criteria · p. 6

B. Comparison with state-of-the-art

1) Network comparison:

On SD-Saliency-900, MINet combines competitive accuracy with low computational cost and fast inference, and qualitative comparisons report advantages over lightweight or real-time methods in clutter suppression and boundary detail.

1) Network comparison:

  • 721 FPS, 0.28M parameters, and 0.30G FLOPs accompany competitive accuracy against traditional methods; MINet runs 13.4 times faster than DACNet.On 1024×1024 images, MINet runs at 95 FPS versus DACNet’s 3 FPS; its MAE is 0.0169 versus DACNet’s 0.0118.Source: 1) Network comparison · pp. 7–8
  • Against real-time and lightweight methods, MINet leads on PR and F-measure curves, MAE, WF, OR, SM, and PFOM, while achieving 721 FPS.Its SM is 0.9207 versus CGNet’s 0.9191; it has 0.28M parameters and 0.30G FLOPs, both lower than CGNet’s.Source: 1) Network comparison · p. 7
  • Qualitative comparisons report that MINet better suppresses cluttered backgrounds and preserves boundary details than lightweight or real-time methods in cited examples.The reported boundary comparison is for the seventh example; clutter suppression is noted for the fifth.Source: 1) Network comparison · p. 8, Figure 7 · p. 8

C. Ablation analysis

  • Ablation studies assess the MI-based backbone, MI-module architecture, feature-combination types, and loss function.Source: C. Ablation analysis · p. 8

1) Backbone comparison:

The MI-based real-time backbone balances detection accuracy and computational efficiency, while the MI module’s feature-extraction and fusion choices shape that trade-off.

1) Backbone comparison:

  • 1.8% of VGG16’s parameters and 0.7% of its FLOPs, with SM only 0.0145 lower; MINet also leads SqueezeNet on MAE, WF, OR, SM, and PFOM.The backbone comparison reports similar results for ResNet50 and VGG16.Source: Table III · p. 8, 1) Backbone comparison · p. 8

2) MI module’s architecture:

  • The full MI architecture achieved the best detection accuracy, while removing multi-scale extraction preserved the same computational efficiency.The variants also tested same-rate DWConvs, summation instead of concatenation, and replacement of the PWConv.Source: Table IV · p. 9, 2) MI module’s architecture · p. 9

3) Feature combination types:

  • Concat performed better than MINet but used 4.1 times its parameters and 3.3 times its FLOPs; Sum performed worse than both.The authors interpret the Sum result as evidence that summation can dilute features.Source: Table IV · p. 9, 3) Feature combination types · p. 9

4) Loss function:

On SD-Saliency-900, the paper reports competitive detection accuracy with improved computational efficiency. It also describes broader industrial applications and identifies variable RGB-camera conditions as a challenge.

4) Loss function:

  • SSIM loss improved detection accuracy without extra computational cost, and the proposed loss outperformed four existing loss variants.Removing SSIM worsened performance; the other variants used focal, Lovász-Softmax, importance-aware, and Abhishek losses.Source: 4) Loss function · p. 9

1) Practical implications:

  • MINet is intended primarily for strip-steel defect detection, with possible use in defect classification and inspection; its backbone may extend to object detection and segmentation.The MI module is described as a fundamental module that can be plugged into existing methods.Source: 1) Practical implications · p. 9

2) Challenges:

  • Variable RGB-camera conditions, including uneven illumination, overexposure, and low light, remain a challenge; depth and thermal data are proposed for future work.The authors identify these modalities as a promising direction for broader industrial applications.Source: 2) Challenges · p. 9

V. CONCLUSION

  • On SD-Saliency-900, MINet has much smaller computational cost, better real-time performance, and competitive detection accuracy than cutting-edge methods.The conclusion also describes the MI-based backbone as balancing computational efficiency and detection accuracy against large-scale and real-time backbones.Source: §V · p. 10
Loading 2405.16096v1…