PaperlayerView exact revision on arXiv

Source-linked AI summary

Feature Pyramid and Hierarchical Boosting Network for Pavement Crack Detection

Fan Yang, Lei Zhang, Sijia Yu, Danil Prokhorov, Xue Mei, Haibin Ling

arXiv:1901.06340v2cs.CV

TL;DR

Pavement crack detection is important for road safety but manual inspection is tedious, while skewed samples and difficult backgrounds challenge automatic detection. FPHBN augments HED with feature pyramids and hierarchical sample reweighting, and experiments report improved detection performance, including a 5% ODS gain over HED.

  • Problem

    Manual pavement crack detection is tedious, while highly skewed classes can cause training loss to be dominated by easily classified negative samples.

  • Method

    FPHBN combines HED with a top-down feature pyramid for integrating high-level context into low-level features and hierarchical boosting for reweighting hard samples.

  • Results

    FPHBN improves ODS by 5% relative to HED and is reported to demonstrate superiority and generalizability across crack-detection experiments.

  • Takeaways & Limitations

    The paper contributes a crack-detection network and proposes average intersection over union (AIU) as a complementary evaluation measurement.

  • Takeaways & Limitations

    All methods fail under low illumination when such scenarios are unseen in the training data.

Abstract

from arXiv · show

Pavement crack detection is a critical task for insuring road safety. Manual crack detection is extremely time-consuming. Therefore, an automatic road crack detection method is required to boost this progress. However, it remains a challenging task due to the intensity inhomogeneity of cracks and complexity of the background, e.g., the low contrast with surrounding pavements and possible shadows with similar intensity. Inspired by recent advances of deep learning in computer vision, we propose a novel network architecture, named Feature Pyramid and Hierarchical Boosting Network (FPHBN), for pavement crack detection. The proposed network integrates semantic information to low-level features for crack detection in a feature pyramid way. And, it balances the contribution of both easy and hard samples to loss by nested sample reweighting in a hierarchical way. To demonstrate the superiority and generality of the proposed method, we evaluate the proposed method on five crack datasets and compare it with state-of-the-art crack detection, edge detection, semantic segmentation methods. Extensive experiments show that the proposed method outperforms these state-of-the-art methods in terms of accuracy and generality.

I. INTRODUCTION

Pavement crack detection is important for road safety but remains difficult because manual inspection is tedious and cracks can resemble complex backgrounds. The paper proposes FPHBN, combining feature pyramids, hierarchical boosting, and AIU evaluation for automatic detection.

  • Manual crack detection is tedious and requires domain expertise, motivating automatic pavement crack detection.
  • Traditional and handcrafted-feature methods struggle with intensity-inhomogeneous cracks, complex topology, and low-level background confusion.
  • FPHBN uses a top-down feature pyramid to transfer higher-level context into lower-level feature maps and improve crack-background discrimination.
  • Visual comparisons include FPHBN, HED, RCF, and FCN across crack images and ground truths from five datasets.
  • AIU complements precision and recall by incorporating crack width and measuring overlap across thresholds while reducing annotation bias.
  • Hierarchical boosting reweights samples layer by layer so the network focuses more on hard examples.

II. RELATED WORKS

Related work spans traditional non-deep-learning methods, including wavelet, thresholding, handcrafted-feature, edge-detection, and minimal-path approaches. These methods use image processing, handcrafted descriptors, filtering, classifiers, or structural information to identify cracks.

  • Traditional crack detection methods are defined as approaches based on non-deep-learning techniques.
  • The traditional literature is grouped into wavelet transform, image thresholding, handcrafted feature and classification, edge detection, and minimal path methods.
  • Wavelet methods decompose pavement images into frequency subbands and search multiscale coefficients for crack regions.
  • Thresholding pipelines reduce illumination artifacts, generate crack candidates, and refine them with morphological or graph-based methods.
  • Handcrafted-feature approaches extract descriptors such as HOG or LBP from image patches before classification, often using support vector machines.
  • Other traditional approaches use morphological filters, Sobel edges, noise-removal preprocessing, or random structure forests to exploit crack structure.

5) Minimal path-based methods:

The paper situates FPHBN among patch-based and deep-learning crack detectors, then describes its multi-scale architecture for improving crack representation across feature levels.

  • Related methods: Patch-based deep networks classify cropped regions but are inconvenient and sensitive to patch scale.
  • Feature pyramid: After feature-pyramid integration, lower-level side outputs and the fused result are reported as clearer than corresponding HED outputs.
  • Bottom-up architecture: The bottom-up architecture extracts hierarchical multi-scale features using VGG conv1–conv5 and intervening max-pooling layers.
  • Bottom-up architecture: Lower feature levels provide higher resolution but less context, whereas upper levels provide more context at lower resolution.
  • Feature pyramid: The feature pyramid merges higher-level maps into lower levels through progressive top-down operations, enriching lower-level representations.

C. Feature pyramid

The feature pyramid addresses context gaps in bottom-up features by progressively merging higher-level information into lower layers, while side networks provide deep supervision and fused predictions support training.

  • Feature pyramid: The feature pyramid progressively feeds higher-level feature maps into lower levels through top-down merging operations.
  • Feature pyramid: The resulting feature maps contain richer features at each level, improving lower-level side outputs and fused crack predictions.
  • Feature merging: At the fourth level, conv5 maps are upsampled twice, concatenated with conv4 maps, and reduced by a 1 × 1 convolutional layer.
  • Deep supervision: Each side network performs crack prediction independently as part of deeply supervised learning.
  • Training objective: The network learns a weighted fusion of side predictions and minimizes side-network and fusion losses jointly.

2) Test Phase:

The hierarchical boosting strategy reweights losses across sequential side networks so deeper predictions emphasize difficult pixels for shallower networks during training.

  • Hierarchical boosting: HED’s class-balancing weight addresses positive–negative imbalance but cannot distinguish easy from hard crack-detection samples.
  • Hierarchical boosting: Hierarchical boosting applies sample reweighting layer by layer from the top side network toward the bottom.
  • Hierarchical boosting: The m-th side-network loss is reweighted using the difference between the deeper side network’s prediction and ground truth.
  • Hierarchical boosting: This design uses deeper side-network predictions to weight shallower losses and encourage communication among side networks.

IV. EXPERIMENTS AND RESULTS

The paper describes an implementation using Caffe, pretrained VGG features, Caffe feature-pyramid layers, and a Python sample-reweighting layer, with specified optimization settings and validation checks.

  • Implementation: The method is implemented with Caffe, an open FCN implementation, pretrained VGG conv1–conv5 features, and Caffe Concat and Convolutional layers.
  • Implementation: Sample reweighting is integrated through a Python layer using the Python Caffe interface.
  • Training settings: Training uses minibatch 10, learning rate 1e-8, momentum 0.9, weight decay 0.0002, and 40,000 iterations.
  • Implementation: Feature-pyramid upsampling uses fixed de-convolutional parameters implementing bilinear interpolation rather than learned parameters.
  • Implementation validation: The implemented class-imbalance cross-entropy loss produces the same experimental results as the original implementation.

4) Computation platform:

The study evaluates crack-detection methods across multiple pavement datasets, including the newly collected CRACK500 and pixel-wise annotated GAPs384 data.

  • CRACK500: CRACK500 contains 500 roughly 2,000 × 1,500-pixel cell-phone images with pixel-level binary annotations.The dataset is split into 250 training, 50 validation, and 200 test images.
  • CRACK500: Cropping CRACK500 images into retained crack-containing regions produces 1,896 training, 348 validation, and 1,124 test images.The validation set selects the best model, while the test set and other datasets evaluate generalizability.
  • GAPs384: GAPs contains 1,969 grayscale distress images at 1,920 × 1,080 resolution, but its original bounding-box annotations are insufficient for pixel-wise prediction.The study manually annotates 384 crack-only images as GAPs384 for testing generalization from CRACK500.
  • GAPs384: GAPs384 images are cropped into 640 × 540 regions, retaining 509 crack-containing test regions.Only regions containing more than 1,000 crack pixels are kept.
  • Cracktree200: Cracktree200 contains 206 800 × 600 images with pixel-wise labels and challenges including shadows, occlusions, low contrast, and noise.The dataset includes various crack types and is used for evaluation.
  • CFD: CFD contains 118 480 × 320 images with manually labeled crack contours captured using an iPhone5.The dataset is used to evaluate the model.

5) Aigle-RN & ESAR & LCMS:

The evaluation combines datasets, edge-detection baselines, and crack-specific metrics to assess performance across thresholds and training iterations.

  • Aigle-RN & ESAR & LCMS: AEL combines the small Aigle-RN, ESAR, and LCMS datasets into one evaluation set.Aigle-RN has 38 images, ESAR has 15 fully annotated images, and LCMS has 5 pixel-wise annotated images.
  • Compared methods: HED, RCF, FCN, and CrackForest are trained or configured as comparison methods for crack detection.HED and RCF use CRACK500 training and validation data, FCN replaces its loss with sigmoid cross-entropy, and CrackForest uses default hyperparameters.
  • Evaluation criteria: ODS and OIS are edge-detection criteria computed after NMS and one-pixel thinning, with a maximum prediction–ground-truth matching tolerance of 0.0075.ODS uses a fixed dataset scale, whereas OIS uses the best scale for each image.
  • Evaluation criteria: The evaluation includes AIU curves over training iterations and IU and precision–recall curves over thresholds.The figures compare FPHBN with HED-FP, HED, RCF, and FCN on CRACK500 validation or five datasets.
  • Evaluation criteria: AIU evaluates predicted and ground-truth crack regions across thresholds without NMS or thinning, ranges from 0 to 1, and averages image-level values over a dataset.Higher AIU indicates better performance.

E. Experimental results

On CRACK500, FPHBN is selected after AIU convergence and outperforms the compared methods on validation and test evaluations.

  • Validation results: FPHBN, HED, and RCF are selected at 12,000 iterations, while FCN is selected at 36,000 iterations after AIU convergence.On CRACK500 validation data, FPHBN surpasses RCF, HED, and FCN in AIU.
  • Ablation study: Feature Pyramid raises AIU from 0.541 for HED to 0.553 for HED-FP, and Hierarchical Boosting raises it further to 0.560 for FPHBN.The component study attributes improvement to both modules.
  • Test results: FPHBN has the highest IU across thresholds and the highest precision–recall curve among the compared methods on the CRACK500 test set.CrackForest cannot contribute IU or precision–recall curves because it produces a binary map.
  • Test results: 5%: FPHBN improves over HED, the second-best method, in ODS on CRACK500 test data.The corresponding visual detections are described as much clearer than those of the other methods.

2) Results on GAPs384:

Across GAPs384, Cracktree200, CFD, and AEL, FPHBN generally achieves the strongest reported performance, although GAPs384 remains challenging because of illumination, background, and annotation characteristics.

  • Results on GAPs384: FPHBN achieves the best performance on GAPs384, but gains are smaller because the dataset has non-uniform illumination and similar backgrounds.A sealed crack beside a true crack is misclassified, and all methods have very small AIU values because the ground truth is only one or several pixels wide.
  • Results on CFD: 15.2% and 12.6%: FPHBN improves over HED in ODS and OIS, respectively, on CFD.The method is reported to outperform HED, RCF, and FCN on the evaluation curves.
  • Results on AEL: 4.9% and 20.4%: FPHBN increases ODS and OIS over the second-best method on AEL.Although RCF has the highest IU curve value, FPHBN achieves the best AIU and fewer false positives.

F. Cross dataset generalization

FPHBN shows stronger cross-dataset generalization than state-of-the-art methods, while improving difficult-case detection at additional computational cost.

  • Cross-dataset generalization: FPHBN achieves the best mean ODS and OIS across GAPs384, Cracktree200, CFD, and AEL, surpassing the second-best method by a large margin.These results are reported as evidence of significantly better generalizability than state-of-the-art methods.
  • Speed comparison: FPHBN is slower than HED by approximately 0.086s to 0.249s because the feature pyramid increases computation for each side network.The authors characterize the method as not real time and mention hardware development and model compression as possible speed improvements.
  • Special cases: FPHBN produces clearer detection results than state-of-the-art methods in complex-background cases involving sealed cracks.All algorithms misclassify sealed cracks as cracks, but FPHBN yields a clearer result.
  • Special cases: Under shadow conditions, FPHBN produces fewer false positives than HED, indicating greater robustness to shadows.The paper attributes this improvement to the feature pyramid and hierarchical boosting modules.
  • Special cases: All evaluated methods fail to detect cracks under low illumination because these scenarios are unseen in the training data.The paper suggests that appropriate data augmentation may address this problem to some extent.
  • Proposed method: FPHBN combines semantic enrichment of low-level features with hierarchical sample reweighting to handle hard examples in pavement crack detection.The paper reports extensive experiments demonstrating the method's superiority and generalizability.
Loading 1901.06340v2…