Source-linked AI summary
Automatic Pavement Crack Detection Based on Structured Prediction with the Convolutional Neural Network
Zhun Fan, Yuming Wu, Jiewei Lu, Wenji Li
TL;DR
Automated pavement crack detection must handle complicated pavement conditions and severe imbalance between crack and non-crack pixels. The paper uses a CNN to learn crack structure from image patches as a multi-label prediction task, with modified sampling ratios for imbalance. Tested on two public databases, the method outperforms the compared methods and produces results close to manual labels, while label subjectivity remains a limitation.
Problem
Automated crack detection is challenging under complicated pavement conditions, while crack pixels are far fewer than non-crack pixels.
Method
A CNN learns crack structure from raw image patches through multi-label structured prediction, using a modified positive-to-negative sampling ratio for imbalanced data.
Results
The proposed method outperforms the compared methods on CFD and AigleRN and produces crack detections close to manual labels.
Takeaways & Limitations
The results support CNN-based learning from raw images for pavement crack structure prediction across different pavement textures and databases.
Takeaways & Limitations
Detection outputs reflect manual-label subjectivity: thinner crack labels lead to thinner crack outputs.
Abstract
from arXiv · showhide
Automated pavement crack detection is a challenging task that has been researched for decades due to the complicated pavement conditions in real world. In this paper, a supervised method based on deep learning is proposed, which has the capability of dealing with different pavement conditions. Specifically, a convolutional neural network (CNN) is used to learn the structure of the cracks from raw images, without any preprocessing. Small patches are extracted from crack images as inputs to generate a large training database, a CNN is trained and crack detection is modeled as a multi-label classification problem. Typically, crack pixels are much fewer than non-crack pixels. To deal with the problem with severely imbalanced data, a strategy with modifying the ratio of positive to negative samples is proposed. The method is tested on two public databases and compared with five existing methods. Experimental results show that it outperforms the other methods.
I. INTRODUCTION
Automated pavement crack detection is important but difficult because real pavement images contain varied conditions and conventional image-processing methods are noise-sensitive. The paper proposes CNN-based structured prediction that learns crack structure from raw patches and addresses severe class imbalance.
- Automated crack detection supports safer, lower-cost, more efficient, and more objective pavement-condition assessment for maintenance.
- Conventional thresholding, morphology, and edge-based methods rely on photometric or geometric assumptions and can be sensitive to noise.
- Machine-learning approaches improve feature extraction and pattern recognition, but existing methods still have limitations in accurately finding all cracks.
- Earlier deep-learning methods operate at block or pixel level, while some require manual preprocessing features or architectures tied to input image size.
- The proposed CNN extracts patches around individual pixels, predicts crack structure as a multi-label output, aggregates predictions into a probability map, and thresholds it into a binary result.
II. THE METHOD
The method builds patch-based training data from RGB and gray-level pavement databases, labels patches by their center pixels, and modifies sampling to address class imbalance. Raw image values are scaled before CNN training.
- Data Preparation: The method uses CFD RGB images and AigleRN gray-level images, representing pavement data collected in Beijing and France.
- Data Preparation: Each image pixel defines the center of an extracted input patch, with 27 × 27 patches used for the network.
- Data Preparation: A patch is labeled positive when its center pixel is a crack pixel and negative when its center pixel is non-crack.
- Data Preparation: The training database contains examples of negative and positive 27×27 patches, while testing uses all image pixels with symmetric padding at boundaries.
- Data Preparation: Training first extracts all positive samples, then randomly selects negative samples according to a chosen positive-to-negative ratio, generally 1:3.
- Data Preparation: Raw image values are scaled from [0, 255] to [-1, 1] before optimization.
B. Network Architecture
The proposed CNN transforms image patches into structured crack predictions rather than classifying only the center pixel. Its multi-label output represents an s × s centered structure, with nonlinear hidden layers and sigmoid outputs.
- B. Network Architecture: The network comprises four convolutional layers, two max-pooling layers, and three fully connected layers.
- B. Network Architecture: Each input patch produces a prediction of the crack structure centered within that patch.
- B. Network Architecture: The output structure has size s × s, with s = 5 in the illustrated method.
- B. Network Architecture: Flattening the s × s structure into s^2 output neurons models structured prediction as a multi-label classification problem.
- B. Network Architecture: The output units use sigmoid activation, while hidden layers use ReLU to model nonlinearity.
C. Training and Testing
The method trains a multi-label CNN with cross-entropy and regularization, then aggregates overlapping pixel-centered outputs into a normalized probability map during testing.
- Training: Cross entropy trains the non-mutually-exclusive output units in the multi-label classification problem.Each output unit corresponds to a label and is evaluated against its prediction.
- Training: An L2 penalty is added to the cross-entropy loss to penalize large network weights and reduce overfitting.The experiment sets the L2 penalty factor β to 0.0005.
- Training: Dropout randomly sets hidden-neuron values to zero in the first two fully connected layers during training.The dropout ratio is 0.5.
- Training: The experiments use 20,000 iterations for AigleRN and 30,000 iterations for CFD.These correspond to 43 epochs in AigleRN and 13 epochs in CFD.
- Testing: During testing, overlapping patches centered on pixels produce multiple decisions whose outputs are summed and normalized into a probability map.Pixels near image boundaries are handled as an exception to the usual s2 decisions per pixel.
III. RESULTS
The reported setup evaluates a CNN architecture using precision, recall, and F1 score, while allowing small localization differences from manually labeled references.
- CNN architecture: The CNN architecture uses convolutional and max-pooling feature maps followed by fully connected processing.The architecture is illustrated in Fig. 4 and specified in Table II.
- CNN architecture: The network uses 3 × 3 convolutions with stride 1 and zero padding, while max pooling uses stride 2 over a 2 × 2 window.These settings are described for the architecture shown in Fig. 4.
- Evaluation: Precision, recall, and F1 score are used to evaluate crack-detection performance.These metrics are introduced as commonly used in classification problems.
- Evaluation: Evaluation accepts detections within 2 pixels of the manually labeled reference.The tolerance addresses subjective labels and transitional regions between crack and non-crack pixels.
B. Results on CFD
On CFD, the proposed method produces probability and binary crack maps and is reported to handle complex backgrounds and crack topology while outperforming CrackForest.
- Proposed method: The proposed method converts a probability map into a binary output using a decision probability of 0.5.Whiter probability-map pixels indicate greater likelihood of being crack pixels, and few pixels are removed by binarization.
- Proposed method: The output images indicate that the method can handle complex backgrounds and complex crack topology.This conclusion is drawn from the exemplar CFD detections in Fig. 5.
- Comparison: Evaluation results on CFD indicate that the proposed method outperforms CrackForest.CrackForest overestimates crack width, producing high recall but low precision.
- Comparison: The comparison uses identical training and testing databases for CrackForest and the proposed method because both are supervised learning methods.The passage frames the comparison under matched database conditions.
- Comparison: The CFD comparison includes the original image, ground truth, Canny, local thresholding, CrackForest, and the proposed method.The methods are arranged from left to right in Fig. 6.
C. Results on AigleRN
On AigleRN, the proposed method outperforms the compared methods, detecting cracks with better precision and higher recall while producing more complete results.
- The proposed method detects cracks with better precision and higher recall than MPS on AigleRN.Traditional methods are sensitive to noise, while FFA struggles to find continuous cracks and MPS is more effective globally.
IV. OUTPUT STRUCTURE
The method uses CNN-based structured prediction as a multi-label task, predicting a centered crack structure rather than only one pixel. Larger output structures, especially s = 5, improve results and reduce noise sensitivity.
- Structured prediction: Structured prediction models crack detection as multi-label classification, predicting the centered structure from patch information.This incorporates spatial relations between neighboring crack pixels.
- Output structure: The output structure size s is tested at 1, 3, 5, and 7, corresponding to 1, 9, 25, and 49 output units.s = 1 reduces the model to pixel classification.
- Results: Structured prediction, particularly with s = 5, performs better than pixel classification.Probability maps show that pixel classification is more sensitive to noise.
- Interpretation: Predicting all outputs of a patch helps prevent the network from misunderstanding the classification problem.The network is trained to accurately predict the complete output structure.
- Testing procedure: Structured prediction produces s^2 decisions for each pixel, which are summed and globally normalized to smooth detected pixels.This scanning procedure provides multiple decisions for each pixel.
V. RATIO OF POSITIVE TO NEGATIVE SAMPLES
Severe positive–negative imbalance affects training: negative predictions can achieve high accuracy and cause non-crack overestimation. Modifying the sample ratio changes precision–recall trade-offs and has database-specific acceptable ranges.
- Imbalanced samples: Positive-to-negative ratios reach 1:65 in CFD and 1:98.5 in AigleRN, creating severe sample imbalance.Crack pixels are much fewer than non-crack pixels in typical images.
- Imbalanced samples: With a natural 1:65 ratio, predicting every pixel as non-crack yields 0.985 accuracy and encourages negative predictions.This can overestimate non-crack pixels during testing.
- Sample definition: Positive samples are extracted from crack-centered patches, while negative samples are extracted from non-crack-centered patches for ratio-controlled training.The particular output position is not important because the method focuses on crack and non-crack probabilities.
- Experimental setup: The experiment keeps total sample counts constant while modifying the positive-to-negative ratio, with 360,000 CFD samples and 60,000 AigleRN samples.Models are tested on the complete testing datasets under different training conditions.
- Results: Increasing negative samples raises precision and lowers recall because the model becomes more likely to predict non-crack pixels.F1 score varies accordingly.
- Results: Acceptable ratios are 2 ⩽ R ⩽ 5 for CFD and 2 ⩽ R ⩽ 3 for AigleRN.Different ranges are attributed to differences in manually labeled crack widths between the databases.
VI. GENERALIZATION STUDY
The generalization study evaluates cross-database and hybrid training because acquisition systems and pavement conditions vary. Cross-testing exposes database-dependent precision–recall differences, while hybrid training provides a trade-off and supports prediction across roads.
- Experimental design: The study tests generalization across CFD, AigleRN, and a hybrid database to address differing acquisition systems and pavement conditions.The experiments include both cross-database and mixed-data training.
- Experimental design: Cross-database testing uses CFD-to-AigleRN, AigleRN-to-CFD, and hybrid training configurations.Hybrid training combines half of both training databases and tests on the remaining images.
- Cross-testing results: AigleRN-trained models produce very thin cracks with low recall and high precision in cross-testing, while CFD-trained models show the opposite pattern.The observed trade-off depends on the training database.
- Hybrid training: Hybrid data training provides a trade-off between the cross-testing behaviors.The results suggest that more data from different pavement conditions can improve prediction across roads using the same architecture.
VII. CONCLUSION AND PERSPECTIVES
The paper proposes an effective CNN-based pavement crack detector that performs better across pavement textures, learns from raw images without preprocessing, and generalizes across databases. Its outputs are influenced by the thickness of manual crack labels, motivating semi-supervised or unsupervised future work.
- The proposed CNN method shows better performance than state-of-the-art methods when dealing with different pavement textures.
- The network learns from raw images without preprocessing and predicts cracks close to manual labels.
- Training and testing on different databases indicate good model generalization.
- Thinner manual crack labels produce thinner outputs, revealing that detection results depend on human labeling judgments.The authors identify semi-supervised or unsupervised learning as a direction for reducing this limitation.