Source-linked AI summary
Pavement Image Datasets: A New Benchmark Dataset to Classify and Densify Pavement Distresses
Hamed Majidifard, Peng Jin, Yaw Adu-Gyamfi, William G. Buttlar
TL;DR
Automated pavement-distress detection is constrained by the limited availability of large labeled ground-truth datasets. This paper introduces PID, pairing wide-view and top-down Google Street View images for classification and density measurement, and reports F1 scores of 0.84 for YOLOv2 and 0.65 for Faster R-CNN.
Problem
Deep-learning pavement-distress models require large ground-truth datasets that are usually unavailable.
Method
PID pairs 7,237 wide-view and top-down images from identical pavement locations, using wide views for classification and top-down views for density calculation.
Results
YOLOv2 achieved an overall F1 score of 0.84, compared with 0.65 for Faster R-CNN.
Takeaways & Limitations
Both models detected distresses across full-sunshine, shadow-containing, car-containing, and different camera-view images.
Abstract
from arXiv · showhide
Automated pavement distresses detection using road images remains a challenging topic in the computer vision research community. Recent developments in deep learning has led to considerable research activity directed towards improving the efficacy of automated pavement distress identification and rating. Deep learning models require a large ground truth data set, which is often not readily available in the case of pavements. In this study, a labeled dataset approach is introduced as a first step towards a more robust, easy-to-deploy pavement condition assessment system. The technique is termed herein as the Pavement Image Dataset (PID) method. The dataset consists of images captured from two camera views of an identical pavement segment, i.e., a wide-view and a top-down view. The wide-view images were used to classify the distresses and to train the deep learning frameworks, while the top-down view images allowed calculation of distress density, which will be used in future studies aimed at automated pavement rating. For the wide view group dataset, 7,237 images were manually annotated and distresses classified into nine categories. Images were extracted using the Google Application Programming Interface (API), selecting street-view images using a python-based code developed for this project. The new dataset was evaluated using two mainstream deep learning frameworks: You Only Look Once (YOLO v2) and Faster Region Convolution Neural Network (Faster R-CNN). Accuracy scores using the F1 index were found to be 0.84 for YOLOv2 and 0.65 for the Faster R-CNN model runs; both quite acceptable considering the convenience of utilizing Google maps images.
1. INTRODUCTION
Automated pavement monitoring needs accurate distress data, but deep-learning models depend on large labeled datasets that are often unavailable. The study introduces PID as a labeled pavement-image resource and evaluates it with two deep-learning frameworks.
- Accurate pavement-condition and distress data support strategic road rehabilitation and maintenance.
- Deep-learning pavement models require large ground-truth databases that are usually unavailable.
- PID was introduced as a dataset for training robust automated pavement-distress characterization models.
- 7,237 images were extracted through Google Street View using Python, hand-annotated with distress bounding boxes, and evaluated with YOLO v2 and Faster R-CNN.
- The dataset supports simultaneous distress classification and density quantification through wide-view and top-down images.
2. PREVIOUS DATASETS
Prior pavement datasets generally used either wide-view or top-down imagery and supported different evaluation goals. The study identifies a gap in comprehensive datasets that combine distress coverage with simultaneous classification and density characterization.
- Wide-view datasets capture larger pavement areas and are useful for distress classification, but may include substantial non-pavement content.
- Top-down images provide more accurate distress views but typically require more sophisticated camera and mounting equipment.
- Prior work included 2D, 3D, and ground-penetrating-radar image datasets for pavement distress detection and characterization.
- Public datasets have supported open-source pavement-evaluation methods and comparisons of model detection accuracy.
- The proposed dataset combines wide-view and top-down images to classify distresses and determine their density, respectively.
2. NEW DATASET
The PID dataset contains paired wide-view and top-down imagery from U.S. pavement sections, with wide-view images annotated for nine distress classes. Its class distributions and image annotations are summarized visually.
- 7,237 images were collected from 22 U.S. pavement sections using Python software and Google API street-view extraction.
- Wide-view images at a -70° pitch supported distress classification, while top-down images at -90° supported more accurate distress quantification.
- The wide-view dataset was hand-annotated for nine pavement distress types and divided into 5,789 training images and 1,448 testing images.
- Figure 1 presents the nine distress classes and examples of annotated wide-view images.
- Reflective, lane longitudinal, sealed longitudinal, and block cracks had the most boundary boxes and images, whereas potholes were scarcest.
- Figure 2 compares the number of boundary boxes and images across distress classes.
3. MODEL TRAINING AND TESTING
The study evaluates YOLO v2 and Faster R-CNN for pavement-distress detection, using transfer learning and model-specific architectures to predict distress locations and classes.
- YOLO v2 Model: YOLO v2 performs one-pass object detection using a convolutional neural network that predicts class probabilities and bounding boxes.Its architecture distributes images across a 13 × 13 grid and predicts five bounding boxes per grid cell.
- Faster R-CNN Model: Faster R-CNN uses a two-stage detection process that combines region selection, feature extraction, classification, and location refinement.It replaces selective search with a Region Proposal Network and classifies features extracted from image segments.
- Transfer learning: Transfer learning initialized both models with pre-trained weights from the Microsoft COCO dataset to improve training speed and performance.COCO contains over 2 million labeled objects across 80 categories and more than 300,000 images.
5. RESULTS
The models were evaluated using overlap-based detection criteria and precision, recall, and F1 measures. YOLO v2 generally outperformed Faster R-CNN, while both models detected distress in varied top-down image conditions.
- Evaluation setup: 1,448 test images were evaluated after training on 5,789 images for 40,000 iterations with a learning rate of 0.01.
- Evaluation criteria: 30% Intersection over Union defined a successful match, while lower overlap was classified as a false positive.A prediction with sufficient overlap but an incorrect class was also treated as a false positive.
- Evaluation metrics: Precision, recall, and F1 score were used to evaluate model accuracy.Precision measures predicted positives that are true positives, recall measures actual positives detected, and F1 combines precision and recall.
- YOLO v2 results: YOLO v2 detections included true positives, false positives, false negatives, and cases where the model detected manually missed annotations.
- Confusion analysis: Reflective and transverse cracks were the most frequently confused classes, while alligator cracking and potholes were also confused in both models.The authors relate alligator-crack and pothole confusion to their visual similarity and progression relationship.
- Model comparison: YOLO v2 achieved an overall F1 score of 0.84 versus 0.65 for Faster R-CNN, with YOLO v2 precision of 0.93 and recall of 0.77.Per-class YOLO v2 F1 scores ranged from 0.95-0.98, compared with 0.8-0.91 for Faster R-CNN.
- Comparison with prior studies: YOLO v2 precision and recall were 0.77 and 0.71 for Maeda et al. and 0.77 and 0.73 for Mandal et al., compared with the proposed model’s F1 score of 0.84.
- Top-down image testing: Both models accurately detected distresses in full-sunshine and shadow-containing top-down images.The comparison included plain top-down, plain wide-view, shadowed top-down, and shadowed wide-view images.
6. CONCLUSIONS
The PID dataset pairs wide-view and top-down street-view images to support distress classification and density calculation. YOLOv2 outperformed Faster R-CNN, while both models detected distresses across varied camera conditions.
- Dataset and applications: 7,237 wide-view images with bounding boxes covered nine pavement distress types, paired with 7,237 top-down images at identical locations.Wide-view images supported classification, while top-down images supported distress-density calculation.
- Model evaluation: 0.84 F1 for YOLOv2 exceeded 0.65 F1 for Faster R-CNN in automatic detection and classification of nine pavement distress types.The reported comparison used F1 scores for model accuracy assessment.
- Model evaluation: Both models accurately detected distresses in full-sunshine, shadow-containing, and car-containing images, including tests on top-down views.The models were trained solely on wide-view images before testing across different camera-view conditions.
- Practical significance: Google street-view images provide broadly available inputs, while expert annotation across common highway distress types supports convenient and cost-effective pavement evaluation.The authors describe the models as robust and flexible for different camera views and pavement-management applications.
7. FUTURE WORK
Future work uses top-down images with a pre-trained U-Net and post-processing to quantify road-crack density. Figure 6 compares original images, U-Net outputs, and overlapped modified outputs.
- Future work: A pre-trained U-Net was applied to top-down pavement images to quantify road-crack density.The network was originally developed for biomedical image segmentation and was repurposed here for pavement imagery.
- Future work: Custom MATLAB post-processing reprocessed U-Net outputs to reduce image noise before density analysis.The code is identified as available in the cited GitHub repository.
- Future work: Figure 6 presents original images, U-Net outputs, and overlapped modified U-Net results from left to right.The three views show the progression from input image to segmentation output and its overlay.
8. AUTHOR CONTRIBUTIONS
The paper attributes study conception, data collection, software, analysis, and manuscript preparation to specified contributors, with all authors reviewing and approving the final manuscript.
- Contributions: Study conception and design were attributed to Hamed Majidifard, Adu-Gyamfi, and William Buttlar.
- Contributions: Hamed Majidifard was credited with data collection and annotation.
- Contributions: Software setup and calibration were attributed to Adu-Gyamfi, Hamed Majidifard, and Peng Jin.
- Contributions: Hamed Majidifard and Adu-Gyamfi handled analysis and interpretation, while manuscript preparation included William Buttlar.
- Contributions: All authors reviewed the results and approved the final manuscript.