PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Machine Learning Approach to Develop a New Asphalt Pavement Condition Index

Hamed Majidifard, Yaw Adu-Gyamfi, William G. Buttlar

arXiv:2004.13314v1stat.MLcs.LGstat.CO

TL;DR

Automated pavement distress assessment remains difficult because robust deep-learning models require large labeled datasets and image conditions such as shadows and poor lighting complicate detection. The paper uses a 7,237-image Google Street View dataset with nine distress classes, combining YOLO classification, U-Net quantification, and hybrid pavement-condition models. It develops indices that rate pavement images by distress type and severity and compares predicted conditions with PASER ratings, while noting limitations in that comparison.

  • Problem

    Automated pavement distress assessment needs large labeled datasets, while shadows, poor lighting, and low contrast challenge reliable detection.

  • Method

    The study combines YOLO distress classification, U-Net segmentation and density quantification, and a hybrid model to develop pavement condition indices.

  • Results

    The developed models produced pavement condition prediction indices from detected distresses, with high coefficients of determination across training, validation, and testing datasets.

  • Takeaways & Limitations

    The resulting tool rates pavement images by distress type and severity using Google Street View imagery and automated deep-learning outputs.

  • Takeaways & Limitations

    Predicted PASER values fluctuated more than measured values, and two cases swapped ranking because measured PASER ratings are approximate and qualitative.

Abstract

from arXiv · show

Automated pavement distress detection via road images is still a challenging issue among pavement researchers and computer-vision community. In recent years, advancement in deep learning has enabled researchers to develop robust tools for analyzing pavement images at unprecedented accuracies. Nevertheless, deep learning models necessitate a big ground truth dataset, which is often not readily accessible for pavement field. In this study, we reviewed our previous study, which a labeled pavement dataset was presented as the first step towards a more robust, easy-to-deploy pavement condition assessment system. In total, 7237 google street-view images were extracted, manually annotated for classification (nine categories of distress classes). Afterward, YOLO (you look only once) deep learning framework was implemented to train the model using the labeled dataset. In the current study, a U-net based model is developed to quantify the severity of the distresses, and finally, a hybrid model is developed by integrating the YOLO and U-net model to classify the distresses and quantify their severity simultaneously. Various pavement condition indices are developed by implementing various machine learning algorithms using the YOLO deep learning framework for distress classification and U-net for segmentation and distress densification. The output of the distress classification and segmentation models are used to develop a comprehensive pavement condition tool which rates each pavement image according to the type and severity of distress extracted.

1. Introduction

The study addresses the need for low-cost, automated pavement distress assessment by combining deep learning with a labeled Google Street View image dataset. It extends distress classification toward segmentation, severity quantification, and pavement condition indexing.

  • Motivation: Automated pavement distress detection is important for pavement management but current human-dependent surveys are expensive, inefficient, and unsafe.The study frames automation as a route toward more cost-effective and consistent pavement-preservation decisions.
  • Motivation: Deep learning has improved image and video analysis, making it an attractive approach for automated pavement distress detection.Existing approaches include conventional image-processing methods and deep architectures such as convolutional neural networks.
  • Dataset and gap: Training robust pavement models requires varied, high-quality labeled data, which is not always available.The cited Pavement Image Dataset contains 7,237 images from 22 pavement sections and covers nine distress classes.
  • Study contribution: The proposed study adds U-Net-based distress-density quantification to YOLO classification and develops pavement condition indices from these model outputs.The work combines classification, segmentation, and distress densification to assess both distress type and severity.
  • Study contribution: The study introduces a dataset annotated for simultaneous distress classification and densification, using Google Street View images with varied camera views.It also reports a segmentation model designed for challenging scenes containing shadows and non-crack-like objects.

2. Background

Pavement condition assessment has relied on inspection-heavy indices and distress models that remain limited by manual measurements, difficult image conditions, and restricted generalization. The reviewed deep-learning work uses labeled pavement imagery with YOLO-based classification and supports later integration with segmentation and condition-rating models.

  • Pavement condition indices: PCI combines distress type, severity, and extent into a quantitative value, whereas PASER, PSI, and PSR are qualitative pavement indices.PCI ranges from 100 for new pavement to 0 for the worst possible condition, while PASER rates conditions from 1–10.
  • Limitations of traditional approaches: Previous pavement condition models still depended on inspectors to measure distress area, length, and severity before computing a condition rate.This dependence motivated the need for a fully automated distress detection model.
  • Computer-vision approaches: Image segmentation extracts object boundaries and can reveal distress severity, while classification groups image objects into categories but cannot measure severity.Thresholding and edge detection are identified as primary segmentation approaches, though pavement imagery presents multiscale, shadow, and illumination challenges.
  • Computer-vision approaches: Existing crack models often perform accurately only with discernible cracks, uniform illumination, or camera views matching those used during training.Fine-grained distress classification has also received limited attention, and prior studies did not jointly classify distress types and quantify distress density.
  • Deep-learning pavement assessment: The study combines distress classification, segmentation, and density information to develop pavement condition indices, addressing inconsistencies observed in qualitative PASER ratings.The proposed workflow includes YOLO classification, U-Net-based quantification, and machine-learning prediction models using distress information.

3. Methodology of Developing Pavement Condition Prediction Models

The methodology combines PASER data, Google Street View images, road segmentation, YOLO distress classification, and U-Net distress quantification to develop pavement-condition prediction models.

  • 71 Missouri pavement sections were selected, and PASER values were checked and corrected by pavement experts.
  • An average of 83 Google Map images per section was extracted at corresponding GPS coordinates using developed Python software.
  • Road segmentation removed shoulders before revised images were processed by YOLO for distress classification.
  • A U-Net model quantified distress, and a hybrid YOLO–U-Net model combined classification with quantification.
  • Figure 6 presents the primary PASER prediction models developed in the study.

3.1. Main Road Segmentation

The study isolates the main road from the shoulder before distress analysis because shoulder materials and distress mechanisms differ from those of the main road.

  • Shoulder distress information was excluded from pavement condition index calculations because shoulder construction mixtures differ from main-road mixtures.
  • A shallow network was annotated and developed to crop the main road from each pavement image.
  • Figure 7 depicts automatic cropping of the main road from the shoulder via a convolutional neural network.

3.2. Developing a U-Net based Model for Distress Quantification

A pre-trained U-Net was adapted to quantify road-crack density, then retrained with manually cleaned images to reduce false detections from shadows and cars.

  • A pre-trained U-Net convolutional network, originally developed for biomedical segmentation, was used to quantify road-crack density.
  • U-Net uses an encoder–decoder architecture with skip connections, supporting pixel predictions through broader contextual information.
  • The pre-trained model detected cars and shadows as distresses, so these objects were manually removed and the model was retrained.
  • Retraining detected cracks accurately in the illustrated cases while significantly decreasing noise from removed shadows and cars.
  • Twenty images were used to test performance, producing a final mean square error of 0.25.
  • Figure 11 compares ground truth, predicted, and error images for one test image.

3.3. Developing a Hybrid Model of YOLO and U-Net

The hybrid model integrates YOLO and U-Net because comprehensive pavement-condition modeling requires both distress type and distress density information.

  • The study integrates YOLO and U-Net because neither model alone provided the most accurate pavement condition model.
  • Images were processed through the proposed YOLO and U-Net models individually before their outputs were combined in the hybrid approach.
  • Figure 12 presents the integrated YOLO–U-Net model and calculation of the ratio of white pixels.

3.4. Developing Pavement Condition Prediction Models

The study develops pavement-condition prediction models from distress detections, using GEP, U-Net-derived density measures, and hybrid approaches. These models predict PASER while accounting for distress types and densities, with block cracks identified as especially influential.

  • YOLO-based model: 71 pavement sections were represented by average distress detections, with predictor-variable distributions summarized using frequency histograms.The detected distress variables were derived from images processed through the YOLO crack-detection model.
  • GEP model: The dataset was randomly divided into 70% training, 15% validation, and 15% testing subsets to calibrate, externally check, and evaluate the models.The split was used to help avoid overfitting.
  • YOLO-based model: The proposed PASER model uses nine distress variables, including reflective, transverse, block, longitudinal, alligator, sealed, lane-longitudinal cracks, and potholes.The variables are average numbers of distresses per section.
  • GEP model: Block cracks had the highest influence on PASER, whereas lane longitudinal, sealed longitudinal cracks, and potholes had the lowest influence.Potholes, alligator cracking, and transverse cracking were absent or insufficiently represented in the dataset, limiting their modeled effects.
  • U-Net model: The U-Net model estimated crack density from the ratio of white pixels to image size, with threshold 127 correlating more strongly with PASER than threshold 200.The density measure captures distress extent but does not distinguish distress types.
  • Hybrid model: A hybrid model combined YOLO distress-type detection with U-Net distress-density estimation to incorporate both properties into a pavement condition index.Linear regression used distress-related variables to construct a PASER prediction equation.
  • Hybrid model: The PCI weighting model achieved R2 values of 0.87 for training and 0.94 for both testing and validation datasets.These values compare measured and predicted PASER for the weighting-based model.

4. Model Validation

The models were further evaluated on six pavement sections with different conditions using newly extracted Google Maps images. Predicted PASER profiles from hybrid and YOLO-based models were compared with measured pavement conditions.

  • Validation dataset: Six pavement sections with different conditions were used to further evaluate the generalization capability of the PASER prediction models.The sections included examples ranging from good condition to dense-block cracking.
  • Validation procedure: New images were extracted from Google Maps and analyzed with the developed YOLO and U-Net models before PASER prediction.The workflow calculated white-pixel ratios for detected distresses.
  • Model comparison: Figure 22 compares hybrid weight-based, hybrid linear-regression, YOLO GEP-based, and measured PASER values alongside pavement profiles.The comparison spans six pavement sections.

5. Conclusions

The study develops a deep-learning pavement condition assessment system that detects distress, quantifies severity, and predicts pavement condition. Its validation indicates useful agreement with measured PASER values, while highlighting limitations from qualitative section-level ratings and limited calibration data.

  • 5. Conclusions: The integrated YOLO and U-Net models classify nine pavement distress types and quantify their severity from road images.The models were trained and fine-tuned using manually annotated Google Street-view and road images collected under varied environmental conditions.
  • 5. Conclusions: Various pavement condition indices were developed from YOLO and hybrid-model distress outputs to predict PASER values.GEP-based, linear-regression, and weight-based models used detected distress information for pavement condition prediction.
  • 5. Conclusions: Block cracking was the most effective predictor of PASER variation among the evaluated distress variables.The authors associate its importance with advanced asphalt age and age hardening, which correlate with other pavement distresses.
  • 5. Conclusions: Predicted PASER values correlated well with measured values across six Missouri sections, but two cases swapped ranking.Predictions fluctuated more image by image because detected distress varied within sections, whereas measured PASER values were generally constant.
  • 5. Conclusions: The proposed tool offers cost-effective, flexible pavement monitoring using freely available Google Street-view imagery instead of expensive conventional equipment.The authors report acceptable average prediction error and potential usefulness for evaluating road-section conditions.

Future Studies

Future work focuses on expanding the PASER calibration data, extending the model to 3D imagery and additional distresses, and integrating deployment steps into software. These directions aim to broaden prediction coverage and operationalize the workflow.

  • Future Studies: The PASER prediction model was calibrated using a relatively limited set of 71 sections, and additional data are expected to enhance performance.The authors propose continued learning as larger databases become available through collaboration with research groups and owner-agencies.
  • Future Studies: Future models could incorporate 3D images and additional distresses such as rutting into pavement condition prediction.Three-dimensional imagery could also support more precise investigation of distress characteristics.
  • Future Studies: An integrated software system is planned to combine image extraction, image analysis, and pavement condition prediction.The stated purpose is to facilitate deployment of the proposed model.
Loading 2004.13314v1…