PaperlayerView exact revision on arXiv

Source-linked AI summary

Pixel-Level Pavement Distress Assessment Using Instance Segmentation

Logan Dewick, Bibesh Pyakurel, Kong Pheng Yang, Nazim Choudhury, M. G. Sarwar Murshed

arXiv:2605.26095v2cs.CV

TL;DR

Maintenance-relevant pavement assessment requires precise localization and area measurement beyond image classification or coarse boxes. This paper evaluates Mask R-CNN instance segmentation on smartphone-collected, polygon-annotated roadway imagery, with ResNet-101 FPN achieving strong detection metrics and close aggregate crack-area agreement with ground truth.

  • Problem

    Pavement distress assessment needs geometric precision and instance-level granularity that object detection or semantic segmentation alone may not provide.

  • Method

    The study fine-tunes five Detectron2 Mask R-CNN backbone variants on UWGB-STREETCRACK and compares the framework with a CSPDarknet53-based YOLO detector.

  • Results

    84.23% precision, 90.04% recall, and 87.04% F1 were achieved by ResNet-101 FPN, whose predicted crack area was 2.164% versus 2.170% ground truth.

  • Takeaways & Limitations

    Instance segmentation is presented as a practical direction for field pavement imagery and aggregate crack-area estimation.

  • Takeaways & Limitations

    The study identifies annotation ambiguity, minority-class sparsity, confounder rejection, and missing mask-level AP metrics as unresolved challenges.

Abstract

from arXiv · show

Automated pavement distress assessment requires more than image-level classification or coarse bounding box detection, demanding precise localization of thin, branching, and irregular cracks to achieve the geometric precision necessary for maintenance-relevant quantification. This paper presents a vision-based pavement distress analysis system based on Mask R-CNN instance segmentation and evaluates it on UWGB-StreetCrack, a custom field-collected roadway image dataset acquired with a vehicle-mounted smartphone and manually annotated with polygon labels for longitudinal cracks, transverse cracks, alligator cracks, and potholes. Five Detectron2-based Mask R-CNN backbone variants were considered under a consistent fine-tuning protocol. The best-performing model, Mask R-CNN with a ResNet-101 FPN backbone, achieved 84.23% precision, 90.04% recall, and an F1 score of 87.04% under the project-specific bounding-box matching protocol. The same model produced an aggregate predicted crack-area fraction of 2.164%, closely matching the 2.170% ground-truth crack-area fraction. To contextualize the segmentation system against a detector-oriented alternative, a CSPDarknet53-based YOLO detector was also adapted and retrained on the dataset, reaching 27.5% precision and 20.7% recall on the validation protocol. The results show that instance segmentation is a practical direction for field pavement imagery and aggregate crack-area estimation, while also exposing open challenges in annotation consistency, class imbalance, confounder rejection, and mask-level benchmarking.

I. INTRODUCTION

Pavement distress assessment needs objective, geometrically precise measurements, but manual inspection and existing detection or segmentation approaches remain limited. The study evaluates Mask R-CNN instance segmentation on field-collected roadway imagery and compares it with detection-oriented methods.

  • Motivation: Deferred pavement maintenance and untreated cracks motivate automated, quantitative distress detection for safer and more evidence-based infrastructure decisions.Manual inspection is labor-intensive, hazardous, and inconsistent across teams, seasons, and regions.
  • Problem: Existing deep learning approaches can analyze roadway imagery but may not provide the geometric precision needed for exact distress-area measurement.Precise surface-area estimates support maintenance prioritization and budget allocation.
  • Problem: Neither object detection nor semantic segmentation alone provides both geometric precision and instance-level granularity for maintenance decision-making.The methodological gap arises between spatial precision without instance separation and instance localization without pixel-level delineation.
  • Approach: Mask R-CNN unifies localization, classification, and pixel-level delineation while preserving separate distress instances.Its masks provide finer geometric representation than rectangular bounding boxes.
  • Study scope: The study evaluates Mask R-CNN on UWGB-STREETCRACK, quantifies aggregate crack-area agreement, documents methodological limitations, and retrains a detector for comparison.The dataset contains polygon annotations for longitudinal, transverse, and alligator cracks, plus potholes.
  • Limitations: The paper identifies annotation ambiguity, minority-class sparsity, and missing mask-level AP metrics as methodological gaps.These issues are presented as part of the study’s failure analysis and future benchmark requirements.

II. RELATED WORK

Pavement distress research has progressed from handcrafted image processing through deep learning classification, semantic segmentation, detection, and instance segmentation. Related work shows that field-relevant assessment still requires both precise geometry and separation of individual distress instances.

  • Classical methods: Early pavement crack studies used thresholding, edge detection, morphology, wavelet analysis, path extraction, and other handcrafted operations.These methods exploited cracks’ often darker appearance relative to surrounding pavement.
  • Deep learning: CNNs learned image representations directly, while encoder-decoder architectures improved pixel-level crack delineation.U-Net established a widely used skip-connected semantic-segmentation pattern for crack extraction.
  • Semantic segmentation: Semantic segmentation predicts crack pixels but generally does not separate adjacent distress instances.This limitation matters for mixed classes and ambiguous alligator-crack patterns.
  • Instance segmentation: The related-work progression motivates instance segmentation as a framework combining pixel-level masks with instance-level differentiation.The paper positions this capability as relevant to practical maintenance-oriented assessment.

C. Object Detection and Hybrid Pipelines

Object detectors efficiently localize multiple pavement-distress classes, while hybrid pipelines add segmentation when crack geometry or area is required. The study addresses the field-imagery gap by evaluating instance segmentation on full-scene, smartphone-collected roadway data.

  • Object detection: Object detectors such as Faster R-CNN, YOLO, CenterNet, and EfficientDet are used for efficient multi-class pavement-distress localization.Their detector-oriented role is localization rather than pixel-level mask production.
  • Hybrid pipelines: Hybrid systems combine detection with segmentation to identify candidate regions and measure crack geometry or surface features.Examples include SSD-style localization with U-Net and YOLOv3 followed by modified U-Net segmentation.
  • Instance segmentation: Mask R-CNN extends Faster R-CNN with a parallel mask branch, while FPN combines features across resolutions for multi-scale detection.These components support instance-level localization and mask prediction.
  • Research gap: Most prior segmentation studies use cropped, single-crack, or narrower-condition imagery, leaving full-scene smartphone roadway assessment relatively understudied.The current study evaluates four distress categories in a custom field-collected dataset.
  • Acquisition protocol: The dataset acquisition used an iPhone 15 Pro Max mounted on a vehicle, with video converted into roadway still frames.Frames were filtered to reduce repeated coverage before manual review.

B. Annotation Taxonomy and Quality Control

UWGB-STREETCRACK uses polygon masks and a four-class distress taxonomy designed for instance segmentation. The annotation process records class, box, and polygon information but remains vulnerable to ambiguity in visually confounded or connected crack patterns.

  • Taxonomy: The taxonomy contains longitudinal cracks, transverse cracks, alligator cracks, and potholes.Longitudinal and transverse classes are distinguished by approximate roadway orientation, while alligator cracks are interconnected networks.
  • Annotation format: Valid annotations include a class label, bounding box, and one or more polygon segments.Polygon representation is required because the target task is instance segmentation rather than box-only detection.
  • Quality control: Stains, markings, manhole edges, and faint linear textures can resemble cracks and create annotation ambiguity.Connected patterns may also be labeled as one alligator-crack instance or as multiple longitudinal and transverse cracks.
  • Annotation workflow: The annotation workflow in Label Studio produced COCO-style polygon annotations for the four target classes.The polygon-based process is illustrated in Fig. 2.
  • Dataset split: The stored training and validation splits contain 1,309 images and 1,484 labeled distress instances, while a held-out test partition contains 231 images and 261 instances.The test partition was reserved solely for evaluating trained models.

IV. METHODOLOGY

The study used a Detectron2 Mask R-CNN pipeline for full-scene pavement-distress instance segmentation, with consistent backbone evaluation and restrained augmentation.

  • Pipeline and model evaluation: Mask R-CNN was implemented in Detectron2 and evaluated across five backbone variants using consistent hyperparameters.The variants included ResNet-50, ResNet-101, and ResNeXt-101 with different feature-extraction strategies.
  • Pipeline and model evaluation: The pipeline comprised field-image curation and annotation, binary-mask conversion and loading, COCO-pretrained fine-tuning, and evaluation.
  • Data preparation: Valid Label Studio polygons were rasterized as closed contours into one binary instance mask per polygon, while invalid polygons with fewer than three vertices were excluded.
  • Preprocessing: Images were resized with preserved aspect ratio to 800-pixel shorter and 1333-pixel larger sides, with padding applied only afterward when required.
  • Augmentation: Training used random horizontal flipping at probability 0.5 and multi-scale resizing, without color, blur, geometric, or synthetic perturbation.

C. Mask R-CNN Variants

The project compared five Mask R-CNN backbone variants and adapted a CSPDarknet53-based YOLO detector as a methodological baseline under separate training configurations.

  • Mask R-CNN variants: Five Mask R-CNN backbone variants were considered in the project pipeline.
  • Mask R-CNN variants: ResNet-50 FPN and ResNet-101 FPN achieved better test performance than other available Detectron2 variants.
  • Training protocol: The annotated images were split 70/15/15 into training, validation, and test subsets, with the held-out test partition containing 231 images and 261 labeled crack instances.
  • Detection baseline: The CSPDarknet53-based YOLO detector was adapted to the dataset using YOLO-format class identifiers and normalized bounding-box coordinates, but it could not perform pixel-level segmentation.
  • Detection baseline: YOLO training used 640 × 640 inputs, batch size 16, 100 epochs, and default optimizer settings; convergence occurred around 70–80 epochs, with no significant improvement at 200 epochs.

F. Baseline Segmentation-Based Evaluation

The study included a crop-level segmentation baseline and emphasized that Mask R-CNN performs localization, classification, and pixel-level segmentation jointly on full roadway images.

  • Baseline segmentation: DeepSegmentor was evaluated as a CNN-based crop-level crack-segmentation baseline because it could not process full roadway images.Crack regions were manually cropped from full roadway images before prediction.
  • Mask R-CNN comparison: Mask R-CNN performs simultaneous localization, classification, and pixel-level segmentation on full roadway images in an end-to-end model.

G. Inference Configuration

Inference used fixed image preprocessing and proposal settings, while evaluation separated object-level matching from pixel-level crack-area computation.

  • Inference settings: Inference reused training image aspect-ratio and size properties without stochastic augmentation.
  • Evaluation protocol: Mask R-CNN precision, recall, and F1 were interpreted as object-level metrics using predicted-to-ground-truth matching based on bounding-box IoU.
  • Evaluation protocol: A prediction counted as a true positive when its class matched and bounding-box IoU was at least 0.1, with one-to-one instance assignment.
  • Pixel-level evaluation: Detected crack area was computed by summing predicted or annotated mask pixels and normalizing by total image area across the evaluated set.

VI. RESULTS

Table III reports precision, recall, F1 score, and detected crack-area percentages for pavement-crack analysis models. ResNet-101 FPN 3x achieved the strongest overall detection performance among the evaluated Mask R-CNN variants.

  • 84.23% precision, 90.04% recall, and 87.04% F1 score were achieved by the ResNet-101 FPN 3x backbone.
  • 75.56% precision, 89.01% recall, and 81.73% F1 score were achieved by the ResNet-50 FPN 3x variant.
  • 8.67 percentage points higher precision and 5.31 percentage points higher F1 score were obtained with ResNet-101 than ResNet-50.Recall also improved modestly with the deeper backbone.

1) Object-level detection:

The ResNet-101 FPN 3x Mask R-CNN outperformed the CSPDarknet53 YOLO detector and closely matched ground-truth aggregate crack area. Qualitative failures involved visual confounders and annotation ambiguity, especially for alligator cracks.

  • 1) Object-level detection:: 84.23% precision, 90.04% recall, and 87.04% F1 score exceeded CSPDarknet53's 27.5% precision, 20.7% recall, and 23.62% F1 score.The reported differences were 56.73, 69.34, and 63.42 percentage points, respectively.
  • 2) Pixel-level segmentation:: 2.164% predicted crack area differed by 0.006 percentage points from the 2.170% ground-truth crack area.DeepSegmentor estimated 2.130%, differing by 0.040 percentage points, and required manually cropped crack-containing regions.
  • C. Qualitative Error Analysis: Painted road markings and oil stains produced false positives because their elongated or high-contrast appearances resembled cracks.Manhole-cover scenes were also ambiguous because nearby faint linear structures could represent distress or artifacts.
  • C. Qualitative Error Analysis: Alligator cracks sometimes caused the model to predict one connected distress region while annotations separated the region into multiple instances.The passage reports one case where the model predicted the complete distress area and the error reflected an annotation flaw.
  • C. Qualitative Error Analysis: Strict annotation guidance is needed because the same crack network may be labeled as one alligator-crack instance or multiple linear cracks.Such ambiguity affects model performance.

VII. CONCLUSION

The Mask R-CNN framework achieved strong pavement-distress detection and closely matched ground-truth aggregate crack-area estimates on UWGB-STREETCRACK, while the dataset remains unreleased.

  • 84.23% precision, 90.04% recall, and 87.04% F1 were achieved by the best-performing ResNet-101 FPN backbone.
  • 2.164% predicted crack-area fraction closely matched the 2.170% ground-truth fraction, supporting area-based maintenance decisions.
  • 27.5% precision and 20.7% recall were achieved by the CSPDarknet53 YOLO reference model.
  • The UWGB-STREETCRACK dataset contains 1,540 smartphone-based roadway images with polygon annotations for four pavement-distress types.
  • The dataset is not yet publicly released and is available from the corresponding author upon reasonable request.
Loading 2605.26095v2…