PaperlayerView exact revision on arXiv

Source-linked AI summary

Depth-Aware Pothole Detection Using YOLO and RT-DETR at the Edge

Md Monjurul Ahsan Prodhan, Md Nour Hossain

arXiv:2608.27633v1cs.CVcs.AIcs.LG

TL;DR

Pothole inspection needs physical depth measurement because RGB-only detection cannot quantify cavity severity, while manual surveys are labor-intensive. The paper compares five RGB-D detection architectures with adverse-condition augmentation and RANSAC orthorectification. YOLOv8n-Seg provides the strongest detection and depth result, while bounding-box models show a consistent structural depth overestimate.

Abstract

from arXiv · show

Pothole detection and its severity measurement is still an important challenges in urban infrastructure management, where late maintenance directly contributes to vehicle damage, road accidents, and escalating repair costs. Existing automated approaches depend on 2D RGB images and cannot measure physical depth of potholes. In this paper, we present a depthaware pothole detection framework and then compare five architectures: YOLOv8n, YOLOv8nSeg, YOLOv9t, RTDETRL, and RTDETRX for RGB-D sensor fusion-based detection and automated depth measurement. A custom offline augmentation pipeline is used here to simulate adverse road monitoring conditions. All models are trained on the PothRGBD dataset with an 80% training and 20% validation split and evaluated using Precision, Recall, mAP@50, and mAP@50_95. Before measuring the depth data, all depth maps are corrected for camera tilt using RANSAC ground-plane orthorectification and all zero-valued sensor pixels are cast to NaN before any statistic is computed. YOLOv8nSeg achieves the highest mAP@50 of 0.9556 and mAP@50_95 of 0.6758 with the most accurate depth estimate of 2.96 cm with the pixel-precise Dseg algorithm. YOLOv8n achieves the fastest inference at 3.6ms. RTDETRX achieves the highest detection confidence at 92.70%. An important finding is that even after full RANSAC orthorectification, bounding box models overestimate pothole depth by 0.16 to 0.21 cm compared to pixel precise segmentation masks. This confirms that the pavement inclusion bias is structural rather than a calibration artifact.

I. INTRODUCTION

Potholes create substantial safety and economic burdens, while conventional inspection cannot quantify cavity depth for proactive severity-based maintenance. The paper addresses this gap with RGB-D depth-aware detection and a comparison of CNN, transformer, and segmentation architectures.

  • 55 million potholes affect nearly four million miles of U.S. roads, with over 30% of urban roads and highways in mediocre condition.Source: §I · p. 1
  • £14.4 billion is the estimated annual economic damage from potholes in England.Source: §I · p. 1
  • Manual inspection is labor-intensive and cannot provide quantitative cavity-depth measurements for objective repair prioritization.Source: §I · p. 1
  • The framework compares YOLOv8n, YOLOv8n-Seg, YOLOv9t, RT-DETR-L, and RT-DETR-X under a unified PothRGBD experimental protocol.Source: §I · p. 1
  • The study contributes adverse-condition augmentation, RANSAC ground-plane orthorectification, and quantification of spatial measurement bias.Source: §I · p. 1
  • Earlier pothole detectors relied on manual features or RGB imagery, while YOLO enabled end-to-end real-time detection and segmentation methods enabled pixel-precise boundaries.Source: A. CNN-Based Pothole Detection · p. 2

B. Transformer Architectures for Object Detection

Transformer-based detectors replace or reduce handcrafted detection components through attention and set prediction, while RT-DETR adapts these ideas for real-time inference. RGB-D sensing complements image detection by adding aligned per-pixel depth for infrastructure monitoring.

D. Image Augmentation for Adverse Condition Robustness

Road-monitoring detectors require augmentation that reflects domain-specific visual degradation rather than relying only on generic transformations. The proposed RGB-D pipeline combines adverse-condition simulation with depth-aware processing for physical measurement.

B. Augmentation Pipeline

The study combines adverse-condition augmentation with five differently sized detection architectures under a common training configuration. The pipeline includes both segmentation and transformer alternatives for RGB-D pothole analysis.

Augmentation Pipeline

  • Six Albumentations techniques simulate precipitation, shadows, glare, low light, and sensor noise for adverse road-monitoring conditions.Each technique is applied at p = 0.4 in the described pipeline.Source: B. Augmentation Pipeline · p. 3

Training Configuration

Augmentation Pipeline

Model Architectures

D. Ground-Plane Orthorectification

The framework corrects RGB-D depth maps for camera tilt before extracting pothole depth, using cleaned point clouds, RANSAC plane fitting, rotation, and reprojection. Detection outputs are then paired with the orthorectified maps for geometry-specific depth estimation.

Cleaning and reprojection

  • Zero-valued depth pixels are cast to NaN before statistics, preventing dead or out-of-range sensor measurements from corrupting depth calculations.The cleaned map is reprojected using approximate Intel RealSense D415 intrinsics.Source: D. Ground-Plane Orthorectification · p. 3

Plane fitting

Tilt correction

  • A Rodrigues rotation aligns the fitted surface normal with the world vertical axis, removing camera pitch and roll.This rotation flattens the road surface perpendicular to the Z-axis.Source: D. Ground-Plane Orthorectification · p. 4

Map reconstruction

  • The rotated point cloud is interpolated onto the original image grid, producing an orthorectified map whose depth differences represent true vertical distances.The result is independent of how the camera was mounted.Source: D. Ground-Plane Orthorectification · p. 4

Depth extraction

  • Orthorectified depth maps are fused with detections, using separate extraction algorithms for rectangular boxes and segmentation masks.The distinction reflects their different geometric outputs.Source: E. RGB-D Physical Depth Extraction · p. 4

Bounding-box extraction

  • Bounding-box depth uses valid pixels inside the crop and robust percentile statistics rather than raw extrema.Measurements with fewer than 10 valid pixels are reported as sensor errors.Source: 1) Bounding Box Depth (YOLOv8n,… · p. 4

2) Segmentation Mask Depth (YOLOv8n-Seg):

YOLOv8n-Seg estimates pothole depth from a pixel-precise mask and a locally defined surrounding-road baseline. This geometry avoids including healthy pavement inside the pothole region, unlike rectangular bounding boxes.

Segmentation mask depth

  • The segmentation mask is thresholded at 0.5, resized to the original image, and surrounded by a 15-pixel buffered ring for local pavement estimation.The pothole mask is removed from the ring to isolate pavement around the cavity.Source: 2) Segmentation Mask Depth (YOLOv8n-Seg) · p. 4

Geometric rationale

  • The segmentation formulation is geometric because the mask traces the pothole edge while the surrounding ring contains genuine road-surface pixels.This contrasts with the healthy pavement enclosed by rectangular boxes.Source: 2) Segmentation Mask Depth (YOLOv8n-Seg) · p. 4

Matched comparison

IV. EXPERIMENTS AND RESULTS

The experiments train and evaluate five architectures under a fixed RGB-D protocol on PothRGBD, with detection performance summarized across four metrics. YOLOv8n-Seg leads the reported mAP measures, while YOLOv9t has the highest precision and RT-DETR-L the lowest mAP@50.

Experimental setup

  • 50 epochs at 640 × 640 train each architecture on 1,600 augmented samples and evaluate on the fixed 200-image validation split.All experiments run on a single NVIDIA Tesla T4 with a confidence threshold of 0.25.Source: A. Experimental Setup · p. 4

Evaluation metrics

  • Precision, Recall, mAP@50, and mAP@50-95 are reported for all five architectures on PothRGBD’s validation set.The evaluation compares the models under a shared metric set.Source: B. Detection Performance · p. 4

Detection results

  • 0.9556 mAP@50 and 0.6758 mAP@50-95 are achieved by YOLOv8n-Seg, the highest values across the compared architectures.The result is presented as evidence that its segmentation head improves localization accuracy without sacrificing detection quality.Source: B. Detection Performance · p. 4
  • 0.9525 Precision is achieved by YOLOv9t, while RT-DETR-L records the lowest mAP@50 at 0.9022.The comparison spans CNN and real-time Transformer architectures.Source: B. Detection Performance · p. 5

Performance comparison

C. Training Convergence Analysis

The convergence curves reveal a divide between steadily learning CNNs and transformer models with a prolonged warm-up before rapid improvement. YOLOv8n-Seg converges fastest among the CNNs and reaches the highest reported mAP@50 among them.

D. Inference Speed Analysis

Inference speed and depth results expose a practical trade-off across architectures: YOLOv8n is fastest, while segmentation provides more accurate physical depth measurement. Bounding-box models consistently overestimate depth even after camera-tilt correction.

Inference speed

  • 3.6 ms and 8.1 GFLOPs make YOLOv8n the fastest architecture measured on the Tesla T4 GPU.YOLOv8n-Seg takes 22.7 ms, while YOLOv9t, RT-DETR-L, and RT-DETR-X take 49.5, 67.3, and 90.2 ms, respectively.Source: D. Inference Speed Analysis · p. 5

RGB-D inference output

Physical depth measurement

  • 2.96 cm is the YOLOv8n-Seg depth estimate using Dseg, compared with 3.12–3.17 cm from all four bounding-box models using Dbox.The measurements use the same matched RGB-depth image pair after RANSAC orthorectification and zero-depth masking.Source: E. RGB-D Physical Depth Measurement… · pp. 5–6
  • 0.16–0.21 cm is the bounding-box depth overestimate relative to pixel-precise segmentation, across lightweight CNN and large Transformer architectures.The bias persists after camera tilt is fully corrected, so it is attributed to the rectangular geometry of bounding boxes.Source: E. RGB-D Physical Depth Measurement… · p. 6
  • 0.05 cm is the spread among the four bounding-box estimates, ranging from 3.12 cm for YOLOv8n to 3.17 cm for YOLOv9t.Their close agreement supports a consistent systematic bias rather than random measurement noise.Source: E. RGB-D Physical Depth Measurement… · p. 6

F. Comparative Analysis and Architectural Trade-offs

The five architectures exhibit distinct performance trade-offs rather than a single overall winner. YOLOv8n-Seg leads accuracy and depth estimation, while other models prioritize latency, precision, parameter efficiency, confidence, or recall.

V. CONCLUSION

The study compares five architectures for edge-based, depth-aware pothole detection using PothRGBD and weather augmentation. YOLOv8n-Seg leads detection and depth accuracy, while bounding-box models systematically overestimate depth; future work targets broader validation and deployment.

  • YOLOv8n-Seg achieves the highest mAP@50 of 0.9556 and the most accurate depth estimate of 2.96 cm.Source: §V · p. 6
  • YOLOv8n provides the fastest inference at 3.6 ms, while RT-DETR-X achieves the highest confidence at 92.70% and Recall at 0.9208.Source: §V · p. 6
  • Bounding-box models overestimate depth by 0.16–0.21 cm relative to segmentation masks, quantifying healthy pavement inclusion bias.Source: §V · p. 6
  • Future work will validate the approach on larger datasets, deploy it on embedded hardware such as NVIDIA Jetson, and extend detection to cracking and rutting.Source: §V · p. 6

Related papers

Loading 2608.27633v1…