PaperlayerView exact revision on arXiv

Source-linked AI summary

Pothole Detection Based on Disparity Transformation and Road Surface Modeling

Rui Fan, Umar Ozgunalp, Brett Hosking, Ming Liu, Ioannis Pitas

arXiv:1908.00894v3cs.CVeess.IV

TL;DR

Pothole detection methods based on 2D images and 3D road-surface modeling are commonly used independently, and reported accuracy remains unsatisfactory. The paper combines disparity transformation with robust quadratic surface modeling and reports 98.7% successful detection accuracy and 99.6% overall pixel-level accuracy. However, its detection parameters do not apply to all cases, and road surfaces are not always quadratic.

  • Problem

    2D image analysis and 3D road-surface modeling are commonly used independently for pothole detection, while detection accuracy remains unsatisfactory.

  • Method

    The system transforms dense disparity maps, extracts potential undamaged regions, models disparities with a quadratic surface using surface normals and robust fitting, and detects potholes from actual-modeled disparity differences.

  • Results

    98.7% successful detection accuracy and 99.6% overall pixel-level accuracy were achieved by the proposed system.

  • Takeaways & Limitations

    The system provides a robust stereo-vision approach that combines disparity preprocessing with 3D road-surface modeling for pothole detection.

  • Takeaways & Limitations

    Detection parameters cannot be applied to all cases, and road surfaces cannot always be considered quadratic.

Abstract

from arXiv · show

Pothole detection is one of the most important tasks for road maintenance. Computer vision approaches are generally based on either 2D road image analysis or 3D road surface modeling. However, these two categories are always used independently. Furthermore, the pothole detection accuracy is still far from satisfactory. Therefore, in this paper, we present a robust pothole detection algorithm that is both accurate and computationally efficient. A dense disparity map is first transformed to better distinguish between damaged and undamaged road areas. To achieve greater disparity transformation efficiency, golden section search and dynamic programming are utilized to estimate the transformation parameters. Otsu's thresholding method is then used to extract potential undamaged road areas from the transformed disparity map. The disparities in the extracted areas are modeled by a quadratic surface using least squares fitting. To improve disparity map modeling robustness, the surface normal is also integrated into the surface modeling process. Furthermore, random sample consensus is utilized to reduce the effects caused by outliers. By comparing the difference between the actual and modeled disparity maps, the potholes can be detected accurately. Finally, the point clouds of the detected potholes are extracted from the reconstructed 3D road surface. The experimental results show that the successful detection accuracy of the proposed system is around 98.7% and the overall pixel-level accuracy is approximately 99.6%.

I. INTRODUCTION

Pothole detection must address the limitations of subjective manual inspection and the separate use of 2D image analysis and 3D road-surface modeling. This work introduces a stereo-vision system that combines disparity transformation with robust surface modeling to improve pothole detection.

  • Manual pothole inspection is time-consuming, tedious, and subjective because results depend on personnel experience.
  • I. INTRODUCTION: 2D image methods can be disrupted by poor illumination and may not explicitly represent potholes’ irregular 3D spatial structure.
  • 3D road information supports pothole-volume measurement, but laser-scanning systems are expensive and Kinect sensors suffer from infrared saturation in direct sunlight.
  • Stereo image pairs provide disparity maps for 3D road-data acquisition using inexpensive, portable, and durable digital cameras.
  • B. Motivation: Existing 2D and 3D pothole-detection algorithms are generally used independently, while detection accuracy remains unsatisfactory.
  • C. Novel Contributions: The proposed system transforms disparity maps, extracts undamaged regions with Otsu’s thresholding, and models them using quadratic surfaces with surface-normal filtering.

D. Paper Outline

The paper outlines a stereo-vision pothole detector that transforms disparity maps, extracts undamaged road areas, models the road surface, and detects potholes. It addresses roll-angle effects and computational cost through angle estimation, map rotation, and efficient search.

  • The algorithm has three main components: disparity transformation, undamaged road area extraction, and disparity map modeling with pothole detection.
  • A dense, subpixel-accurate disparity map is generated from a stereo road image pair and converted into a v-disparity map using row-wise disparity histograms.
  • The undamaged road surface is represented by a quadratic disparity model using coordinates centered in the disparity map.
  • 1) θ Estimation and Disparity Map Rotation: A non-zero roll angle makes row disparities less compact, degrading least squares fitting and increasing the minimum energy used for model estimation.
  • 1) θ Estimation and Disparity Map Rotation: The roll angle is estimated by rotating the disparity map and finding the angle that minimizes E_0min; golden section search reduces the computational burden of searching the angle interval.
  • 1) θ Estimation and Disparity Map Rotation: After rotation, the y-disparity map is formed from horizontal-row histograms, making disparity values on each row more uniform for subsequent processing.

2) 𝜶Estimation and Disparity Transformation:

The method estimates disparity-transformation parameters by extracting a target path with dynamic programming and refining parameters with RANSAC, then transforms disparities so undamaged and damaged regions separate more clearly.

  • α estimation: Dynamic programming extracts the road disparity projection model as a minimum-energy target path in the y-disparity map.The path coordinates are substituted into the model to obtain α.
  • α estimation: RANSAC refines α because outliers in the target path can reduce transformation-parameter estimation accuracy.It uses 50 iterations, three coordinate pairs per iteration, and four tolerance values.
  • Disparity transformation: Each disparity is transformed using the estimated parameters, with δ = 30 ensuring non-negative transformed disparity values.The transformation is represented by Eq. (15).
  • Disparity transformation: The transformed map makes undamaged-road disparities more uniform and separates them from pothole disparities, simplifying undamaged-region extraction.The transformed disparity map is shown in Fig. 6(a).

B. Undamaged Road Area Extraction

Otsu’s thresholding segments the transformed disparity map to extract undamaged road areas, while modeling is added because thresholding always produces two classes.

  • Undamaged road area extraction: Otsu’s method obtains the segmentation threshold by maximizing inter-class variance.The method uses the transformed disparity distribution and class statistics.
  • Undamaged road area extraction: The segmentation successfully extracts the undamaged road area from the transformed disparity map.The result is shown in Fig. 6(b).
  • Undamaged road area extraction: Because Otsu’s method always divides disparities into two categories, disparity-map modeling is used to avoid false pothole detections on undamaged roads.This safeguard applies when the transformed map contains no road damage.

C. Disparity Map modeling and Pothole Detection

The system models disparities on undamaged regions with a quadratic surface, excludes several outlier types, and uses the model–measurement difference to detect potholes.

  • Disparity map modeling: Disparity-map modeling fits a quadratic surface directly to disparities rather than modeling a reconstructed point cloud.Direct disparity modeling avoids point-cloud errors caused by disparity errors larger than one pixel.
  • Outlier handling: Outliers are discarded when points lie in damaged areas, have substantially different surface normals, or differ greatly from the modeled disparity.Only disparities in extracted undamaged areas are used for modeling.
  • Outlier handling: Surface-normal consistency is incorporated into disparity modeling to identify points whose geometry differs greatly from the optimal normal.This adds a geometric criterion beyond disparity-value fitting.

1) Optimal Normal Vector Estimation:

Optimal normal-vector estimation uses neighborhood-based PlanePCA estimates and selects the sphere location where normalized normal projections are most concentrated.

  • Optimal Normal Vector Estimation: PlanePCA estimates a normal vector for each disparity point from neighboring points in the undamaged road area.PlanePCA is selected for its reported speed and accuracy.
  • Optimal Normal Vector Estimation: Normalized normal vectors are projected onto a sphere, where their concentration reveals the optimal normal vector.The optimal vector is found at the position with the most intensive projection distribution.
  • Optimal Normal Vector Estimation: The two candidate spherical solutions are compared through the energy function to select the optimal normal vector.The selected vector is then used for angular outlier testing.
  • Optimal Normal Vector Estimation: Normals differing from the optimal vector by more than π/36 rad identify outliers, under an assumed 10% outlier proportion.These outliers are removed before disparity-map modeling.

2) Disparity Map Modeling:

RANSAC robustly models the disparity map by reducing the effects of outliers during quadratic-surface estimation.

  • 2) Disparity Map Modeling:: RANSAC reduces outlier effects when modeling the disparity map with a quadratic surface.Randomly selected disparities estimate the surface parameters, with disparities sampled uniformly across square blocks.

3) Pothole Detection:

Potholes are identified from sufficiently large regions whose actual disparities differ from the modeled surface beyond a threshold, then extracted from the reconstructed 3D road surface.

  • 3) Pothole Detection:: Potholes are detected where actual and modeled disparities differ by more than ε_d.Connected components smaller than w pixels are removed, and small holes are filled as noise before labeling.
  • 3) Pothole Detection:: Detected potholes are extracted as point clouds from the reconstructed 3D road surface.The corresponding extraction results are shown in Fig. 9.

III. EXPERIMENTAL RESULTS

The experiments evaluate the proposed system's roll-angle estimation and disparity transformation using stereo datasets and compare experimental outputs across processing stages.

  • III. EXPERIMENTAL RESULTS: The proposed algorithm was evaluated qualitatively and quantitatively in MATLAB using a single-threaded Intel Core i7-8700K CPU.The evaluation covers roll-angle estimation, disparity transformation, and pothole detection.
  • III. EXPERIMENTAL RESULTS: The experimental setup used a manually calibrated ZED stereo camera and three datasets containing 67 stereo-image pairs.The datasets had resolutions of 1028 × 1730, 1030 × 1720, and 1028 × 1710 pixels.
  • III. EXPERIMENTAL RESULTS: GSS reduces roll-angle estimation complexity from whole-interval search to logarithmic dependence on the search interval.The proposed algorithm requires 21 iterations and achieves accuracy higher than π/18000 rad.
  • III. EXPERIMENTAL RESULTS: 1.129 × 10^-4 rad is the average absolute roll-angle error on the synthesized EISATS stereo dataset.The error is lower than π/18000 rad.

C. Evaluation of Disparity Transformation

The evaluation examines disparity transformation and pothole detection using KITTI data and comparisons with prior algorithms, reporting strong detection and pixel-level performance.

  • C. Evaluation of Disparity Transformation: 194 and 200 disparity maps were used from the KITTI stereo 2012 and 2015 datasets, respectively, to evaluate disparity transformation.Because the algorithm focuses on road surfaces, regions of interest were manually selected.
  • C. Evaluation of Disparity Transformation: 0.5188 pixels is the average σ_d value after eliminating non-zero roll-angle effects in the two KITTI datasets.The average σ_d values were reduced by approximately half, and disparity transformation runtime was about 142 ms.
  • D. Evaluation of Pothole Detection: 6.2 and 3100 produced the least sum of Δn_PD, equal to one, during threshold and component-size selection.The text labels both selected values as ε_d, although the search is described for ε_d and w.
  • D. Evaluation of Pothole Detection: An incorrect detection occurred when a subsiding pothole center and selected parameters caused one pothole to be detected as two.Successful examples and corresponding ground truth are shown in Fig. 15.
  • D. Evaluation of Pothole Detection: 98.7% successful detection accuracy was achieved by the proposed algorithm, compared with 73.4% and 84.8% for and.The proposed method was compared with the two prior algorithms using comparative pothole-detection results.
  • D. Evaluation of Pothole Detection: The proposed algorithm outperformed and in pixel-level accuracy and F-score, with intermediate precision and recall performance.Its precision and recall were close to the highest values achieved by and.

IV. CONCLUSION AND FUTURE WORK

The paper contributes disparity transformation and disparity map modeling algorithms for pothole detection, achieving high detection accuracy while identifying parameter and surface-model limitations for future work.

  • The proposed system combines disparity transformation and disparity map modeling to improve pothole detection from stereo vision data.GSS and DP estimate transformation parameters, while surface-normal filtering improves modeled disparity-map accuracy.
  • 98.7% overall successful detection accuracy and approximately 99.6% pixel-level accuracy were reported for the proposed algorithm.
  • The authors created three datasets to support stereo vision-based pothole detection research.
  • Detection parameters cannot be applied to all cases, motivating future deep-neural-network detection from transformed disparity maps.
  • Because road surfaces are not always quadratic, future work proposes segmenting reconstructed surfaces into localized planes before detection.
Loading 1908.00894v3…