Global Road Damage Detection: State-of-the-art Solutions

• Specialized survey vehicles with laser linescan and 3D cameras can be unaffordable for local agencies, motivating low-cost smartphone-based road monitoring.A preliminary smartphone-based system was developed by the University of Tokyo, and a 2018 challenge evaluated methods for this goal.Source: §1 · p. 1
• Feedback from Japanese municipalities using the 2018 system suggested algorithms needed greater robustness, while existing models were often limited to one country.Source: §1 · p. 2
• The 2020 challenge builds on RDD2020, which augmented Japanese road images with data from India and the Czech Republic, and summarizes participant solutions for those countries.RDD2020 comprises 26,620 images, described as almost thrice the volume of the 2018 dataset.Source: §1 · p. 2

2.1 Overview

• GRDDC pairs a public dataset with annotation and standard evaluation methods with an online competition and workshop measuring that year's state of the art.The paper focuses mainly on the 2020 challenge.Source: §2.1 · p. 2

2.2 Dataset

• The challenge data are divided into Train, Test1, and Test2; training images have PASCAL VOC-format XML annotations, while test images are unannotated.Figures 1 and 2 show dataset and country statistics and counts of training instances by damage type, respectively.Source: §2.2 · pp. 2–3, Figure 1 · p. 3, Figure 2 · p. 3
• Challenge dataset statistics differ slightly from Arya et al. (2020) because some images were excluded for privacy reasons.Source: §2.2 · p. 3

2.3 Tasks

• Participants predict damage labels and bounding boxes for images from India, Japan, and the Czech Republic, covering detection location and damage type.All algorithm types and pretrained models are allowed, but training is restricted to the provided road-data training sets; augmentation and GAN-generated images are permitted.Source: §2.3 · pp. 3–4

3.1 Submission of Results

• The four-phase submission process released test1 first and test2 later, with source code required in the final phase to verify replicability.Test1 was available for about four months; test2, comprising 2,664 images, was released 15 days before the deadline.Source: §3.1 · p. 4

3.2 Evaluation

• A prediction counts as correct only when its bounding box has IoU > 0.5 and its predicted label matches the ground truth.The evaluation script compares participant predictions with ground-truth annotations.Source: §3.2 · pp. 4–5
• The evaluation script calculates F1 as the harmonic mean of precision and recall, with true and false positives and negatives determined by instance presence and correct label and box prediction.Source: §3.2 · p. 5
• Teams are ranked by the average of their separate test1 and test2 F1 scores, which weights precision and recall equally.Source: §3.2 · p. 5

• 121 teams registered; 13 were shortlisted, 12 reached the final round, and the top three teams averaged more than 60% F1-score.Source: §4 · p. 5
• 0.67 F1-score for test1 was the highest reported for IMSC’s EM+EP, which was also the slowest approach by detection time per image.Source: §4 · pp. 6–7
• EM+EP combines selected u-YOLO variants with test-time image transformations, averaging predictions before non-maximum suppression.The transformations include horizontal flipping and increasing image resolution.Source: §4 · p. 6
• Tables 2 and 3 present contributor details and proposed-solution details for the top 12 teams.Source: Table 2 · p. 7, Table 3 · p. 8

5 Discussion and Future Scope

• Researchers could continue submitting to new test1 and test2 leaderboards after the 2020 competition closed, but those submissions did not count toward that competition.Source: §5 · p. 9

5.1 The Positive Points of the challenge

• Standard scoring and fixed test sets improved comparability by limiting differences in evaluation choices and test-image composition.The passage notes that class and country imbalances could otherwise advantage teams whose test images had a more favorable composition.Source: §5.1 · p. 9
• Data expansion from Japan to India and the Czech Republic increased dataset size and geographic coverage, adding varied road textures and damage levels.Source: §5.1 · p. 9

5.2 Scope for Future and the room for improvement

• Current challenges restrict methods to the provided training data; a separate category could allow methods trained on other data.Source: §5.2 · p. 10
• Future challenges could add severity analysis and pixel-level damage analysis beyond detection and classification.Source: §5.2 · p. 10
• Multiple evaluation metrics, including inference speed and model size alongside F1-score, could better assess real-time smartphone-based methods.Source: §5.2 · p. 10
• Future evaluation could report performance for individual damage classes rather than relying only on average F1-score.Source: §5.2 · p. 10
• The dataset could be augmented to represent damage classes more evenly.Source: §5.2 · p. 10

• GRDDC targets detection and classification of four damage types across India, Japan, and the Czech Republic, where road quality and damage characteristics vary.Source: §6 · p. 10
• The paper offers insights for algorithm designers, challenge designers, and transport engineers interested in automating road-damage detection.Source: §6 · p. 10
• Future challenges could add tasks such as crack-ratio analysis, pothole-severity analysis, and road segmentation, alongside additional evaluation metrics.Source: §6 · p. 10