PaperlayerView exact revision on arXiv

Source-linked AI summary

Assessing the Impact of Fleet Size on Crowdsourced Mapping Using a Dissimilarity Measure

Marie-Ngoïe Badibanga Kalenda, Philippe Bonnifait, Marie-Anne Mittet

arXiv:2609.29198v1cs.ROphysics.data-an

TL;DR

Traffic-sign mapping requires accurate localization and semantic classification, while classical vector-map metrics do not provide one overall map-quality value. The paper simulates uncertain multi-vehicle observations and evaluates semantic and spatial aggregation; the largest improvements occur as the fleet grows from 5 to about 30 vehicles, with limited gains beyond that in the studied scenario.

  • Problem

    Traffic signs require both accurate localization and correct semantic classification, and classical vector-map metrics do not provide a single numerical value for overall map quality.

    Source: §1 · p. 1, §5.1 · p. 6

  • Method

    A fleet simulation models sensing and detection errors, then combines sign-type grouping and spatial clustering to aggregate observations and assess mapping quality.

    Source: §1 · p. 1, §4.2.1 · p. 5, §4.2.2 · p. 5

  • Results

    The most significant mapping improvements occur as contributing vehicles increase from 5 to approximately 30; beyond that, improvement is limited to sign-location accuracy in the considered scenario.

    Source: §6.3 · p. 7, §7 · p. 8

  • Takeaways & Limitations

    GOSPAM supports comparisons of data-processing approaches for crowdsourced mapping, and semantic-first grouping prevents nearby signs of different types from being merged.

    Source: §6.3 · p. 7

    The observed trends still need evaluation in more diverse road environments, including urban, highway, and rural settings.

    Source: §7 · p. 8

Abstract

from arXiv · show

Accurate digital maps are essential for Advanced Driver Assistance Systems (ADAS) or Autonomous Driving (AD), providing critical information such as road geometry, traffic signs and speed limits required by safety functions including Intelligent Speed Assistance (ISA). Maintaining these map layers using traditional surveying methods is costly and difficult to scale. Crowdsourced approaches based on fleets provide a promising alternative for continuously validating and updating map information. However, the relationship between the number of contributing vehicles and the quality of the resulting map remains poorly understood. To address this gap, this paper presents a simulation-based framework for evaluating crowdsourced traffic sign maintenance using a dissimilarity measure called GOSPAM (Generalized Optimal SubPattern Assignment for Maps), which combines localization errors with detection performance by accounting for False Positives (FP) and False Negatives (FN). The proposed system models multivehicle observations with representative sensor noise, detection errors, and semantic recognition uncertainties. Observations from multiple vehicles are aggregated using spatial clustering and semantic filtering to estimate traffic sign locations. Using simulated trajectories generated from data carried out by an experimental vehicle in an area containing ground-truth traffic signs, we assess the influence of fleet size on the performance of crowdsourced mapping. The number of vehicles ranges from 5 to 50, and performance is analyzed using standard evaluation metrics which are compared to the GOSPAM . The results show that GOSPAM can be used to effectively assess the quality of crowdsourced mapping, such as the contributions made by the first vehicles or the improvements made by numerous vehicles.

1. INTRODUCTION

Traffic-sign map updates must get both locations and sign semantics right, yet the relationship between fleet size and map quality is poorly understood. The paper proposes controlled simulation, aggregation-strategy comparisons, and quality assessment with classical metrics and GOSPAM.

1. INTRODUCTION

  • Traffic signs are discrete point features requiring accurate localization and correct semantic classification, while similar objects can generate false positives.A spatially close detection of the wrong sign type cannot be corrected by spatial aggregation alone.Source: §1 · p. 1
  • The paper asks how many vehicles are needed for reliable map updates and explores GOSPAM to combine localization errors with false-positive and false-negative penalties.The fleet-size relationship remains poorly understood, and assessing it calls for metrics that jointly capture localization accuracy and detection completeness.Source: §1 · p. 1
  • The study compares spatial clustering strategies combined with semantic classification for aggregating traffic-sign detections.This is one of the stated main contributions.Source: §1 · p. 1
  • A configurable fleet simulator models multi-vehicle traffic-sign observations with localization noise, false positives, false negatives, and semantic recognition errors.The framework enables controlled evaluation of the effect of contributing fleet size on mapping quality.Source: §1 · p. 1
  • The paper evaluates crowdsourced traffic-sign mapping quality using both classical metrics and GOSPAM.The paper identifies traffic-sign mapping quality evaluation as a contribution.Source: §1 · p. 2

2. RELATED WORK

Prior work establishes the potential of crowdsourced mapping across road features and traffic signs, but leaves questions about fleet-size scaling, multi-vehicle aggregation, and joint evaluation of localization and detection performance. The paper situates its framework against those open questions.

2. RELATED WORK

  • Crowdsourced HD-map pipelines span heterogeneous-sensor collection, feature extraction, change detection, and map fusion, each challenged by noise, sparse sampling, and sensor variation.Surveys describe crowdsourcing as a route toward map freshness and accuracy for autonomous driving.Source: §2 · p. 2
  • Prior systems use graph-based SLAM, filtering, end-to-end 3D mapping, semantic edges, expectation-maximization, and fleet data for lane-level updates.These works address diverse mapping stages and data sources.Source: §2 · p. 2
  • Traffic-sign mapping work includes multi-journey incremental updates and low-cost-camera reconstruction, while some pipelines assume fixed sensor characteristics.The cited low-cost sensing systems report sub-meter reconstruction through multi-route triangulation, clustering, and bundle adjustment.Source: §2 · p. 2
  • Crowdsourced road-network and lane mapping has addressed continuous features, whereas discrete landmarks such as traffic signs receive less focus.Reviews also note challenges in heterogeneous-sensor modeling, detection-error mitigation, and aggregation that scales with fleet size.Source: §2 · p. 2
  • Open questions include how map quality scales with fleet size, how traffic-sign detections should be combined, and how to evaluate localization and detection jointly.These questions motivate the study's uncertainty-aware simulation and multi-vehicle aggregation framework.Source: §2 · p. 2

3. CROWDSOURCING ARCHITECTURE

The architecture sends vehicle observations to a cloud back end for aggregation and map maintenance. Localized job requests solicit data from passing vehicles until the map reaches its required quality threshold.

3. CROWDSOURCING ARCHITECTURE

  • The architecture has a vehicle-side front end and cloud-based back end that collect and aggregate traffic-sign observations to maintain map tiles.The system diagram describes data harvesting from vehicles to the cloud.Source: §3 · p. 2, Figure 1 · p. 3
  • Vehicles detect, track, and map-match signs locally, then transmit georeferenced observations when a change is detected or a tile job request is active.Observations include timestamps, spatial coordinates, and semantic attributes.Source: §3 · p. 2
  • The back end cleans and normalizes data, fuses detections, updates map landmarks, and evaluates map quality using GOSPAM and conventional metrics.Job requests remain active until the required quality threshold is reached, after which the updated map is distributed and requests stop.Source: §3 · p. 3, Figure 1 · p. 3

4. SIMULATION METHODOLOGY

The methodology simulates traffic-sign observations along an experimental-vehicle route, adds configurable sensing and detection errors, and aggregates observations across vehicles. It evaluates front-end tracking and back-end semantic and spatial aggregation.

4. SIMULATION METHODOLOGY

  • Simulated fleets follow variants of an RTK-tracked experimental-vehicle route over a ground-truth HD map, generating uncertain sign observations for an active map-update request.The simulator models localization noise, false detections, missed detections, and semantic recognition errors before aggregating observations to estimate sign locations.Source: §4 · p. 3

4.1 Detection Simulation (front-end)

  • A sign is observable within the sensor field of view, and the simulation uses representative camera range and recognition, false-positive, and false-negative rates.The stated settings are up to 100 m, 95% semantic recognition, around 2% false positives, and around 10% false negatives.Source: §4.1 · pp. 3–4

4.1.1 Traffic Sign Detection Generation

  • Position errors combine sensor-setting noise shared across a run with independently sampled measurement noise, while failed semantic recognition assigns a random valid alternative type.The two-level position model produces errors of a few meters; the simulation also lists visual similarity, occlusion, viewing angle, and weather as misclassification factors.Source: §4.1.1 · p. 4

4.1.2 Traffic Sign False Positives and False Negatives

  • False negatives randomly remove detected signs, while false positives are generated by a Poisson process and placed uniformly within the detection cone.The false-positive count depends on the false-positive rate and cone area.Source: §4.1.2 · p. 4

4.1.3 Temporal Tracking and Fusion

  • Temporal tracking associates detections across frames within a 2.0 m gate and averages spatially fused positions to reduce measurement noise.Position uncertainty is quantified using the standard deviation of frame-level positions around the mean.Source: §4.1.3 · pp. 4–5

4.2 Multi-Vehicle aggregation (back-end)

  • The back end aggregates independent vehicle observations using the detected sign's semantic class and location to estimate its map position and class.The paper describes aggregation as a two-step process and compares clustering methods to select a strategy.Source: §4 · p. 3, §4.2 · p. 5

4.2.1 Semantic filtering

  • Semantic management groups detections by sign type and can be applied either before or after spatial clustering.The examples include SPEED LIMIT and YIELD signs.Source: §4.2.1 · p. 5

4.2.2 Spatial Clustering

  • The study evaluates DBSCAN, K-Means, K-Medoids, and HDBSCAN using Euclidean distances in ENU coordinates.The methods differ in density-based, centroid-based, medoid-based, and hierarchical density-based clustering; K-Means is sensitive to outliers, while K-Medoids uses actual detections as centers at greater computational cost.Source: §4.2.2 · p. 5

HDBSCAN (Hierarchical Density-Based Spatial Clustering)

HDBSCAN performed differently depending on whether semantic-first filtering was used: it over-segmented with that filtering but performed competitively without it.

HDBSCAN (Hierarchical Density-Based Spatial Clustering)

5. EVALUATION METRICS AND QUALITY MEASURES

The section distinguishes conventional metrics for map completeness and localization error from GOSPAM, a single dissimilarity measure that combines position accuracy with penalties for missing and phantom features.

5.1 Usual metrics

  • Evaluation against a reference map labels matched points TP, other evaluated points FP, and unmatched reference points FN.Source: §5 · p. 5, §5.1 · p. 5
  • F1 combines precision and recall as their harmonic mean, representing a compromise between the two.Source: §5.1 · p. 5
  • MSEM averages squared error over matched map-reference pairs, measuring localization accuracy without accounting for false positives or false negatives.Source: §5.1 · pp. 5–6
  • Conventional map metrics do not provide a single numerical value for overall map quality.Source: §5.1 · p. 6

5.2 GOSPAM

  • GOSPAM combines matched-feature position accuracy with an (FN+FP)/TP penalty, using the map-matching gate c.The penalty compares incorrect or missing map elements with correct ones.Source: §5.2 · p. 6
  • GOSPAM is a single map-quality indicator: lower values mean greater map similarity, with zero for identical maps.Source: §5.2 · p. 6

6. RESULTS

The results compare clustering strategies and show that mapping quality improves as more vehicles contribute, with gains tapering at larger fleet sizes. DBSCAN with semantic-first grouping performed best in the clustering comparison, while GOSPAM captures continued spatial-accuracy gains.

6.1 Clustering strategy selection

  • At 50 vehicles, DBSCAN with semantic-first grouping achieved the best overall performance: F1-score 90.6%, precision 93.5%, recall 87.9%, and GOSPAM 0.756 m.The comparison used 33 ground-truth signs; K-Means and K-Medoids were given the cluster count under favorable conditions.Source: §6.1 · p. 6, Table 2 · p. 7

6.2 Influence of fleet size on mapping quality

  • F1-score rises from approximately 0.75 with five vehicles to about 0.95 with more than 25, then appears to plateau.The authors interpret the increase as more reliable aggregated detections as observations accumulate.Source: §6.2 · p. 6, Figure 3 · p. 7
  • After data from 10 vehicles has been collected, uncertainty no longer changes significantly.GOSPAM supports calculation of a confidence interval.Source: §6.2 · p. 7

6.3 Discussion

  • Semantic grouping before spatial clustering prevents nearby signs of different types from being merged, a relevant issue where signs share a structure.The discussion presents GOSPAM as a measure for comparing data-processing approaches and selecting a mapping architecture.Source: §6.3 · p. 7

6.2 Influence of fleet size on mapping quality

  • As fleet size increases, true positives rise slightly while false positives and false negatives decline, with these detection counts stabilizing beyond roughly 30 vehicles.The authors state that most signs have then been sufficiently observed and confirmed.Source: §6.2 · pp. 6–7, Figure 4 · p. 7
  • GOSPAM decreases most from 5 to about 30 vehicles, with slower improvement thereafter and diminishing returns once observation density is sufficient.Beyond approximately 30 vehicles, the reported improvement is limited to sign-location accuracy; the authors suggest an application-dependent fleet optimum relative to collection and processing costs.Source: Figure 5 · p. 7, §6.2 · p. 7, §6.3 · p. 7

7. CONCLUSION

The paper reports that approximately 30 journeys were sufficient for high-quality traffic-sign mapping in the considered map tile, and that DBSCAN with semantic-first filtering performed best among four clustering algorithms. Further evaluation across diverse road environments and mixed fleets is proposed to assess how general the observed trends are.

7. CONCLUSION

  • Approximately 30 journeys appeared sufficient for high-quality traffic-sign mapping within the considered map tile.The result is specific to the experimental scenario.Source: §7 · p. 8
  • DBSCAN with semantic-first filtering was the best approach in a systematic comparison of four clustering algorithms.Source: §7 · p. 8
  • The fleet simulation system combines realistic sensor modeling, aggregation, and clustering-algorithm evaluation for crowdsourced traffic-sign mapping.Tests used a route traveled by an experimental vehicle and a real HD map in a limited area.Source: §7 · p. 8
  • Evaluations across urban, highway, and rural environments are proposed to assess the generality of the observed trends; future work will also examine mixed-sensing fleets.Source: §7 · p. 8

Related papers

Loading 2609.29198v1…