PaperlayerView exact revision on arXiv

Source-linked AI summary

Multi-Modal Traffic Sign Detection with Semantic Attributes for Autonomous Driving

Meda Lazar, Sourab Sridhar, Shashwata Gupta, Alexandra Tripcea, Varun Ravi, Senthil Yogamani

arXiv:2608.20874v1cs.CVcs.RO

TL;DR

Traffic-sign detection remains difficult across regions, high ranges, and environmental conditions, despite its role in regulatory compliance and road safety. The paper combines camera and LiDAR with nonlinear-perspective tracking and reports 0.49% Object Miss Ratio across 2,500+ hours of driving data.

  • Problem

    The gap is robust traffic-sign detection across unseen regions, high ranges, and difficult conditions, despite its importance to regulatory compliance and road safety.

    Source: §I · p. 1

  • Method

    The framework fuses LiDAR depth and intensity with camera features, uses dual motion-model tracking for nonlinear perspective, and classifies operational sign attributes.

    Source: §I · p. 2, §VII · p. 17

  • Results

    0.49% Object Miss Ratio across 2,500+ hours of driving data was reported on a dataset spanning diverse global regions.

    Source: §I · p. 2

  • Takeaways & Limitations

    The conclusion is region-invariant performance across the dataset's 60+ countries, alongside incremental Recall and Precision improvements over a constant-velocity baseline.

    Source: §VII · p. 17

    The initial 3D detection approach did not scale to traffic signs, with calibration drift or millisecond-level desynchronization causing proposal misalignment and precision loss.

    Source: A. Initial 3D Detection Approach and… · p. 5

Abstract

from arXiv · show

Reliable traffic sign detection is a prerequisite for the global deployment of autonomous driving systems, where regulatory compliance and road safety depend on perceiving signs correctly across regions, ranges, and weather conditions. Despite recent progress, vision-based methods continue to face three fundamental limitations: poor cross-regional generalization due to high diversity across countries, degraded performance on small-object detection at long ranges (traffic signs occupy as little as $10{\times}10$ pixels at 200m), and fragile temporal tracking under the strongly non-linear perspective distortion that occurs as a vehicle approaches a sign. In this paper, we address the problem of robust, long-range, region-agnostic traffic sign perception by combining camera and Light Detection and Ranging (LiDAR) sensing. We present a multi-modal detection framework whose Intensity-Aware Deformable Fusion module aligns retro-reflective LiDAR cues with camera features, anchoring detection on geometric invariants rather than region-specific visual appearance. We further introduce a dual motion-model tracker that explicitly accounts for non-linear perspective transformations during vehicle approach, substantially improving temporal consistency over linear motion assumptions. Additionally, we develop a semantic attribute classification pipeline that estimates occlusion level, readability, sign embeddedness, and road relevance, providing actionable context to downstream planning. Extensive evaluation on our dataset, spanning 60+ countries and 2,500+ hours of driving data, shows that the proposed pipeline achieves an Object Miss Ratio (OMR) of 0.49% across 221,068 evaluation sequences, demonstrating globally generalizable traffic sign perception in commercial-grade autonomous driving systems.

I. INTRODUCTION · II. RELATED WORK · A. Vision-Based Traffic Sign Detection and Benchmarks

The paper motivates global traffic-sign perception by vision-only systems’ weaknesses in regional generalization, adverse conditions, long-range detection, and tracking, and proposes LiDAR-camera fusion, dual-motion tracking, and semantic attributes. It positions this work against camera-focused methods and benchmarks with limited geographic or multimodal coverage.

I. INTRODUCTION

  • The motivating gaps are region-specific appearance overfitting, camera fragility under adverse conditions, and loss of visual features at extreme ranges.LiDAR provides metric depth and geometric structure, while retroreflective signs yield high intensity returns even when cameras degrade.Source: §I · pp. 1–2
  • LiDAR-camera fusion uses geometric depth and reflectance to anchor region-invariant detection and extend the effective range to 200 m.LiDAR supplies structural geometry and retroreflective intensity cues that complement camera features, including for low-pixel-count targets.Source: §I · p. 2
  • A dual motion model handles linear and non-linear perspective regimes, improving temporal stability and reducing track fragmentation versus standard linear motion models.The motivation is the rapid scale change and perspective distortion as a vehicle approaches a sign.Source: §I · p. 2
  • The semantic attribute pipeline estimates occlusion, readability, relevance, and embeddedness so downstream planning can filter non-actionable detections.Source: §I · p. 2
  • 0.49% Object Miss Ratio across 2,500+ hours of driving data supports the pipeline’s large-scale validation across diverse global regions.The Qualcomm internal dataset was collected across multiple continents and evaluated against manual ground-truth annotations.Source: §I · p. 2

A. Vision-Based Traffic Sign Detection and Benchmarks

  • Camera-only detection remains dominant: Wang et al. and Zhang et al. target efficient real-time detectors, while other work addresses small signs with attention or vehicle-mounted systems.Prior work and surveys also report persistent failures under adverse weather, motion blur, and varying spectral conditions,,,, [16].Source: A. Vision-Based Traffic Sign Detection… · p. 2
  • GTSDB, TT100K, and BelgiumTS are region-specific, while BDD100K and Mapillary [21] broaden coverage but remain image-only; Zenseact covers 14 European countries with far sparser sign annotations.CLFNet fuses visual streams but does not use physical sensing modalities beyond the camera.Source: A. Vision-Based Traffic Sign Detection… · pp. 2–3

B. LiDAR-Camera Fusion for Object Detection … 1) The Qualcomm Dataset: Scale and Advantages:

The paper motivates traffic-sign-specific camera–LiDAR fusion and non-linear tracking, then distinguishes its approach and dataset from prior work. The dataset offers broad geographic coverage, large scale, synchronized 2D/3D annotations, and labels extending to 200 m.

B. LiDAR-Camera Fusion for Object Detection

  • Prior camera–LiDAR fusion methods [25]–[28] mainly target dense object classes, leaving extreme small-object traffic signs at 200 m comparatively understudied.PointPainting decorates LiDAR points with camera semantics, TransFusion addresses sensor misalignment, and BEVFusion uses a shared bird’s-eye-view representation.Source: B. LiDAR-Camera Fusion for Object… · p. 3

C. Tracking-by-Detection with Non-Linear Motion

  • SORT –[36] trackers generally assume constant-velocity image-plane motion, which poorly models stationary signs’ non-linear scale and position changes during vehicle approach.The discrepancy is especially pronounced in the final tens of meters before passing a sign.Source: C. Tracking-by-Detection with… · p. 3

D. Key Differences

  • The framework uses LiDAR-invariant detection and retro-reflective, misalignment-tolerant 2D fusion, constant-acceleration/jerk tracking, and classification of four semantic attributes.The attributes are occlusion, readability, embeddedness, and relevance.Source: D. Key Differences · p. 3

A. Dataset Description and Comparison

  • The evaluation uses a proprietary Qualcomm dataset because existing academic benchmarks often lack the geographic diversity and multimodal primitives needed for global deployment.Source: A. Dataset Description and Comparison · p. 3

1) The Qualcomm Dataset: Scale and Advantages:

B. Evaluation Process and Metrics · 1) Detection Metrics: · 2) Tracking and Attribute Metrics:

The evaluation follows the COCO protocol for detection and assesses the detector, tracker, and semantic attribute classifier. Detection uses AP across IoU thresholds and object scales, while tracking and attribute assessment includes accuracy.

B. Evaluation Process and Metrics

1) Detection Metrics:

  • AP averages results across IoU thresholds from 0.50 to 0.95 in 0.05 increments, following the Microsoft COCO benchmark [40].Source: 1) Detection Metrics · p. 4
  • The detection metrics also include AP50 and AP75, measuring precision at IoU thresholds of 0.50 and 0.75.Source: 1) Detection Metrics · p. 4
  • Scale-specific AP reports APs, APm, and APl for small (< 322 px), medium (322–962 px), and large (> 962 px) objects.Source: 1) Detection Metrics · p. 4

2) Tracking and Attribute Metrics:

3) End-to-End Efficacy: … 1) Multi-Modal Depth and Intensity Map Generation:

The section defines Object Miss Ratio for evaluation and explains why the approach moves from 3D detection to 2D multimodal fusion, using temporally accumulated LiDAR depth and intensity maps to support long-range sign detection.

3) End-to-End Efficacy:

  • Object Miss Ratio (OMR) measures undetected signs against manual annotations, with lower values indicating better performance.It ranges from 0% for no missed annotated objects to 100% for a complete miss.Source: 3) End-to-End Efficacy · pp. 4–5

IV. PROPOSED MULTI-MODAL DETECTION AND TRACKING

  • The initial 3D-centric approach encountered scalability and reliability issues for traffic-sign detection, prompting development of a 2D multimodal framework.Source: §IV · p. 5

A. Initial 3D Detection Approach and Limitations

  • DAL-Large with ResNet-50 and Lk3D was trained on 10-frame sweeps for 24 epochs and localized large obstacles accurately, but failed to scale to traffic signs.Source: A. Initial 3D Detection Approach and… · p. 5
  • Beyond 100 m, multi-sweep LiDAR yields only a handful of returns on a sign, making stable 3D bounding-box regression difficult without substantial visual support.Source: A. Initial 3D Detection Approach and… · p. 5
  • Subdegree calibration drift or millisecond-level desynchronization detached 3D proposals from imagery, motivating 2D fusion with LiDAR as supplemental features rather than a rigid constraint.The authors state that a 2D detector with a LiDAR-derived depth attribute is sufficient for their driving use cases.Source: A. Initial 3D Detection Approach and… · p. 5

B. Proposed 2D Multi-Modal Detection Framework

  • The proposed pipeline fuses image features with LiDAR-derived depth and intensity maps, then tracks detections and classifies semantic attributes.The architecture uses an Intensity-Aware Deformable Fusion module, a multi-scale detection head, a dual motion-model tracker, and an attribute classifier.Source: Figure 3 · p. 5, B. Proposed 2D Multi-Modal Detection… · p. 5

1) Multi-Modal Depth and Intensity Map Generation:

  • Sequential LiDAR frames are accumulated in camera coordinates using ego-vehicle odometry, then projected into depth and intensity maps; median filtering suppresses depth noise and max-pooling preserves peak retro-reflectivity.The accumulated points increase density for detecting small signs at 200 m, while intensity provides a geometric anchor for the visual head.Source: 1) Multi-Modal Depth and Intensity Map… · pp. 5–6

2) Intensity-Aware Deformable Fusion: … 1) Dual Motion Models:

The method aligns camera and LiDAR through intensity-constrained deformable offsets, then detects and tracks signs using multi-scale features and complementary motion models.

2) Intensity-Aware Deformable Fusion:

  • Intensity-constrained offset prediction uses joint geometric-intensity representations to dynamically align modalities and pull RGB features toward high-reflectivity regions.This creates a self-correcting alignment mechanism that addresses rigid fusion’s failure under calibration drift.Source: 2) Intensity-Aware Deformable Fusion · p. 6
  • Geometric attention checks LiDAR planar consistency while intensity attention emphasizes sign reflectivity, jointly suppressing distractions such as metallic barriers.The cross-verification assigns high weights only to features that are structurally consistent and semantically relevant.Source: 2) Intensity-Aware Deformable Fusion · p. 6

3) Backbone and Multi-Scale Detection Head:

  • A SwinTransformer backbone [43] builds multi-scale fused features for a transformer-based anchor-driven detection head trained with Co-DETR-style collaborative hybrid assignment [44].The hierarchical features capture fine-grained sign details and broader context, supporting bounding-box regression at extreme distances.Source: 3) Backbone and Multi-Scale Detection… · p. 6

C. Tracking Algorithm

  • The tracking module pairs a dual motion-model Kalman filter with multi-stage data association to maintain identity despite non-linear scale and perspective changes during vehicle approach.It addresses limitations of trackers such as BoostTrack, [36] and OC-SORT [32], which often rely on linear motion or constant-velocity assumptions [30],.Source: C. Tracking Algorithm · p. 6

1) Dual Motion Models:

  • The third-order constant-jerk filter captures smooth higher-order trajectory curvature and subtle, accelerating perspective shifts for distant signs.Its state tracks bounding-box center position, velocity, acceleration, and jerk, plus width, height, and their first-order rates of change.Source: 1) Dual Motion Models · p. 6
  • The second-order constant-acceleration filter captures rapid scale expansion and large pixel displacements in close-range signs, where looming makes constant-velocity assumptions inadequate.Both higher-order filters run in parallel on every track and their predictions are fused, rather than activating only in separate distance regimes.Source: 1) Dual Motion Models · p. 6
  • Enabling the second-order filter and then the third-order filter each yields incremental improvement over the baseline tracker in the cumulative ablation reported in Table III.The complementary predictions contribute throughout each sign’s tracked lifetime.Source: 1) Dual Motion Models · p. 6

2) Multi-Stage Data Association: · 3) Fusion and Trajectory Synthesis:

The method improves association using a composite spatial-and-shape cost, then reconciles second- and third-order track predictions to produce temporally stable sign tracks for semantic attribute assignment.

2) Multi-Stage Data Association:

  • The composite matching cost S combines spatial consistency and shape similarity to improve association when blur or tiny distant targets undermine IoU matching.The shape term penalizes unrealistic aspect-ratio changes between predicted tracks and candidate detections.Source: 2) Multi-Stage Data Association · p. 6
  • The matching-cost weights were grid-searched on a held-out validation split to maximize tracking F1, yielding w1=0.25, w2=0.25, and w3=0.5.The multi-metric approach addresses cases where detection confidence alone may be insufficient.Source: 2) Multi-Stage Data Association · p. 7

3) Fusion and Trajectory Synthesis:

  • At each frame, second- and third-order filters independently update state estimates for every active track.Their predictions are subsequently reconciled through duplicate suppression.Source: 3) Fusion and Trajectory Synthesis · p. 7
  • When predicted boxes overlap, duplicate suppression retains the estimate with the lower posterior covariance trace; otherwise, it treats predictions as independent observations of the same sign.The lower covariance trace is interpreted as the more confident estimate.Source: 3) Fusion and Trajectory Synthesis · p. 7
  • The unified track is propagated forward to support consistent assignment of semantic attributes.Figure 5 shows tracklets across four sequences, including occlusions, distant signs, and substantial perspective variations.Source: 3) Fusion and Trajectory Synthesis · p. 7
  • Figure 5 presents three representative frames from each of four distinct sequences, with green boxes marking detections and pink labels showing tracking IDs.The examples depict challenging edge cases including occlusions, distant signs, and significant perspective variations across frames.Source: 3) Fusion and Trajectory Synthesis · p. 7

V. SEMANTIC ATTRIBUTE CLASSIFICATION · A. Occlusion Attribute

The system supplements 2D sign localization and classification with semantic attributes that provide operational context for downstream planning. Its occlusion classifier distinguishes physical visibility limits from detection failures and supports more reliable benchmarking in cluttered scenes.

V. SEMANTIC ATTRIBUTE CLASSIFICATION

  • The proposed semantic layer augments 2D localization and classification with sign attributes, a feature rarely modeled jointly in public datasets.It decouples spatial localization from a composite attribute vector.Source: §V · p. 7
  • Each detection is represented as (B, A): a 2D image-plane bounding box and attributes for occlusion, readability, embeddedness, and road relevance.Embeddedness captures the sign’s hierarchical relationship to parent signs; relevance reflects significance relative to the ego-vehicle’s path.Source: §V · pp. 7–8
  • Attribute predictions let downstream planning prioritize actionable detections and ignore irrelevant or unreadable signs, reducing its input space and computational burden.The paper presents this as a filtering capability not found in traditional vision-only systems.Source: §V · p. 8

A. Occlusion Attribute

  • The occlusion attribute distinguishes detection failures from cases where environmental obstructions make sign detection physically impossible.Urban signs may be obstructed by dynamic agents or static infrastructure.Source: A. Occlusion Attribute · p. 8
  • For occlusion classification, the system uses DINOv2 object-centric representations and LoRA to adapt the backbone efficiently to partially visible signs.The paper describes the representations as generalizing well to occluded geometries.Source: A. Occlusion Attribute · p. 8
  • The three-stage pipeline crops and resizes tracked-box regions, adapts frozen DINOv2 layers with trainable low-rank matrices, and classifies five occlusion levels from 0% to 100% in 25% intervals.Source: A. Occlusion Attribute · p. 8
  • Filtering instances whose defining sign features are compromised makes 2D detection benchmarks reflect model performance rather than visibility limits in cluttered scenes.The paper says this provides a more accurate assessment of system reliability.Source: A. Occlusion Attribute · p. 9

B. Readability Attribute · C. Embedded Attribute · D. Relevance Attribute

The three attributes add semantic filtering for readable sign content, hierarchical identification of embedded signs, and relevance filtering that limits downstream processing to signs applicable to the ego vehicle. Their rules combine image and LiDAR cues, physical relationships, and spatial, temporal, and semantic checks.

B. Readability Attribute

  • Readability distinguishes readable, unreadable, and back-of-sign detections so the system can prioritize interpretable content and suppress low-confidence or non-actionable signs.Unreadable signs have visible faces compromised by blur, weathering, or partial occlusion; readable signs have distinct, unobstructed symbols and borders. Back-facing signs are spatially valid but excluded from actionable signs.Source: B. Readability Attribute · p. 9
  • Readability uses patch-based classification with a DINOv2 backbone and LoRA adapters, balanced training across three classes, and LiDAR intensity to correct some front-facing signs classified as back-of-sign.Back-of-sign detections are excluded from actionable signs because the reverse side does not produce high LiDAR intensity values.Source: B. Readability Attribute · p. 9

C. Embedded Attribute

  • The embedded attribute identifies signs nested within larger panels, addressing redundant detections and misrouted downstream logic.It models the hierarchical relationship explicitly to stabilize scene representation and reduce high-priority evaluation overhead.Source: C. Embedded Attribute · p. 9
  • A sign is embedded only when its box is contained by a parent sign, its 3D centroid is within an adaptive distance threshold, and its yaw is consistent.These conjunctive constraints distinguish physical integration from perspective overlap.Source: C. Embedded Attribute · p. 9
  • Once detected in any frame, an embedded attribute is propagated across the object’s track as a static, frame-invariant property.This temporal anchoring prevents attribute flicker from occlusion or noise and provides a stable signal for planning and mapping.Source: C. Embedded Attribute · p. 9

D. Relevance Attribute

  • Relevance filtering suppresses signs outside the ego vehicle’s driving task, including signs on adjacent frontage roads and distant background signs.The pipeline integrates spatial, temporal, and semantic constraints to keep irrelevant detections from downstream decision-making modules.Source: D. Relevance Attribute · p. 9
  • The relevance pipeline filters signs by drivable-area proximity, a 100 m operational horizon, facing direction, and whether their directive applies to the ego vehicle’s trajectory.The detector still reports signs out to 200 m; the tighter 100 m relevance horizon corresponds to roughly 3.3 seconds of look-ahead at 30 m/s.Source: D. Relevance Attribute · p. 9

VI. RESULTS … C. Occlusion Classification

The detector improves Qualcomm-dataset detection over the Zenseact-initialized baseline, while higher-order tracking and task-adapted occlusion classifiers improve temporal consistency and classification. DINOv2 with LoRA performs strongly in binary and five-class occlusion evaluation, though 75% occlusion remains difficult.

C. Occlusion Classification

  • The occlusion study compared frozen DINOv2, LoRA-adapted ViT-Base and DINOv2, and ViT-Base trained from scratch under binary and five-class protocols.Frozen training used a 4×10^-3 learning rate, LoRA settings varied by backbone, and scratch ViT-Base trained for 250 epochs.Source: C. Occlusion Classification · pp. 10–11

A. Detector

  • AP rose 0.27, from 0.38 to 0.65, and AP75 rose 0.34, from 0.39 to 0.73, for the Qualcomm-trained detector versus the baseline.APs improved from 0.27 to 0.51 and APm from 0.54 to 0.80. The detector split contains over 16,000 training and nearly 7,000 test sequences; objects are evaluated up to 200 m.Source: A. Detector · p. 10, §VI · p. 10, Table II · p. 11

B. Tracker

  • Recall reached 0.74 with the third-order tracker, exceeding the single-frame detector’s 0.73 while maintaining 0.71 precision.The model adds jerk to account for nonlinear perspective shifts, supporting association from 200 m to the point of passing.Source: B. Tracker · p. 10, Table III · p. 11
  • OC-SORT and BoostTrack++ raised precision to 0.70 and 0.72 but lowered recall to 0.68 and 0.69; interpolation raised recall to 0.72, and the baseline tracker balanced precision and recall at 0.71.The second-order model recovered detector recall at 0.73 while maintaining 0.71 precision.Source: B. Tracker · p. 10, Table III · p. 11

C. Occlusion Classification

  • DINOv2 + LoRA achieved 0.92 recall and 0.93 precision for occluded signs, plus 0.74 recall and 0.72 precision for non-occluded signs in binary classification.ViT-Base + LoRA improved non-occluded precision from the frozen model’s 0.47 to 0.64 and achieved 0.89 recall for occluded signs.Source: Table IV · p. 11, C. Occlusion Classification · p. 11
  • At 75% occlusion, DINOv2 + LoRA raised recall from 0.13 to 0.36 versus frozen DINOv2, and its 0%-occlusion precision rose from 0.47 to 0.72.These five-class results indicate that adapting the foundation model helps recover performance as occlusion increases.Source: Table V · p. 12, C. Occlusion Classification · p. 12
  • With identical LoRA adaptation, DINOv2 exceeded ViT-Base recall at 0% occlusion (0.74 vs. 0.71) and 100% occlusion (0.59 vs. 0.51).Source: C. Occlusion Classification · p. 12
  • The 75% occlusion class remained hardest, with precision between 0.35 and 0.56 across backbones, amid similarity to the 50% and 100% classes.DINOv2 + LoRA offered the best recall-oriented trade-off; Figure 6 shows qualitative examples of predicted occlusion levels.Source: C. Occlusion Classification · p. 12, Figure 6 · p. 12

D. Readability Classification

The readability analysis compares vision-only and camera–LiDAR fusion, finding that LiDAR intensity cues help distinguish sign backs from readable fronts and inform which signs should affect the ego-path.

D. Readability Classification

  • Camera + LiDAR raised BackOfSign precision from 0.81 to 0.83 and achieved 0.93 recall for Readable using DINOv2 and LoRA.The comparison evaluates the readability attribute against a vision-only approach.Source: D. Readability Classification · p. 13
  • LiDAR intensity addresses false positives and temporal instability by distinguishing retroreflective sign fronts, which return higher intensity, from matte or metallic backs.The intensity threshold was chosen empirically.Source: D. Readability Classification · p. 13
  • Readability is categorized as Readable, NotReadable, or BackOfSign to help determine which signs should influence the vehicle’s ego-path.Source: D. Readability Classification · p. 13

E. Embedded Classification · F. Relevance Classification · G. Comparative Evaluation of 3D Detector on Zenseact Dataset

The embedded classifier performs reliably and supports annotation checks, while adding a lane-polygon constraint improves relevance classification. The 3D detector comparison reports an advantage over the Zenseact LiDAR-only baseline, and Qualcomm results show gains from multi-sweep aggregation alongside substantial long-range performance decay.

E. Embedded Classification

  • 0.99 recall and 0.99 precision are achieved by the embedded-attribute classifier on the full evaluation set.The classifier also helps identify annotation artifacts, including apparent embedded configurations caused by perspective overlap or sign placement; it correctly resolves some visually suggestive cases as NotEmbedded.Source: E. Embedded Classification · p. 13

G. Comparative Evaluation of 3D Detector on Zenseact Dataset

  • At least 0.22 AP improvement over the LiDAR-only baseline is reported for our Centerpoint detector on Zenseact.For direct comparison, the Zenseact baseline’s front-facing and back-facing AP values are averaged, while our method reports a single AP across sign types.Source: Table IX · p. 14, G. Comparative Evaluation of 3D… · p. 14

F. Relevance Classification

  • Adding the lane-polygon point-inside constraint at 5 m raises “Relevant” precision from 0.54 to 0.64 and accuracy from 0.71 to 0.76, with recall unchanged.The constraint is added to a lane-distance check that marks a sign relevant when it is within 5 m of the nearest lane, reducing false-positive Relevant assignments.Source: F. Relevance Classification · pp. 13–14, Table VIII · p. 14

G. Comparative Evaluation of 3D Detector on Zenseact Dataset

  • Multi-sweep accumulation raises overall AP from 0.55 to 0.60 and AP at 150–200 m from 0.14 to 0.20 on Qualcomm.The multi-sweep configuration aggregates 10 future point-cloud sweeps.Source: G. Comparative Evaluation of 3D… · pp. 14–15, Table X · p. 15
  • AP falls from 0.68 at close range to 0.20 at 200 m, where signs typically occupy only 10×10 pixels and sparse returns make precise 3D box regression difficult.The passage relates this distance decay to scale-based metrics in the 2D image plane.Source: G. Comparative Evaluation of 3D… · p. 15
  • Absolute AP differs between the Zenseact and Qualcomm results because of differences in geographic coverage, sign distributions, annotation density, and training-set scale and diversity.The Zenseact-trained model uses a smaller, geographically narrower distribution than the Qualcomm-trained model.Source: G. Comparative Evaluation of 3D… · p. 15

H. Comparison of Camera-Only and Camera+LiDAR 2D Detector · I. Evaluation of effect of weather, lighting condition and road type · VII. CONCLUSIONS

The sections report complementary camera–LiDAR gains and evaluate detection robustness across lighting, road types, and weather, alongside the framework’s tracking and semantic-attribute contributions.

H. Comparison of Camera-Only and Camera+LiDAR 2D Detector

  • APs for small objects improved by +0.04, from 0.47 to 0.51, with camera+LiDAR versus camera-only detection.The text also reports smaller absolute improvements for medium objects (+0.02) and large objects (+0.01); depth and intensity provide complementary geometric and reflectivity cues for distant, pixel-sparse signs.Source: H. Comparison of Camera-Only and… · pp. 15–16

I. Evaluation of effect of weather, lighting condition and road type

  • The Qualcomm evaluation covers diverse weather, road types, and illumination across 60+ countries, using manually verified data reviewed and corrected by trained annotators.The authors describe the dataset as capturing long-tail global driving variability, including varied sign shapes, LED displays, and multilingual content.Source: §I · p. 16
  • 0.49% OMR across 221,068 evaluation sequences over 2,500+ driving hours demonstrates the pipeline’s overall performance.The conclusion reports validation across 60+ countries, with stability across illumination, road type, and weather regimes.Source: §I · p. 16, Table XII · p. 16, §VII · p. 17
  • Night OMR was 0.17%, versus 0.67% during the day, while daytime sequences made up 145,412 of the evaluation sequences.The authors attribute the night result to sign retroreflection under headlights and more frames per sign at lower speeds, but caution that verifying distant signs at night is difficult, making the night value a lower bound.Source: §I · p. 16, Table XII · p. 16
  • Rural and highway OMRs rose to 0.66% and 1.16%, respectively, while Urban sequences had the best performance.The authors attribute Urban performance partly to lower speeds and greater temporal redundancy; higher speeds and long-range viewing reduce observation time and can destabilize temporal association and cross-sensor consistency.Source: §I · p. 16, Table XII · p. 16
  • Fog had the highest OMR at 0.72%, while Moderate Rain was 0.34% and Heavy Snow 0.27%, below Clear weather at 0.52%.The authors attribute lower precipitation-condition OMRs to slower driving and improved temporal consolidation; fog affects both camera and LiDAR sensing. Thunderstorm and hail results are indicative rather than conclusive because those categories have few sequences, including 27 thunderstorm and 1 hail sequence.Source: §I · pp. 16–17, Table XII · p. 16

VII. CONCLUSIONS

  • The framework fuses LiDAR depth and intensity with camera features to support region-invariant performance across the dataset’s 60+ countries.Its dual motion-model Kalman filter handles non-linear perspective transformations during vehicle approach and yields incremental Recall and Precision improvements over a constant-velocity baseline.Source: §VII · p. 17
  • The semantic attribute pipeline estimates occlusion, readability, embeddedness, and relevance to provide downstream autonomous-driving tasks with filtering context.These attributes extend the framework beyond sign localization.Source: §VII · p. 17

Related papers

Loading 2608.20874v1…