PaperlayerView exact revision on arXiv

Source-linked AI summary

HDNET: Exploiting HD Maps for 3D Object Detection

Bin Yang, Ming Liang, Raquel Urtasun

arXiv:2012.11704v1cs.CV

TL;DR

3D object detection must handle occlusion and sparse long-range LiDAR, while HD maps are underused by perception systems. HDNET fuses map geometry and semantics with BEV LiDAR and predicts those priors online when maps are unavailable. Across mapped and unmapped benchmarks, it consistently outperforms baseline detectors while running at over 20 frames per second.

  • Problem

    3D object detection remains challenged by occlusion and long-range LiDAR sparsity, while HD maps are largely ignored by perception systems.

  • Method

    HDNET builds a single-stage BEV detector that fuses geometric and semantic HD-map priors with LiDAR and predicts map features online from a single sweep when needed.

  • Results

    HDNET consistently outperforms baseline detectors with and without HD maps, including 2.87% AP gain on KITTI and 2.42%, 3.43% and 5.49% gains across TOR4D ranges.

  • Takeaways & Limitations

    The complete map-aware framework operates at over 20 frames per second across mapped and unmapped detection settings.

Abstract

from arXiv · show

In this paper we show that High-Definition (HD) maps provide strong priors that can boost the performance and robustness of modern 3D object detectors. Towards this goal, we design a single stage detector that extracts geometric and semantic features from the HD maps. As maps might not be available everywhere, we also propose a map prediction module that estimates the map on the fly from raw LiDAR data. We conduct extensive experiments on KITTI as well as a large-scale 3D detection benchmark containing 1 million frames, and show that the proposed map-aware detector consistently outperforms the state-of-the-art in both mapped and un-mapped scenarios. Importantly the whole framework runs at 20 frames per second.

1 Introduction

3D object detection remains difficult under occlusion and long-range LiDAR sparsity. HDNET addresses this by fusing geometric and semantic HD-map priors with BEV LiDAR, while predicting maps online when unavailable.

  • 3D object detection is fundamental to perception, but occlusion and long-range LiDAR sparsity remain major challenges.
  • HDNET fuses rasterized geometric and semantic HD-map information with LiDAR in a BEV single-stage detector.The BEV representation supports efficient inference while retaining metric space.
  • HDNET also predicts map geometry and semantics from a single online LiDAR sweep when HD maps are unavailable.
  • 2.42%, 3.43% and 5.49% AP gains are achieved on TOR4D over 0-70 m, 30-50 m and 50-70 m, respectively, with HD maps available.
  • 2.87% AP gain is achieved on KITTI using a map predictor trained on another continent, while the detector runs at 20 frames per second.

2 Related Work

Prior 3D detectors use LiDAR projections for efficient detection, while map-based perception methods have explored geographic, geometric, and semantic priors with varying computational or modeling limitations.

  • Dense sliding-window detectors search 3D space but incur expensive computation because sparse LiDAR leaves many evaluated proposals unsupported.
  • Range-view and BEV projections improve point-cloud efficiency, with PIXOR and FAF demonstrating strong speed and accuracy using BEV alone.
  • Map-based methods use geographic, geometric, and semantic priors for tasks including joint perception and road segmentation.
  • Existing 3D detectors often model ground as a plane, which can be inaccurate on curved roads.

3 Exploiting HD Maps for 3D Object Detection

HDNET incorporates HD-map geometry and semantics into a BEV LiDAR representation for single-stage dense detection, and uses input-level dropout to remain effective when maps are unavailable or noisy.

  • HDNET bridges map-based planning and perception by exploiting semantic and geometric priors from offline-built or online-estimated HD maps.
  • Input Representation: BEV projects LiDAR into a compact representation that enables efficient inference.
  • Input Representation: Ground-relative height features reduce LiDAR translation variance caused by road slope, including a 1.22-meter Z offset at 70 meters for a 1-degree slope.The method queries map ground height z_0i at each (x_i, y_i) and replaces absolute z_i with relative distance z_i − z_0i.
  • Input Representation: The discretized BEV representation combines occupancy and intensity features, producing H/dH + 3 channels at L/dL by W/dW spatial resolution.
  • Input Representation: A rasterized binary road mask is concatenated with the discretized LiDAR representation as a semantic prior.
  • Network Structure: A fully convolutional detector uses a multi-scale backbone and a header that outputs pixel-wise dense object estimates.The output encodes confidence, orientation, center offsets, and object size; dense predictions are efficient and permit 100% maximum recall.
  • Network Structure: Data dropout randomly removes the semantic prior to reduce overfitting and improve robustness when maps are unavailable or noisy.

4 Online Estimation of HD Maps

The paper estimates geometric and semantic HD-map priors online from a single LiDAR sweep, producing BEV features that integrate with the detector when pre-built maps are unavailable.

  • Online map estimation: Online estimation predicts BEV geometric and semantic priors from a single LiDAR sweep rather than constructing dense 3D HD maps.The estimated features are designed for seamless integration into the existing detector framework.
  • Online map estimation: Two separate U-Net networks estimate ground height and segment roads while retaining low-level details for high-resolution predictions.Both tasks use the same U-Net structure.
  • Geometric prior: The ground estimator predicts a height value at every BEV location instead of assuming the ground is planar.Training uses an ℓ2 loss only where LiDAR points exist.
  • Evaluation: Figure 4 reports range-wise ground-height error on the left and compares predicted versus ground-truth road masks on the right.The road-mask comparison places predictions in the first row and ground truth in the second row.

5 Experiments

The experiments evaluate the baseline and map-aware detectors on TOR4D, a large-scale mapped benchmark, and KITTI, a public BEV detection benchmark.

  • Experimental setup: The method is validated on TOR4D, containing over one million frames with corresponding HD maps, and on the public KITTI BEV benchmark.The experiments compare a detector without map priors, PIXOR++, with the map-aware detector HDNET.

5.1 Evaluation on TOR4D Benchmark

On TOR4D, the map-aware HDNET detector improves over PIXOR++ with both online and offline map priors, while the baseline becomes substantially weaker at longer ranges.

  • Baseline: 81.78% AP within 0-70 meters is achieved by baseline PIXOR++, which runs at 17 ms per frame.The baseline outperforms FAF by 4.3% and PIXOR by 2.96%.
  • Online map estimation: < 5 cm L1 error pixel-wise within 50 meters is achieved for online ground-height estimation.The error increases at longer ranges because LiDAR observations become sparser.
  • Online map estimation: 97.70% pixel-wise accuracy and 92.86% IoU are achieved for road segmentation on the validation set.These results use one single LiDAR sweep only.
  • Map-aware detection: 1.33%/2.42% AP gains over PIXOR++ are obtained with online/offline maps, with offline maps reaching a 5.49% gain at 50-70 meters.The reported gains increase at longer ranges, especially for offline maps.

5.2 Evaluation on KITTI BEV Benchmark

On KITTI, HDNET transfers an online map predictor trained on TOR4D and improves the already strong PIXOR++ baseline while outperforming competing detectors.

  • KITTI benchmark: 35 ms is the runtime of PIXOR++, which surpasses all competing LiDAR detectors and exceeds VoxelNet by 4.44% AP in the moderate setting.KITTI provides no HD maps, so the transferred map predictor is used without fine-tuning.
  • KITTI benchmark: 2.87% absolute AP gain in the moderate setting enables HDNET to set a new KITTI benchmark record.The map predictor is pre-trained on TOR4D, and no TOR4D object labels are used.
  • Ablations: Table 3 reports AP (%) ablations covering ground and road priors, robustness to unavailable maps, and range-wise detection performance.The table organizes experiments across prior-exploitation strategies and map-availability conditions.

5.3 Ablation Studies

The ablations examine prior design, semantic-road fusion, missing-map robustness, and performance across object ranges. Map-aware detection benefits from flexible ground priors and data dropout, while online-map gains weaken at long range because single-sweep maps are sparse.

  • 5.3.1 Ground prior: Point-wise ground parameterization achieves significantly more detection gain than a ground-plane prior, although both improve the baseline.The two priors use the same detection integration and are evaluated on the KITTI validation set without data augmentation.
  • 5.3.2 Semantic prior: Semantic road priors are evaluated through input-level fusion, multi-task learning, and output masking.The input representation incorporates the road prior, while the other variants use it at the output level.
  • 5.3.3 Data dropout: 11.57%/6.37% AP drops occur without data dropout at 0%/50% map availability, whereas dropout keeps unavailable-map performance almost as good as the baseline.With dropout, the map-aware detector surpasses the baseline when maps are available.
  • 5.3.4 Performance at various ranges: Range evaluation divides 70 meters into 7 bins and ignores detections and labels outside each evaluated bin.Range is defined as object distance from the ego-car on the XY plane; evaluations cover online and offline maps.
  • 5.3.4 Performance at various ranges: Offline-map gains exceed 3% beyond 40 meters, while online gains decrease beyond 40 meters because single-sweep map estimates become sparse.Map priors help more with fewer LiDAR observations, but online estimation itself is limited at long range.

5.4 Qualitative Results

Qualitative KITTI validation results compare PIXOR++ without maps against HDNET with online map estimation. HDNET shows higher long-range detection accuracy and localization precision, while PIXOR++ exhibits false positives and false negatives.

  • 5.4 Qualitative Results: Figure 5 compares baseline PIXOR++ without a map against HDNET with online map estimation on the KITTI validation set.Ground truth, detections, and detection scores are shown in green, red, and blue, respectively.
  • 5.4 Qualitative Results: HDNET increases long-range detection accuracy and localization precision relative to PIXOR++.The qualitative comparison highlights false positives and false negatives for PIXOR++ at long range.

6 Conclusion

The paper incorporates geometric and semantic HD-map priors into BEV LiDAR detection and predicts those priors online when maps are unavailable. The resulting map-aware detector consistently outperforms the baseline in mapped and unmapped settings while running at over 20 frames per second.

  • 6 Conclusion: The paper targets using HD-map information to improve modern 3D object detection for autonomous driving.It identifies geometric and semantic priors in HD maps and incorporates them into a BEV LiDAR representation.
  • 6 Conclusion: The detector fuses map priors with BEV LiDAR features and predicts both priors online from a single LiDAR sweep when maps are unavailable.The framework uses a map prediction module to estimate the map online.
  • 6 Conclusion: Over 20 frames per second is achieved while consistently outperforming the baseline with or without HD maps.The reported efficiency is attributed to the BEV representation and single-stage detection framework.
Loading 2012.11704v1…