Source-linked AI summary
StreamMapNet: Streaming Mapping Network for Vectorized Online HD Map Construction
Tianyuan Yuan, Yicheng Liu, Yue Wang, Yilun Wang, Hang Zhao
TL;DR
Online vectorized HD-map methods are limited by single-frame input, restricted perception ranges, and evaluation splits with substantial geographic overlap. StreamMapNet combines Multi-Point Attention with streaming temporal fusion and proposes fairer dataset splits, outperforming existing methods across settings while maintaining online inference.
Problem
Existing vectorized HD-map methods have limited perception ranges and lack temporal information, while common nuScenes and Argoverse2 evaluation splits contain substantial location overlap.
Method
StreamMapNet uses Multi-Point Attention for long-range map-element modeling and streaming temporal fusion to propagate historical information across video frames.
Results
StreamMapNet consistently outperforms existing state-of-the-art approaches on original and new benchmark settings.
Takeaways & Limitations
100 × 50 meters: Multi-Point Attention extends local vectorized HD-map perception without a significant performance drop.
Takeaways & Limitations
The current nuScenes evaluation setup has substantial fairness issues caused by a problematic training-validation split.
Abstract
from arXiv · showhide
High-Definition (HD) maps are essential for the safety of autonomous driving systems. While existing techniques employ camera images and onboard sensors to generate vectorized high-precision maps, they are constrained by their reliance on single-frame input. This approach limits their stability and performance in complex scenarios such as occlusions, largely due to the absence of temporal information. Moreover, their performance diminishes when applied to broader perception ranges. In this paper, we present StreamMapNet, a novel online mapping pipeline adept at long-sequence temporal modeling of videos. StreamMapNet employs multi-point attention and temporal information which empowers the construction of large-range local HD maps with high stability and further addresses the limitations of existing methods. Furthermore, we critically examine widely used online HD Map construction benchmark and datasets, Argoverse2 and nuScenes, revealing significant bias in the existing evaluation protocols. We propose to resplit the benchmarks according to geographical spans, promoting fair and precise evaluations. Experimental results validate that StreamMapNet significantly outperforms existing methods across all settings while maintaining an online inference speed of $14.2$ FPS. Our code is available at https://github.com/yuantianyuan01/StreamMapNet.
1. Introduction
StreamMapNet addresses limited perception range and missing temporal information in online vectorized HD mapping with Multi-Point Attention and streaming temporal fusion. It also proposes fairer geographically separated benchmarks after identifying overlap between training and validation locations.
- Motivation: Existing vectorized map methods face small perception ranges and limited temporal modeling for complex scenes.These limitations are identified alongside the need for temporal-consistent maps covering wider areas.
- Method: Multi-Point Attention captures longer attention ranges efficiently for irregular and elongated map elements.The model frames map construction as detection with a BEV encoder and DETR-like decoder.
- Temporal fusion: Streaming temporal fusion propagates hidden states across frames, encoding historical information for longer associations while reducing memory and latency costs relative to stacking.Each frame is processed individually rather than stacking multiple frames.
- Benchmark evaluation: More than 84% of nuScenes validation locations overlap with training locations, motivating new non-overlapping splits for fairer evaluation.The paper reports a similar issue in Argoverse2 and evaluates both original and new settings.
- Method: 100 × 50 meters: Multi-Point Attention extends local vectorized HD-map perception without a significant performance drop.This contribution targets broader-range mapping while preserving practicality.
- Temporal fusion: The method improves temporal consistency and map quality by leveraging temporal information in a streaming strategy.The contribution is presented as a temporal fusion module for vectorized local HD maps.
- Results: StreamMapNet consistently outperforms existing state-of-the-art approaches in both original and newly established benchmark settings.The paper presents this as a cross-setting evaluation result.
2. Related Works
Related work progresses from rasterized BEV semantic maps to end-to-end vectorized mapping, while temporal BEV methods commonly stack historical frames at increasing computational and memory cost. StreamMapNet combines a BEV extraction pipeline with transformer decoding and propagated memory features.
- Vectorized map construction: HDMapNet generates BEV semantic segmentations and then uses heuristic post-processing to obtain vectorized map instances.This approach also introduced mean Average Precision as an evaluation metric.
- Vectorized map construction: VectorMapNet introduced an end-to-end transformer-based approach for detecting and refining map elements.It uses a DETR decoder for map-element detection.
- StreamMapNet: StreamMapNet’s pipeline uses a BEV encoder, a transformer decoder with Multi-Point Attention, and a memory buffer for propagated features.These components support BEV extraction, prediction generation, and temporal feature storage.
- BEV perception: BEV perception methods aggregate image features into BEV representations using depth lifting, deformable attention, or inverse perspective mapping.The cited methods include Lift-Splat-Shoot, BEVFormer, and SimpleBEV.
- Temporal modeling: Stacking historical frames processes them together but increases computational and memory costs linearly with the number of frames.This reduces training and inference speed as more frames are added.
3. StreamMapNet Model
StreamMapNet constructs vectorized local HD maps from synchronized multi-view images using a BEV encoder, a transformer decoder with Multi-Point Attention, and temporal memory. Its decoder models elongated map elements through predicted polyline points, while query and BEV fusion propagate information across frames.
- 3. StreamMapNet Model: The architecture combines a BEV image encoder, a Multi-Point Attention transformer decoder, and a memory buffer for propagated features.The model represents each map instance with a class label and polyline points.
- 3. StreamMapNet Model: Each object query represents one map element and predicts its class score and Np polyline coordinates.Queries encode semantic and geometric information and are matched with ground-truth instances during training.
- 3. StreamMapNet Model: Multi-Point Attention uses predicted points from the previous layer as reference points, enabling long-range attention for irregular map elements at O(Np) complexity.This replaces object-center references while avoiding the O(HW) cost of global attention.
- 3. StreamMapNet Model: Query Propagation transforms and reuses high-confidence queries, then combines them with top Nq − k new queries to refresh the decoder query set.Predicted polylines are transformed into the new coordinate system as initial reference points.
- 3. StreamMapNet Model: BEV Fusion recurrently warps historical BEV features using ego pose and combines them with current features through a GRU followed by layer normalization.This supplies dense temporal information while supporting training stability.
4. Experiments
The experiments reassess dataset splits and compare mapping methods under a geographically separated Argoverse2 benchmark. The reported setup highlights substantial location overlap in conventional splits and shows StreamMapNet leading across perception ranges on the new split.
- 4. Experiments: The conventional nuScenes split overlaps training and validation locations by more than 84%, allowing models to memorize location-map pairs rather than generalize to unseen scenes.The paper identifies this as inconsistent with the goal of online map construction.
- 4. Experiments: StreamMapNet outperforms other methods across all categories at both 30 m and 50 m perception ranges on the new Argoverse2 split.The table caption attributes robustness at longer ranges to temporal association and long-range attention.
- 4. Experiments: The new evaluation uses geographically separated splits designed to reduce overlap and balance location, object, and weather distributions.The same reassessment covers both nuScenes and Argoverse2.
4.2. Implementation Details
The implementation evaluates three vectorized map-element categories across two perception ranges using Average Precision, with a larger range aligned to 3D object-detection scope.
- 4.2. Implementation Details: The evaluation covers pedestrian crossings, lane dividers, and road boundaries.These are the three map-element types considered in the experiments.
- 4.2. Implementation Details: The larger perception range spans 50 m front and back and 25 m left and right, while the smaller range spans 30 m front and back and 15 m left and right.The larger range aligns with the scope used in 3D object detection.
- 4.2. Implementation Details: Average Precision is evaluated at thresholds of {1.0 m, 1.5 m, 2.0 m} for the 50 m range and {0.5 m, 1.0 m, 1.5 m} for the smaller range.
4.4. Comparison with Baselines
StreamMapNet consistently outperforms existing methods on original and geographically resplit Argoverse2 and nuScenes benchmarks, including broader perception ranges. The results also expose substantial overfitting in the original validation splits.
- Argoverse2: StreamMapNet outperforms existing methods across categories on the new Argoverse2 split at both 30 m and 50 m perception ranges.The comparison reports online inference speed and greater robustness as perception range increases.
- Original split: 10.2 mAP is StreamMapNet’s minimum advantage over other methods on the original Argoverse2 split at 50 m.
- NuScenes: 13.0 mAP at 30 m and 8.2 mAP at 50 m separate StreamMapNet from existing methods on the new NuScenes split.
- Benchmark fairness: Approximately 50% performance decreases after moving to the new split across methods, indicating that the original validation split is prone to overfitting.
4.5. Ablation Studies
Ablations show that Multi-Point Attention and temporal fusion are important components of StreamMapNet’s performance. Replacing Multi-Point Attention with conventional deformable attention hampers convergence, while progressively adding temporal components improves results.
- Component contributions: Each modification from the single-frame baseline to the full model contributes to performance gains.
- Occlusion setting: The qualitative occlusion comparison contrasts temporal and single-frame models when a white truck blocks the crossroad.
- Multi-Point Attention: Replacing Multi-Point Attention with conventional deformable attention hampers convergence because its attention range is restrictive.
- Temporal fusion: Progressively introducing temporal fusion components consistently enhances performance in the ablation sequence.
4.6. Qualitative Analysis
Qualitative results show that temporal information helps StreamMapNet reconstruct road structure when a truck occludes the crossroad. The resulting map is described as stable and reliable, unlike the single-frame output.
- Occlusion comparison: Without temporal information, the single-frame model fails to accurately reproduce the crossroad hidden by a large truck.
- Occlusion comparison: Temporal information from previous frames enables StreamMapNet to correctly reproduce the occluded road structure.
- Map quality: The temporally informed model generates a stable and reliable HD map in the occlusion scenario.
5. Conclusion & Acknowledgement
The paper proposes an end-to-end model for online vectorized local HD-map construction and new non-overlapping splits for NuScenes and Argoverse2. It concludes that temporal information supports stability in wide-range perception while emphasizing safety testing and data-governance precautions.
- Contributions: StreamMapNet is an end-to-end model for online construction of vectorized, local HD maps.
- Contributions: Temporal information promotes stability in wide-range map perception.
- Benchmark: New non-overlapping NuScenes and Argoverse2 splits are proposed to foster a more balanced benchmark.
- Deployment considerations: The model may still make false predictions in challenging scenarios, requiring comprehensive safety testing before real-vehicle deployment.
- Data governance: HD-map collection and use may implicate privacy and legal rights, requiring precautions in applicable countries or regions.
A. Dataset Statistics
The original Argoverse2 and nuScenes splits contain substantial geographic overlap between training and validation data. The proposed re-divisions reduce or eliminate this overlap while preserving balanced 700/150 training-validation splits.
- The original splits of both datasets exhibit high training-validation overlap, motivating re-division for map-construction evaluation.The supplementary analysis explicitly frames re-splitting as necessary for this task.
- Cumulative training, validation, and overlap areas are compared using locations within a 30 m radius of each ego-vehicle position.Table 6 summarizes these areas across the original and revised splits.
- 54% of Argoverse2 locations overlap between the original training and validation sets.The new split removes overlap across the 1,000 scenes and balances objects, weather, and cities.
- The proposed Argoverse2 re-division uses a 700/150 train-validation split with balanced distributions and no geographic overlap.The split is formed from the full set of 1,000 scenes, without an additional testing set.
- More than 85% of NuScenes validation locations appear in training originally, but the new split reduces overlap to 11%.The revised split substantially mitigates the overfitting concern associated with the original geographic overlap.
B. Dataset Visualization
The dataset visualizations compare original and revised geographic splits across Argoverse2 cities and NuScenes districts. They show how the revised divisions balance training and validation coverage while reducing overlap.
- Figures 7–12 compare original and revised train-validation splits across six Argoverse2 cities using green training, blue validation, and red overlap regions.The visualizations cover Austin, Detroit, Miami, Palo Alto, Pittsburgh, and Washington D.C.
- The revised visual splits maintain balanced training and validation data across Argoverse2 cities while ensuring no overlaps.The original split contains substantial overlap that the new split alleviates.
- Argoverse2 has significantly greater location diversity than NuScenes, a difference associated with the performance gap observed in the experiments.
- Figures 13–16 compare original and revised splits across four NuScenes districts using the same training, validation, and overlap color encoding.The districts are Boston seaport, Singapore Holland, Singapore Queenstown, and Singapore One-North.