PaperlayerView exact revision on arXiv

Source-linked AI summary

MapLightning: Online Vectorized HD Map Construction with 1D Map Tokens

Shen Zheng, Anurag Ghosh, Mani Ramanagopal, Srinivasa Narasimhan

arXiv:2610.01905v1cs.CV

TL;DR

Online vectorized map construction commonly relies on dense BEV grids, despite vectorized map outputs occupying only a small portion of the perception range. MapLightning replaces the grid with self-attention-aggregated 1D map tokens and reports state-of-the-art accuracy and efficiency on nuScenes and Argoverse 2.

  • Problem

    Dense BEV representations are less suited to sparse vectorized map prediction, whose elements occupy only 4.27% of nuScenes and 2.65% of Argoverse 2 rasterized perception ranges.

    Source: §1 · p. 1

  • Method

    MapLightning replaces dense BEV features with compact 1D learnable map tokens, jointly updates image and map tokens through self-attention, then decodes from the retained map tokens.

    Source: §1 · p. 2, §5 · pp. 8–9

  • Results

    MapLightning reports state-of-the-art results across online map construction and related tasks; with MobileNetV3-Large, it exceeds MapTRv2 by +10.1 mAP on nuScenes and +16.2 mAP on Argoverse 2.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The representation also generalizes to uncertainty-aware map construction, downstream trajectory prediction, and 3D object detection.

    Source: §1 · p. 2, §5 · p. 9

    Compact 1D tokens may require adaptation in end-to-end frameworks whose task decoders rely explicitly on BEV structure, and the fixed token budget may need tuning for other tasks or datasets.

    Source: §5 · p. 9

Abstract

from arXiv · show

Online vectorized HD map construction is essential for scaling safe autonomous driving and requires accurate, real-time inference. Prior methods typically rely on dense bird's-eye-view (BEV) grids as the intermediate representation. We propose \textit{MapLightning}, which replaces the dense BEV grid with a compact set of 1D learnable map tokens. To construct map tokens from image features, we choose self-attention over vanilla cross-attention because it enables joint interactions and contextual aggregation among image and map tokens. Our transformer-based mapper concatenates map and image tokens, applies full self-attention, discards the image tokens, and retains the updated map tokens for decoding. This design offers three advantages. First, our representation is efficient, using fewer tokens, consuming less memory, and running faster. Second, the lightweight design allows the map decoder to use full rather than deformable cross-attention for better global context. Third, unlike BEV-based methods, our network does not use camera projection parameters, making it robust to camera-extrinsic perturbations. MapLightning uses up to 16.7$\times$ fewer intermediate tokens than dense BEV-based methods and achieves state-of-the-art accuracy and efficiency on nuScenes and Argoverse~2. Its lightweight variant surpasses MapTRv2 by +10.1 mAP on nuScenes and +16.2 mAP on Argoverse~2, while delivering 1.73$\times$ faster inference (40+ FPS) with 53\% less memory. We further show improvements on uncertainty-aware map construction and downstream trajectory prediction. Code and models will be released.

1 INTRODUCTION

Online vectorized HD mapping often uses dense BEV grids despite sparse map outputs. MapLightning replaces the grid with self-attention-aggregated 1D map tokens and reports state-of-the-art accuracy and efficiency on nuScenes and Argoverse 2.

1 INTRODUCTION

  • Online maps build local maps from onboard sensors to reduce reliance on costly offline construction, but dense BEV intermediates poorly fit sparse vectorized outputs.Map elements occupy 4.27% of the nuScenes and 2.65% of the Argoverse 2 rasterized perception range.Source: §1 · p. 1
  • Compactness permits full decoder cross-attention for global context; unlike BEV methods, the network avoids camera projection parameters and is robust to extrinsic perturbations.The decoder queries read from the fixed map-token representation.Source: §1 · p. 2
  • MapLightning concatenates image and map tokens for joint self-attention, then discards image tokens and retains the updated map tokens for decoding.Unlike vanilla cross-attention, this also allows image-to-image and map-to-map interactions for contextual aggregation.Source: §1 · p. 2, Figure 1 · p. 2
  • With MobileNetV3-Large, MapLightning exceeds MapTRv2 by +10.1 mAP on nuScenes and +16.2 mAP on Argoverse 2, with 1.73× faster inference (40+ FPS) and 53% less memory.The framework uses up to 16.7× fewer intermediate tokens than dense BEV-based methods.Source: §1 · p. 2
  • The authors also report improvements in uncertainty-aware map construction, downstream trajectory prediction, and 3D object detection.Source: §1 · p. 2

2 RELATED WORK

Prior work spans dense-BEV online mapping, approaches beyond BEV intermediates, and attention-based token compression. MapLightning applies compact attention-based map tokens to sparse online vectorized map construction.

2.1 ONLINE VECTORIZED HD MAP CONSTRUCTION

2.2 BEYOND BEV-BASED REPRESENTATION

  • BEV suits dense segmentation and occupancy; SparseBEV matches dense-BEV detection, but SparseMeXt trails MapTRv2, while MapLightning uses a compact 1D intermediate instead of retaining or dropping BEV.Source: §2.2 · p. 3

2.3 COMPRESSING TOKENS VIA ATTENTION MECHANISMS

  • Prior learned tokens pool inputs via cross-attention or joint self-attention; MapLightning tailors attention-based compression to sparse online vectorized map construction.Source: §2.3 · p. 3

3 METHOD

MapLightning replaces the dense BEV intermediate with compact learnable map tokens, formed by jointly refining image and map tokens and then decoded with full cross-attention.

3.1 BACKGROUND

  • Map points are decoded from queries rather than tied to BEV cells, making the dense 2D grid a design choice rather than a requirement.This motivates replacing the grid with compact learned map tokens.Source: §3.1 · p. 4

3.2 CONSTRUCTING FLAT IMAGE-TOKEN EMBEDDINGS

  • Flattened image features regain spatial, camera, and temporal structure through learned embeddings, without camera intrinsics, extrinsics, depth, or explicit ego-pose alignment.The learned embeddings encode within-image position, camera identity, and time.Source: §3.2 · p. 4

3.3 SELF-ATTENTION MAPPER

  • The mapper concatenates image tokens with K learnable map tokens, jointly refines both through self-attention, then discards image tokens and retains normalized map tokens.The default uses K=1,200 for nuScenes and 1,400 for Argoverse 2.Source: §3.3 · p. 5

3.4 MAP DECODER

  • The decoder replaces deformable BEV sampling with full cross-attention to map tokens, then iteratively refines coordinates and predicts map-element classes.Reference points guide coordinate regression only, not feature sampling.Source: §3.4 · p. 5
  • On nuScenes, the mapper and decoder run at 18.9 FPS, faster than MapTRv2’s 14.1 FPS and comparable to cross-attention’s 18.8 FPS.Past-frame features are cached so only the current frame is encoded; compute is quadratic in N + K within the four mapper layers.Source: §3.4 · p. 5

4 EXPERIMENTS

Experiments evaluate map construction on nuScenes and Argoverse 2 under a geographic Near-Extrapolation split, then assess uncertainty-aware mapping and trajectory prediction. The reported comparisons favor self-attention in accuracy, token efficiency, and qualitative map completeness.

4.1 DATASETS AND SPLITS

  • The Near-Extrapolation split uses a 60 m × 30 m perception range and reduces train–validation overlap to 0.9% on nuScenes and 0% on Argoverse 2.The experiments use six nuScenes cameras and seven Argoverse 2 cameras, with corresponding ground-truth vectorized map elements.Source: §4.1 · p. 6

4.3 ONLINE VECTORIZED HD MAP CONSTRUCTION

  • With ResNet-50 and T = 1, self-attention gains +3.4/+5.2 mAP over dense BEV projection and +4.6/+5.7 over cross-attention on nuScenes/Argoverse 2.Table 1 reports the best accuracy with 66.7× fewer intermediate tokens than dense BEV projection, while matching cross-attention in token count and FPS.Source: Table 1 · p. 6, §4.3 · p. 7

4.2 METRICS AND BASELINES

  • Evaluation reports Chamfer-distance mAP at 0.5, 1.0, and 1.5 m thresholds, and FPS on an NVIDIA GeForce RTX 3090; public-code baselines are retrained on Near-Extrapolation.SparseMeXt is excluded from the main comparisons because its code is unavailable and it reports no results on the Near-Extrapolation or StreamMapNet splits.Source: §4.2 · p. 6

4.3 ONLINE VECTORIZED HD MAP CONSTRUCTION

  • Self-attention outperforms cross-attention and prior methods in accuracy on both datasets and achieves the highest inference speed; Figure 4 shows fewer spurious elements and more complete geometries.The qualitative comparisons include nuScenes and Argoverse 2 examples.Source: Table 2 · p. 7, Figure 4 · p. 8

4.4 UNCERTAINTY-AWARE MAP CONSTRUCTION AND TRAJECTORY PREDICTION

  • +14.4 mAP over MapTRv2 and +9.0 mAP over cross-attention on uncertainty-aware map construction; self-attention also improves minADE, minFDE, and MR.These trajectory-prediction metrics improve across all three reported measures.Source: §4.4 · p. 8
  • Adding the uncertainty head lowers mAP for all methods on Near-Extrapolation, but MapLightning degrades least: -3.0 mAP for cross-attention and -2.9 mAP for self-attention.The authors suggest NLL may down-weight gradients for poorly predicted, high-variance samples; the head enables uncertainty-aware downstream trajectory prediction.Source: §4.4 · p. 8

5 CONCLUSION

MapLightning replaces dense BEV features with compact 1D map tokens and uses self-attention to construct uncertainty-aware maps. The paper reports benefits across map construction and downstream tasks, while noting adaptation and token-budget constraints.

5 CONCLUSION

  • MapLightning retains updated 1D map tokens after self-attention over concatenated map and image tokens, replacing the dense 2D BEV grid.The representation uses fewer tokens, less memory, and runs faster, while enabling full cross-attention for global context and avoiding camera projection parameters.Source: §5 · pp. 8–9
  • Self-attention produces more localized uncertainty-aware maps with less diffuse uncertainty, and Table 3 reports better map-construction and trajectory-prediction results than cross-attention and prior methods.The qualitative comparison is on nuScenes; the table trains a separate HiVT model on precomputed uncertainty-aware maps.Source: Figure 5 · p. 9, Table 3 · p. 9
  • Compact 1D tokens may require adaptation in end-to-end frameworks with BEV-dependent decoders, and the fixed token budget may need tuning across tasks or datasets.Dynamic token selection is identified as a promising direction for adapting the budget to input complexity.Source: §5 · p. 9

A ADDITIONAL QUALITATIVE RESULTS

Qualitative examples compare map reconstructions across methods and datasets, while split-specific results show gains over MapTracker under both the StreamMapNet and original splits.

A.1 ONLINE VECTORIZED HD MAP CONSTRUCTION

  • Self-attention yields higher-quality, more consistent maps than Dense BEV Projection and cross-attention, with fewer spurious elements and more complete geometries.The examples include nuScenes and Argoverse 2, with better-preserved intersection geometry.Source: Figure 6 · p. 14

B.1 MAP CONSTRUCTION UNDER DIFFERENT SPLITS

  • Under the StreamMapNet split, MapLightning surpasses MapTracker by +5.5 mAP on nuScenes and +2.8 mAP on Argoverse 2 at 60m×30m, and +5.9 and +2.4 mAP at 100m×50m.Under the original split, gains are +1.6 mAP on nuScenes and +1.1 mAP on Argoverse 2; substantial train–validation geographic overlap means evaluation reflects memorization as well as geographic generalization.Source: Table 4 · p. 15, §B.1 · p. 15, Table 5 · p. 15

B.2 REAL-TIME MAP CONSTRUCTION AND TRAJECTORY PREDICTION

MapLightning is evaluated for real-time map construction, uncertainty-aware mapping and trajectory prediction, 3D object detection, and efficiency. The reported results show gains over comparison methods alongside lower token use and faster inference.

B.2 REAL-TIME MAP CONSTRUCTION AND TRAJECTORY PREDICTION

  • With MobileNetV3-Large, self-attention surpasses MapTRv2 by +10.1 mAP on nuScenes and +16.2 mAP on Argoverse 2, while the cross-attention variant gains +6.5 and +5.5 mAP, respectively.The comparison covers real-time image backbones; MapLightning with self-attention also has the highest reported inference speed in the table.Source: Table 6 · p. 16, §B.2 · p. 16
  • For uncertainty-aware map construction, self-attention surpasses MapTRv2 by +10.7 mAP with MobileNetV3-Large and +10.6 mAP with ResNet-18, and achieves the best minADE, minFDE, and MR across both backbones.The downstream trajectory-prediction comparison is on nuScenes.Source: Table 7 · p. 16, §B.2 · p. 16

B.3 3D OBJECT DETECTION

  • Replacing BEVFormer v2 projection modules with the self-attention mapper improves +2.6 NDS and +3.7 mAP, and increases speed from 8.1 to 26.9 FPS.Against STUR3D, it improves +0.5 NDS and +1.2 mAP, with lower errors across all five nuScenes metrics.Source: Table 8 · p. 17, §B.3 · p. 17

B.4 EFFICIENCY COMPARISONS

  • MapLightning uses 1.2k intermediate tokens versus 5k–20k for existing approaches, and self-attention reaches 18.9 FPS with ResNet-50, 40.2 FPS with MobileNetV3-Large, and 43.0 FPS with ResNet-18.The component comparison reports reduced mapper and decoder runtime and memory, while backbone cost remains unchanged.Source: Table 9 · p. 18, Table 10 · p. 18

C ABLATION STUDIES

Ablations identify a 1.2k-token, four-layer mapper and four timesteps as the best-accuracy settings among those tested, with nearly constant inference speed across configurations.

Ablation on Timesteps

  • Four timesteps achieve the best accuracy among tested settings; accuracy improves consistently up to four timesteps and slightly degrades at six.The paper notes prior conjectures that excessive temporal fusion can dilute current-frame features or introduce redundancy and noise.Source: C ABLATION STUDIES · p. 19

Ablation on Token Count and Mapper Depth

  • The 1.2k-token, four-layer configuration achieves the best accuracy among tested token-count and mapper-depth settings, with FPS varying only from 18.8 to 19.2.Increasing tokens from 0.9k to 1.2k improves accuracy, but 1.5k degrades it; depth 3 is insufficient, while depth 5 slightly degrades accuracy.Source: Table 11 · p. 19, C ABLATION STUDIES · p. 19

D JUSTIFICATION FOR THE NEAR-EXTRAPOLATION SPLIT

The Near-Extrapolation split has almost no train–validation geographic overlap, unlike the original and StreamMapNet splits, which retain substantial and residual overlap, respectively.

D JUSTIFICATION FOR THE NEAR-EXTRAPOLATION SPLIT

  • The Near-Extrapolation split has almost no train–val overlap, compared with 79.4% for the original split and 2.1% for the StreamMapNet split.The authors identify the overlap in the original and StreamMapNet splits as data leakage that undermines their reliability for evaluating online vectorized HD map construction.Source: Figure 7 · p. 20

E ROBUSTNESS TO CAMERA EXTRINSIC PERTURBATIONS

MapLightning maintains its mAP across camera-extrinsic perturbations, while Dense BEV Projection loses mAP under the largest perturbations.

E ROBUSTNESS TO CAMERA EXTRINSIC PERTURBATIONS

  • 35.3 mAP on nuScenes and 63.3 mAP on Argoverse 2 remain unchanged across perturbation levels; Dense BEV Projection drops sharply under the largest perturbations.On nuScenes, Dense BEV Projection falls from 26.7 mAP to 12.8 under rotation and 7.5 under translation; Figure 8 compares robustness as rotation and translation noise increase.Source: Figure 8 · p. 21, E ROBUSTNESS TO CAMERA EXTRINSIC… · p. 21

F IMPLEMENTATION DETAILS

The implementation replaces BEV projection modules with a self-attention mapper across map construction and 3D object detection, with task-specific training settings.

F IMPLEMENTATION DETAILS

  • For map construction and 3D object detection, the self-attention mapper replaces BEV projection, and the decoder uses full rather than deformable cross-attention.With four timesteps and 50 map tokens per camera per timestep, this yields 1.2k map tokens for nuScenes and 1.4k for Argoverse 2.Source: F IMPLEMENTATION DETAILS · p. 22

F.1 ONLINE VECTORIZED HD MAP CONSTRUCTION

  • Map construction follows MapTRv2 training settings but removes depth, BEV-segmentation, and PV-segmentation auxiliary losses because the mapper uses neither depth estimation nor a BEV grid.Training runs for 24 epochs on nuScenes and 6 on Argoverse 2; adjacent frames are spaced 0.5 s and 0.4 s apart, respectively.Source: §F.1 · p. 22

F.2 3D OBJECT DETECTION

  • For 3D detection, the nuScenes setup uses four past frames and learned object queries, unlike BEVFormer v2’s future-frame fusion and first-stage perspective proposals.The setup uses 256×704 input resolution, a ResNet-50 backbone, and a six-layer detection decoder.Source: §F.2 · p. 22
Loading 2610.01905v1…