Source-linked AI summary
MapTR: Structured Modeling and Learning for Online Vectorized HD Map Construction
Bencheng Liao, Shaoyu Chen, Xinggang Wang, Tianheng Cheng, Qian Zhang, Wenyu Liu, Chang Huang
TL;DR
Autonomous driving needs precise, structured HD maps, but online vectorized construction must handle complex shapes efficiently. MapTR addresses this with permutation-equivalent point-set modeling and a structured Transformer pipeline. On nuScenes, it reports the best performance and efficiency among existing vectorized approaches, including real-time camera-only operation.
Problem
Online vectorized HD-map construction methods are restricted by efficiency, while map elements have dynamic shapes and ambiguous point orderings.
Method
MapTR models each map element as a point set with equivalent permutations and uses hierarchical queries and bipartite matching in an end-to-end Transformer.
Results
MapTR achieves the best performance and efficiency among existing vectorized approaches on nuScenes; MapTR-nano runs at 25.1 FPS and is 8× faster with 5.0 higher mAP than the state-of-the-art camera-based method.
Takeaways & Limitations
MapTR precisely perceives arbitrary-shape map elements and maintains stable, robust construction quality in complex and varied driving scenes.
Abstract
from arXiv · showhide
High-definition (HD) map provides abundant and precise environmental information of the driving scene, serving as a fundamental and indispensable component for planning in autonomous driving system. We present MapTR, a structured end-to-end Transformer for efficient online vectorized HD map construction. We propose a unified permutation-equivalent modeling approach, i.e., modeling map element as a point set with a group of equivalent permutations, which accurately describes the shape of map element and stabilizes the learning process. We design a hierarchical query embedding scheme to flexibly encode structured map information and perform hierarchical bipartite matching for map element learning. MapTR achieves the best performance and efficiency with only camera input among existing vectorized map construction approaches on nuScenes dataset. In particular, MapTR-nano runs at real-time inference speed ($25.1$ FPS) on RTX 3090, $8\times$ faster than the existing state-of-the-art camera-based method while achieving $5.0$ higher mAP. Even compared with the existing state-of-the-art multi-modality method, MapTR-nano achieves $0.7$ higher mAP, and MapTR-tiny achieves $13.5$ higher mAP and $3\times$ faster inference speed. Abundant qualitative results show that MapTR maintains stable and robust map construction quality in complex and various driving scenes. MapTR is of great application value in autonomous driving. Code and more demos are available at \url{https://github.com/hustvl/MapTR}.
1 INTRODUCTION
MapTR addresses the efficiency and representation challenges of online vectorized HD map construction with a unified, structured Transformer framework. It models map elements through permutation-equivalent point sets and reports strong nuScenes performance with stable qualitative results.
- HD maps provide instance-level vectorized road information and semantic topology vital for autonomous-driving navigation.
- Offline HD-map construction incurs complicated pipelines and high maintenance costs, motivating online construction around the ego-vehicle at runtime.
- Current online vectorized methods are restricted by efficiency, while rasterized BEV maps lack instance-level structure and require time-consuming post-processing.
- MapTR models each map element as a point set with equivalent permutations, resolving ambiguity in point ordering while representing dynamic shapes.
- MapTR uses hierarchical query embeddings, parallel prediction, hierarchical bipartite matching, and point- and edge-level supervision in an end-to-end framework.
- 25.1 FPS enables MapTR-nano to run in real time on RTX 3090; it is 8× faster and 5.0 mAP higher than the existing state-of-the-art camera-based method.
2 RELATED WORK
Related work spans rasterized BEV segmentation, vectorized map construction, lane detection, and contour-based instance segmentation. MapTR differs by targeting unified vectorized perception of varied map elements across a 360° horizontal field of view.
- BEV semantic-segmentation methods generate rasterized maps, while HDMapNet uses heuristic post-processing to group pixels into vectorized instances.
- Lane-detection methods often focus on single-view annotations and open-shape lane elements, limiting their coverage of general HD-map construction.
- Figure 2 illustrates start-point and direction ambiguity for polylines and polygons, including two directions for lane dividers and clockwise or counter-clockwise ordering for crossings.
- MapTR extends beyond limited single-view perception to various map elements across a 360° horizontal field of view with unified modeling and learning.
- Contour-based instance-segmentation methods reformulate segmentation as contour prediction and estimate image coordinates of contour vertices.
3 MAPTR
MapTR unifies map-element representation with permutation-equivalent point sets and uses structured Transformer prediction with hierarchical matching. Its architecture combines BEV encoding, hierarchical queries, parallel map-element inference, and geometry-aware supervision.
- Permutation-equivalent modeling: Map elements are discretized as polygons for closed shapes and polylines for open shapes by sequential boundary sampling.
- Permutation-equivalent modeling: Permutation-equivalent modeling avoids ambiguity from imposing a fixed point order, stabilizing the learning process.
- Permutation-equivalent modeling: A map element is represented by a point set and a group of equivalent permutations covering organization sequences with the same geometry.Polyline groups include two directions, while polygon groups include 2 × Nv permutations.
- Hierarchical matching and prediction: MapTR parallelly predicts a fixed-size set of map elements in one pass using a DETR-style end-to-end paradigm.The prediction set is larger than the typical number of map elements in a scene, with ground truth padded by no-object entries.
- Hierarchical matching and prediction: Hierarchical bipartite matching first assigns predicted instances to ground truth using class and point-set position costs, then matches points within each assigned instance.Hungarian matching finds the instance-level assignment; point-level matching selects among the predefined permutation group using Manhattan distance.
- Training loss: Training combines classification, point2point, and edge-direction losses to supervise labels, point positions, and geometric edge shape.The point2point loss uses Manhattan distance, while edge-direction loss uses paired predicted and ground-truth edge directions.
- Architecture: The architecture encodes surround-view camera features into BEV and decodes map elements with hierarchical queries in a fully end-to-end encoder-decoder pipeline.The decoder explicitly represents instance-level and point-level information; MapTR is also compatible with LiDAR and RADAR inputs.
4 EXPERIMENTS
Experiments on nuScenes evaluate MapTR’s camera-based performance, efficiency, convergence, and proposed design choices. MapTR achieves strong real-time results, while ablations support permutation-equivalent modeling, edge-direction supervision, and compatibility with multiple BEV transformations.
- Dataset and metric: Experiments use nuScenes, whose samples contain six RGB camera views covering a 360° horizontal field of view, and evaluate pedestrian crossings, lane dividers, and road boundaries.Average precision is used for evaluation, with annotations sampled at 2 Hz.
- Comparisons with state-of-the-art methods: 25.1 FPS enables MapTR-nano to achieve real-time inference on an RTX 3090, while outperforming camera- and multi-modality baselines.MapTR-nano is 8× faster than the existing state-of-the-art camera-based method with 5.0 higher mAP, and 8× faster than the state-of-the-art multi-modality method with 0.7 higher mAP.
- Ablation study: 5.9 mAP improvement over vanilla modeling validates permutation-equivalent modeling, with pedestrian-crossing AP improving by 11.9.The proposed representation also stabilizes the learning process, as shown by convergence visualization.
- Ablation study: β = 5e−3 is adopted as the default edge-direction-loss weight because it provides appropriate supervision.β = 0 denotes removing the edge direction loss; the ablation varies this loss weight in Table 3.
- Ablation study: MapTR is compatible with various 2D-to-BEV transformation methods and maintains stable performance across them.The evaluated alternatives include IPM, LSS, Deformable Attention, and GKT.
- Qualitative visualization: Qualitative results show stable map construction across complex and varied driving scenes, with supplementary videos supporting robustness.Additional qualitative results are provided in the appendix.
5 CONCLUSION
MapTR is a structured end-to-end framework for efficient online vectorized HD map construction. Its permutation-equivalent modeling, hierarchical matching, and Transformer architecture support precise perception of arbitrary-shaped map elements on nuScenes.
- Conclusion: MapTR combines permutation-equivalent modeling, hierarchical bipartite matching, and an encoder-decoder Transformer for online vectorized HD map construction.The framework is designed as an efficient structured end-to-end approach.
- Conclusion: MapTR precisely perceives arbitrary-shaped map elements on the challenging nuScenes dataset.The authors propose MapTR as a basic self-driving-system module for downstream tasks such as motion prediction and planning.
A IMPLEMENTATION DETAILS
The implementation uses resized multi-view images, standard training settings, and ego-vehicle-centered map preprocessing. Configuration choices differ between MapTR-tiny and MapTR-nano.
- Data Augmentation: 1600×900 source images are resized by 0.2 for MapTR-nano and 0.5 for MapTR-tiny, with color jitter enabled by default.
- Model Setting: λ = 2, α = 5, and β = 5e−3 are used during training, while MapTR-tiny uses 50 instance-level and 20 point-level queries.MapTR-tiny uses six decoder layers, a 0.3m BEV grid, batch size 32, learning rate 6e−4, and 24 epochs for ablations.
- Dataset Preprocessing: Ground-truth map elements are extracted within the ego-vehicle perception range of [−15.0m, 15.0m] on X and [−30.0m, 30.0m] on Y.
B ABLATION STUDY
Ablations examine representation, point and element counts, decoder depth, matching cost, backbone, modality, camera perturbations, and runtime. They identify efficiency–capacity trade-offs, stable alternatives, and robustness boundaries.
- Point Number: 20 points per map element is the default because too few points miss complex geometry, while too many reduce efficiency.
- Element Number: 50 map elements is the default for MapTR-tiny.
- Decoder Layer Number: 6 decoder layers are used because performance improves with depth but saturates at six layers.
- Position Matching Cost: Position matching compares Chamfer-distance and point2point cost designs for instance-level assignment.The point2point cost finds the best point assignment and sums Manhattan distances.
- Modality: Multi-modality MapTR improves over the previous state-of-the-art result by 17.3 mAP while running 2× faster under a 24-epoch schedule.
- Robustness to camera deviation: MapTR retains comparable performance when camera translation noise has 0.1m standard deviation or rotation noise has 0.02rad standard deviation.The perturbations are applied to all cameras and coordinates, with some samples experiencing extremely large deviations.
C QUALITATIVE VISUALIZATION
MapTR is visualized on the nuScenes validation set across varied weather and challenging road environments, where it maintains stable and impressive construction results.
- MapTR results are visualized under sunny and cloudy weather on the nuScenes validation set.
- MapTR results are also visualized under rainy weather on the nuScenes validation set.
- Nighttime visualizations further cover challenging driving conditions, with stable and impressive results reported across the displayed scenarios.