MapFusion: A Novel BEV Feature Fusion Network for Multi-modal Map Construction

• Existing BEV-level fusion can leave camera and LiDAR features semantically misaligned and relies on simple operations that do not address misalignment or information loss.Source: §1 · p. 2
• MapFusion is presented as a plug-and-play approach compatible with existing pipelines for different map tasks.Source: §1 · p. 3
• CIT uses self-attention to enable interaction between camera and LiDAR BEV feature spaces, while DDF adaptively selects useful information from each modality.CIT provides modality interaction; DDF refines and fuses the CIT results by concentrating on modality-specific information.Source: §1 · p. 3
• 3.6% and 6.2% absolute improvements over state-of-the-art methods on nuScenes for HD map construction and BEV map segmentation, respectively.Source: §1 · p. 3
• This paper extends the conference version from vectorized HD map construction to include BEV map segmentation as evidence of the method’s versatility.Source: §1 · p. 3

2.1. Map Construction Task

• Camera-based mapping lacks explicit depth and may require higher-resolution images or larger backbones, while LiDAR offers 3D geometry but faces sparsity and sensing noise.Source: §2.1 · p. 4
• HDMapNet, VectorMapNet, and MapTR use channel concatenation and convolution, which the authors say overlooks modality interaction and risks misalignment and information loss.Source: §2.1 · p. 4
• BEV map-segmentation methods avoid homography artifacts, but the cited approaches are described as lacking explicit depth information and achieving unsatisfactory performance.Source: §2.1 · p. 4

2.2. Multi-sensor Fusion

• Point-level fusion loses camera information—only about 5% aligns with typical 32-beam LiDAR—and feature-level fusion also struggles to generalize across tasks.Source: §2.2 · p. 5
• MapFusion pairs Cross-modal Interaction Transform (CIT) for camera-LiDAR BEV interaction with Dual Dynamic Fusion (DDF) for adaptive selection of modality information.The paper presents both modules as plug-and-play components for map-construction pipelines.Source: §2.2 · p. 5

3. Methodology

• MapFusion converts modality-specific features into a shared BEV representation, applies CIT and DDF, then sends fused features to a decoder and prediction heads.The framework is described as plug-and-play and compatible with existing pipelines.Source: §3 · p. 5, Figure 3 · p. 6

3.1. Preliminaries

• The framework takes multi-modal sensor data, including camera images and LiDAR point clouds, as input and predicts map elements in BEV space.Each LiDAR point contains 3D coordinates, reflectivity, and beam index.Source: §3.1 · pp. 5–6

3.2. Map Encoder

• The map encoder transforms camera and LiDAR features into a unified BEV representation that preserves geometric and semantic information.Camera features use image backbones, an FPN, and a 2D-to-BEV transformation; LiDAR features use voxelization, a sparse encoder, and flattening.Source: §3.2 · p. 6

3.3. Cross-modal Interaction Transform (CIT)

• CIT concatenates flattened camera and LiDAR BEV tokens and uses self-attention to model within- and cross-modality relationships.Its correlation matrix weights feature positions and supports simultaneous intra- and inter-modality fusion.Source: §3.3 · pp. 6–8

3.4. Dual Dynamic Fusion (DDF)

• DDF generates attention weights from the summed modality features, gates both inputs with complementary weights, and fuses them adaptively.Its 3×3 convolution combines channel and spatial information, while global average pooling supplies global weights.Source: §3.4 · p. 8

3.5. Map-Task Heads

• The fused BEV features feed task-specific heads for vectorized HD map elements or multi-class BEV map segmentation.The HD map head predicts pedestrian crossings, lane dividers, and road boundaries; segmentation uses separate binary predictions because categories may overlap.Source: §3.5 · p. 9

4.1. Dataset

• nuScenes evaluation uses 1,000 sequences, with six cameras covering 360° and samples annotated at 2 Hz.HD map construction evaluates pedestrian crossings, lane dividers, and road boundaries; segmentation predicts six semantic classes.Source: §4.1 · p. 9
• The nuScenes segmentation task predicts six classes: drivable lanes, pedestrian crossings, walkways, stop lines, carparks, and lane dividers.The HD map task uses three elements: pedestrian crossings, lane dividers, and road boundaries.Source: §4.1 · p. 9
• Argoverse2 contains 1,000 logs with 15-second recordings, seven RGB cameras, 10 Hz LiDAR sweeps, and a 700/150/150 train/validation/test split.Both tasks evaluate pedestrian crossings, lane dividers, and road boundaries.Source: §4.1 · p. 9

4.2. Evaluation Metrics

• HD map AP is averaged across Chamfer-distance thresholds of 0.5 m, 1.0 m, and 1.5 m to produce mAP.The thresholds determine whether predictions and ground truth are matched.Source: §4.2 · p. 9
• The evaluation range is [−15.0m, 15.0m] for X and [−30.0m, 30.0m] for Y.Source: §4.2 · p. 9
• BEV map segmentation uses class-averaged mIoU, averaging each class’s highest IoU across thresholds under a separate binary-segmentation protocol.The protocol is attributed to BEVFusion.Source: §4.2 · pp. 9–10
• The HD map comparison tables cover nuScenes and Argoverse2, with the nuScenes table reporting single-class APs and overall mAP.Source: Table 1 · p. 10, Table 2 · p. 10

4.3. Experimental Setting

• HD map construction uses MapTR as its baseline, with ResNet50 and SECOND backbones and GKT as the default 2D-to-BEV module.Source: §4.3 · p. 10
• BEV map segmentation uses BEVFusion as its baseline, with Swin-T and VoxelNet backbones and LSS as the default 2D-to-BEV module.The results table reports backbone and input-modality information.Source: §4.3 · p. 10, Table 3 · p. 11
• CIT is added before the baseline fuser, whereas DDF replaces the naive convolutional fuser.Source: §4.3 · p. 10

4.4.1. HD map construction task

• 3.6% and 4.1% absolute mAP improvements over MapTR are reported for HD map construction on nuScenes and Argoverse2, respectively.The authors also report that MapFusion outperforms all compared approaches in single-class APs and overall mAP.Source: §4.4.1 · p. 11
• The authors associate MapFusion’s HD map gains with addressing limitations of prior methods that use straightforward channel concatenation and convolution without modality interaction.Source: §4.4.1 · p. 11
• CIT enables the two feature spaces to interact through self-attention, while DDF adaptively selects valuable modality information for fusion.Source: §4.4.1 · p. 11

4.4.2. BEV map segmentation task

• 6.2% and 6.5% absolute mIoU improvements over BEVFusion are reported for BEV map segmentation on nuScenes and Argoverse2, respectively.Source: §4.4.2 · p. 12
• In single-modality BEV map segmentation, camera-based models perform significantly better than LiDAR-based models, which the authors relate to the task’s emphasis on image semantics.Source: §4.4.2 · p. 12
• For BEV segmentation, the authors note that PointPainting focuses on foreground LiDAR points and MVP on foreground-object densification, while BEVFusion and X-Align neglect modality interactions.They report that these approaches do not effectively segment map components or use simple fusion strategies associated with misalignment and information loss.Source: §4.4.2 · p. 12

4.5.1. Contribution of each component

• 3.6% and 6.2% absolute improvements are reported for the full model over the baseline on HD map construction and BEV map segmentation, respectively.DDF-only and CIT-only variants also improve over baseline, and the full model outperforms either single-module variant.Source: §4.5.1 · p. 13
• CIT supports camera-LiDAR BEV-space interaction through cross-attention, while DDF adaptively selects valuable information from each modality.Source: §4.5.1 · p. 13

4.5.2. Analysis on different Fusion methods

• DDF improves over convolutional fusion by 2.5% on HD map construction and 1.4% on BEV map segmentation.The authors describe DDF as automatically selecting valuable information from different modalities.Source: §4.5.2 · p. 14

4.5.3. Compatibility with other HD Map Construction methods

• Adding MapFusion to HDMapNet, VectorMapNet, and MapTR improves nuScenes validation accuracy by 7.6%, 5.5%, and 3.6%, respectively.The authors present these results as evidence of compatibility with other HD map construction methods.Source: §4.5.3 · p. 14, Table 9 · p. 14

4.5.4. Accuracy-Computation Analysis

• CIT achieves the highest accuracy improvement at higher computational cost, while DDF adds less cost but yields less performance gain.The authors report that their fusion modules offer better accuracy-computation trade-offs than the baselines.Source: §4.5.4 · p. 14

4.6. Visualization

• After CIT, camera and LiDAR BEV features from 500 nuScenes validation samples appear aligned in a shared space, unlike their pre-CIT separation.Figure 7 shows the two modalities’ features close together after CIT.Source: §4.6 · p. 14
• CIT makes camera and LiDAR feature maps more similar; visualizations also show enhanced camera features and recovery of missing LiDAR features.The paper identifies these changes in highlighted regions of the feature maps.Source: §4.6 · p. 14, Figure 8 · p. 16
• Figure 6 compares the accuracy-computation trade-off for CIT with different fusion strategies.Source: Figure 6 · p. 15
• For HD map predictions, CIT corrects substantial baseline errors, and full MapFusion further improves accuracy.The comparison includes MapTR baseline, MapFusion with CIT only, and full MapFusion.Source: §4.6 · pp. 15–16

5. Conclusion

• MapFusion uses CIT to enhance one modality with the other through cross-attention, then DDF adaptively selects valuable information from both modalities.The conclusion also reports an ablation study verifying the effectiveness of MapFusion components.Source: §5 · p. 16
• MapFusion brings consistent accuracy improvements for two map reconstruction task types across different datasets.Source: §5 · p. 16
• MapFusion can be integrated into existing pipelines in a plug-and-play manner; benefits to other multi-modal perception tasks are proposed as future work.Source: §5 · p. 16