Source-linked AI summary
MAP-Net: Multi Attending Path Neural Network for Building Footprint Extraction from Remote Sensed Imagery
Qing Zhu, Cheng Liao, Han Hu, Xiaoming Mei, Haifeng Li
TL;DR
Building-footprint extraction from remote-sensing imagery is challenged by lost spatial detail, inaccurate boundaries, and discontinuous large low-texture buildings. MAP-Net addresses these issues with fixed-resolution parallel paths, attention-based multiscale fusion, and spatial pooling. Its ablations show a 0.58% IoU improvement from combining feature squeeze and global enhancement over baseline, while shallow features add 0.04% accuracy but coarser boundaries.
Problem
CNN-based methods lose spatial localization, may miss small buildings, and produce discontinuous or holey large-building footprints because of pooling, shallow semantic noise, and local receptive fields.
Method
MAP-Net progressively generates fixed-resolution parallel paths, adaptively squeezes channel-wise multiscale features, and uses spatial pooling to capture global dependency.
Results
The combined feature squeeze and global enhancement modules improve baseline IoU by 0.58%, while adding shallow features improves accuracy by 0.04% but introduces coarse boundary noise.
Takeaways & Limitations
MAP-Net extracts multiscale footprints with precise edges and continuous large-building entities while avoiding the coarse noise introduced by shallow features.
Abstract
from arXiv · showhide
Accurately and efficiently extracting building footprints from a wide range of remote sensed imagery remains a challenge due to their complex structure, variety of scales and diverse appearances. Existing convolutional neural network (CNN)-based building extraction methods are complained that they cannot detect the tiny buildings because the spatial information of CNN feature maps are lost during repeated pooling operations of the CNN, and the large buildings still have inaccurate segmentation edges. Moreover, features extracted by a CNN are always partial which restricted by the size of the respective field, and large-scale buildings with low texture are always discontinuous and holey when extracted. This paper proposes a novel multi attending path neural network (MAP-Net) for accurately extracting multiscale building footprints and precise boundaries. MAP-Net learns spatial localization-preserved multiscale features through a multi-parallel path in which each stage is gradually generated to extract high-level semantic features with fixed resolution. Then, an attention module adaptively squeezes channel-wise features from each path for optimization, and a pyramid spatial pooling module captures global dependency for refining discontinuous building footprints. Experimental results show that MAP-Net outperforms state-of-the-art (SOTA) algorithms in boundary localization accuracy as well as continuity of large buildings. Specifically, our method achieved 0.68\%, 1.74\%, 1.46\% precision, and 1.50\%, 1.53\%, 0.82\% IoU score improvement without increasing computational complexity compared with the latest HRNetv2 on the Urban 3D, Deep Globe and WHU datasets, respectively. The TensorFlow implementation is available at https://github.com/lehaifeng/MAPNet.
I. INTRODUCTION
Building footprint extraction matters for applications including illegal-building monitoring, 3D reconstruction, urban planning, and disaster response, but buildings vary widely in appearance, structure, and scale. Existing CNN methods lose spatial detail, mislocalize boundaries, miss small buildings, and produce discontinuous large-building footprints, motivating MAP-Net.
- Building footprints support illegal-building monitoring, 3D reconstruction, urban planning, and disaster emergency response.
- CNN encoder-decoder methods lose spatial details during encoding and recover them with shallow-feature fusion, which can impair boundary localization and small-building recognition.
- Local receptive fields restrict extracted features, causing large low-texture buildings to appear discontinuous and holey.
- MAP-Net progressively generates parallel paths with fixed-resolution convolutions, combines multiscale features using channel attention, and applies pyramid spatial pooling for global semantics.
- MAP-Net improves F1-score by 0.88%, 0.93%, and 0.45% over HRNetv2 on Urban 3D, Deep Globe, and WHU, respectively.
II. METHODOLOGE
MAP-Net combines localization-preserved multiscale feature extraction, attention-based feature enhancement, spatial pooling, and interpolation-based upsampling. Its design targets lost spatial localization, coarse shallow semantics, and missing global dependency in building extraction.
- Repeated pooling or strided convolution loses spatial localization, while shallow-feature fusion introduces coarse semantic noise.
- MAP-Net uses a parallel multi-path network to extract multiscale high-level semantics while preserving spatial details at fixed resolution.
- Multi-path features are upsampled by bilinear interpolation, concatenated, adaptively optimized with attention, and refined through spatial pooling.
- The spatial pooling module captures global dependency to suppress holes and obtain more continuous building footprints.
B. Localization-Preserved Multi-path Network
The localization-preserved multi-path network maintains separate fixed-resolution paths while progressively adding lower-resolution, higher-channel paths. Higher-resolution paths retain localization details, whereas lower-resolution paths capture broader semantic information.
- MAP-Net extracts multiscale features with rich semantics and spatial localization instead of recovering localization by fusing shallow decoder features.
- Each stage generates a new path with downsampled resolution and doubled channels for higher-level semantic representations.
- Features remain unfused between stages, and each path maintains its spatial resolution throughout feature extraction.
- Higher-resolution features preserve localization details, while lower-resolution features provide richer global semantics.
- The three paths use resolutions of 1/4, 1/8, and 1/16 of the input, with 64, 128, and 256 corresponding channels.
C. Attention-Based Feature Squeeze
MAP-Net aligns and concatenates multiscale path features, uses channel attention to reweight them, and applies spatial pooling to capture global dependencies. Its building-footprint head upsamples the representation to produce a full-resolution probability map.
- Multiscale path features are bilinearly upsampled to a common resolution and concatenated before adaptive channel-wise optimization.
- Channel attention measures channel significance and reweights features to reconstruct enhanced multiscale representations.
- Spatial pooling uses four differently sized average-pooling layers to capture global relations and improve building integrity.
- The extraction module restores resolution through two bilinear-interpolation stages and outputs a single-channel building-probability map matching the input resolution.
III. EXPERIMENT AND ANALYSIS
MAP-Net was evaluated on three open building-extraction datasets spanning aerial and satellite imagery, varied geographic areas, resolutions, and image sizes.
- The experiments used the WHU, Deep Globe Building Extraction Challenge, and USSOCOM Urban 3D Challenge datasets.
- WHU aerial imagery contains over 187,000 buildings across more than 450 km2 at 30 cm resolution, divided into 4,736 training, 2,416 test, and 1,036 validation tiles.
- Deep Globe imagery includes Las Vegas and Shanghai subsets with approximately 243,382 buildings across more than 1,216 km2 at 30 cm resolution.
- Urban 3D contains approximately 157,000 buildings across more than 360 km2 at 50 cm resolution, with 204 orthorectified RGB images.
B. Evaluation Metric
The study evaluates pixel-wise semantic segmentation using precision, recall, F1-score, and IoU, with IoU also expressed through precision and recall.
- Pixel-level metrics were selected because building footprint extraction classifies every image pixel as building or non-building.
- Precision measures true positives among positive predictions, while recall measures true positives among all positive samples.
- IoU = Precision ∗Recall Precision + Recall −Precision ∗Recall.
1) Significance of Multi-path:
The multi-path design preserves spatial localization while developing deeper semantic features at fixed resolutions, supporting small-building recognition and boundary extraction. Structure experiments favored three paths and four residual blocks as a balance of accuracy and complexity, although optimal settings may depend on the dataset.
- Feature-map analysis: Deeper fixed-resolution feature maps retain richer semantics, whereas decreasing resolution blurs building edges and can eliminate small buildings.
- Feature-map analysis: Encoder-decoder skip connections recover localization using shallow features but introduce noise, while lost small buildings cannot be accurately refined during decoding.
- Feature-map analysis: Parallel paths keep each resolution fixed and fuse multi-scale features only after extraction, preserving localization and semantic information for fine boundaries and small buildings.
- Network-structure analysis: IoU initially increased with residual-block depth before declining beyond a dataset-specific value, while trainable parameters grew linearly with the number of blocks.
- Network-structure analysis: Trainable parameters increased exponentially with the number of paths because each new path doubled feature channels.
- Network-structure analysis: Three paths achieved better IoU than two or four, and three paths with four residual blocks provided the best accuracy-complexity balance on WHU.
- Network-structure analysis: Optimal path configurations may vary with input-image resolution, while feature-map channels and input resolution also affect performance and complexity.
3) Performance Evaluation:
MAP-Net was compared with established segmentation and building-extraction methods across WHU, Deep Globe, and Urban 3D. It reported higher segmentation scores and improved recognition of small and large buildings with more accurate boundaries.
- Benchmark comparison: MAP-Net was compared with U-Net, PSPNet, ResNet101, and HRNetv2 on three building-extraction datasets.
- Quantitative results: 0.82%, 1.50%, and 1.53% IoU improvements over HRNetv2 were reported on WHU, Deep Globe, and Urban 3D, respectively.
- Quantitative results: 0.45%, 0.93%, and 0.88% F1-score improvements over HRNetv2 were reported on WHU, Deep Globe, and Urban 3D, respectively.
- Qualitative results: Visual examples show more accurate recognition of small buildings, more complete extraction of large buildings, and boundaries closer to ground truth.
4) Comparison of Recent Methods:
MAP-Net outperformed recent building-extraction methods on the WHU dataset, achieving the strongest reported IoU and F1-Score without pre-training or post-processing.
- The comparison included CU-Net, SiU-Net, SRI-Net, DE-Net, EU-Net, and MA-FCN on the WHU test dataset.Their results were not reproduced because source code was unavailable.
- SRI-Net achieved 89.23% IoU and EU-Net achieved 90.56% IoU on the WHU dataset among the referenced prior methods.
- MAP-Net achieved 90.86% IoU and 95.21% F1-Score on the WHU dataset, outperforming the compared recent studies.
- MAP-Net slightly outperformed MA-FCN despite limited improvement space, without pre-training or post-processing.
5) Ablation Experiments:
Ablation experiments evaluated localization-preserved multi-path extraction, channel-wise squeezing, spatial enhancement, and shallow-feature skip connections, showing accuracy gains alongside boundary trade-offs.
- The experiments assessed IoU, precision, recall, and F1-score across localization-preserved baselines and MAP-Net modules on the WHU dataset.
- The modified multi-path localization-preserved baseline exceeded HRNetv2 by 0.24% IoU and HRNetv1 by 0.74% IoU.
- Channel-wise feature squeezing improved IoU by 0.29%, while spatial pooling enhancement improved IoU by 0.34%.
- Together, the feature squeeze and global enhancement modules increased IoU by 0.58% over the baseline.
- Adding shallow features improved accuracy by 0.04% but introduced coarse noise and inaccurate building boundaries.Without shallow features, MAP-Net produced smoother edges with little accuracy loss.
- MAP-Net methods achieved higher precision than HRNetv2 at threshold 0.5, with the authors attributing this to suppressed false-positive predictions.
6) Complexity of MAP-Net:
MAP-Net was designed to balance multi-scale feature preservation with computational cost, achieving improved performance with fewer FLOPs than HRNetv2 despite feature-enhancement overhead.
- The complexity study compared FLOPs, trainable parameters, and IoU scores of related methods on the WHU dataset.
- U-NetPlus had the lowest complexity but poor performance, whereas ResNet101 was the most complicated model.
- HRNetv2 used slightly more parameters than HRNetv1 and performed better through decoder-stage concatenation of multi-scale features.
- MAP-Net’s baseline reduced complexity and improved performance relative to HRNetv2 through encoder redesign.
- Feature enhancement increased MAP-Net’s complexity but substantially improved performance while retaining far fewer FLOPs than HRNetv2.
- MAP-Net maintained high-resolution feature maps with relatively few channels to extract multi-scale features efficiently.
IV. DISCUSSION AND CONCLUSION
MAP-Net combines localization-preserved multi-path extraction with channel and spatial feature enhancement to improve multiscale building footprints, boundaries, and continuity. Across three benchmarks, it achieved higher accuracy and lower complexity than classic methods and reached an SOTA result on WHU, while broader multiclass extraction remains future work.
- MAP-Net addresses inaccurate boundaries, missed small buildings, and discontinuous large-building footprints with localization-preserved multi-path feature extraction.
- Parallel paths preserve multiscale local details and semantic representations for accurate edges and recognition of small buildings.
- Experiments on three benchmarks showed higher accuracy and lower complexity than classic semantic-segmentation algorithms, with an SOTA result on WHU.
- Ablation experiments supported higher precision from localization-preserved multi-path building extraction than previous methods.
- The experiments currently cover building extraction, while multiclass tasks such as land-cover extraction are reserved for future work.