Source-linked AI summary
Topological Map Extraction from Overhead Images
Zuoyue Li, Jan Dirk Wegner, Aurélien Lucchi
TL;DR
PolyMapper addresses the mismatch between pixel-wise segmentation and applications needing graph or polygon outputs by directly extracting vectorized topological maps from aerial imagery. Its end-to-end model and graph-to-polygon representation perform better than or on par with state-of-the-art methods for building footprints and road networks.
Problem
Many vision applications require graph or polygon representations, while conventional aerial-image mapping typically produces raster maps that require subsequent vectorization and refinement.
Method
PolyMapper uses an end-to-end CNN-RNN architecture to directly extract building footprints and road networks as polygons, with a sequentialization method that reformulates graph structures as closed polygons.
Results
PolyMapper performs better than or on par with state-of-the-art methods for delineating building footprints and road networks across multiple datasets.
Takeaways & Limitations
The polygon-based output produces topological structures that are closer to real map representations than conventional per-pixel masks.
Takeaways & Limitations
The road-network reformulation assumes that each directed edge’s partner is always on its left when facing the direction of travel.
Abstract
from arXiv · showhide
We propose a new approach, named PolyMapper, to circumvent the conventional pixel-wise segmentation of (aerial) images and predict objects in a vector representation directly. PolyMapper directly extracts the topological map of a city from overhead images as collections of building footprints and road networks. In order to unify the shape representation for different types of objects, we also propose a novel sequentialization method that reformulates a graph structure as closed polygons. Experiments are conducted on both existing and self-collected large-scale datasets of several cities. Our empirical results demonstrate that our end-to-end learnable model is capable of drawing polygons of building footprints and road networks that very closely approximate the structure of existing online map services, in a fully automated manner. Quantitative and qualitative comparison to the state-of-the-art also shows that our approach achieves good levels of performance. To the best of our knowledge, the automatic extraction of large-scale topological maps is a novel contribution in the remote sensing community that we believe will help develop models with more informed geometrical constraints.
1. Introduction
PolyMapper replaces conventional raster segmentation with an end-to-end architecture that directly predicts vectorized building and road maps from aerial imagery. It combines CNN and ConvLSTM components with a graph-to-polygon sequentialization strategy and performs competitively across mapping benchmarks.
- Pixel-wise segmentation is often only an intermediate step when applications require graph or polygon outputs describing geometric objects.
- Automated map generation commonly converts orthophotos into raster maps before shape refinement, vectorization, and generalization.
- PolyMapper turns this multi-step workflow into an end-to-end architecture that directly outputs topological maps of buildings and roads from aerial imagery.
- The model uses CNNs to extract keypoints and edge evidence, then feeds them sequentially to ConvLSTM modules that produce vector representations for objects.
- For roads, a maze-solving procedure reformulates graph topology as polygons, after which tile-level roads and buildings are combined into a city map.
- Experiments on public and newly collected datasets show performance that outperforms or matches state-of-the-art methods across building and road delineation.
2. Related work
Prior remote-sensing approaches largely produce pixel-level outputs, use multi-step processing, or target only one object type. PolyMapper addresses this gap with automated polygon-based extraction of both buildings and roads.
- Building-footprint extraction has used multi-step methods combining multispectral imagery and airborne LiDAR, alongside newer convolutional models using optical imagery and surface models.
- Road-network extraction evolved from local image processing toward methods modeling long structures, connectivity, higher-order relations, and centerlines.
- Earlier deep-learning road methods used separate networks or graph-based approaches to detect roads, repair gaps, or predict network structure.
- Direct polygon prediction has been explored with level sets, active contours, unsupervised methods, and CNN-RNN systems, but some approaches require user-provided bounding boxes.
- Prior work generally focused on pixel-level outputs or a single object type, motivating fully automated extraction of building and road geometries from overhead imagery.
3. Method
PolyMapper extracts building and road maps directly as polygons, replacing a multi-step raster-mapping workflow with an end-to-end learnable architecture. It sequentializes road graphs into polygon-like representations so one model can handle both buildings and road networks.
- PolyMapper introduces a generic polygon-based approach for extracting topological maps from aerial images.The method targets vector representations rather than conventional pixel-wise outputs.
- Buildings: The workflow detects building instances before predicting separate building polygons, addressing multiple buildings within one image patch.An FPN adds bounding-box detection and partitions the image into individual building instances.
- Roads: Road graphs are converted into ordered polygon sequences using wall-follower traversal rules that connect intersections and dead ends by travel order.The procedure turns bidirected road segments into sequences by walking forward, turning right at intersections, and reversing at dead ends.
- Roads: Larger or denser road patches can contain multiple polygons, requiring traversal of all directed edges until every road segment has been visited.Unvisited directed edges initiate additional traversals after an arbitrary polygon is processed.
- Pipeline: The CNN extracts image features, boundaries or centerlines, and candidate keypoints, while ConvLSTM modules sequentially predict object vertices until an end-of-sequence signal.For roads, the model predicts an outer polygon and potential inner polygons using edge-based starting candidates; predicted segments are then connected into the road graph.
4. Experiments
PolyMapper is evaluated separately for building footprints and road networks on task-specific datasets, plus a self-collected dataset spanning multiple cities. Results assess both output quality and structural properties against established methods.
- Datasets: The evaluation uses crowdAI for building footprints and RoadTracer for road networks because no large-scale public dataset combines both annotations with aerial imagery.The self-collected PolyMapper dataset addresses this gap, while task-specific benchmarks support comparisons with prior methods.
- Evaluation Measures: Building extraction is measured with COCO average precision and recall metrics, using mask IoU despite PolyMapper producing polygon representations.Reported measures include AP, AP50, AP75, APS, APM, APL, and AR.
- Evaluation Measures: Road topology is evaluated with shortest-path connectivity and a proposed AP/AR-based measure comparing predicted and ground-truth graph paths.SP counts path-length matches within a 10% buffer, while the proposed measure enables evaluation analogous to building footprints.
- Building Results: On crowdAI, PolyMapper outperforms Mask R-CNN and PANet on all AP and AR metrics except APL for large buildings.The authors attribute the large-building exception to less accurate vertex localization after resizing large feature maps.
- Road Results: On RoadTracer, PolyMapper outperforms DeepRoadMapper across SP, AP, and AR, while performing on par with RoadTracer.The road test uses images from 15 cities, with training on 25 other cities, providing an indication of generalization to new scenes.
- PolyMapper Dataset Results: On the PolyMapper dataset, PolyMapper outperforms both road baselines across all measures and matches building baselines overall while retaining AP and AR advantages.Results are averaged across Boston, Chicago, and Sunnyvale, with testing performed on different areas of each city.
5. Conclusion
PolyMapper directly extracts topological maps from overhead imagery using a CNN-RNN architecture and reformulates graph structures as closed polygons. Across varied datasets, it achieves strong building and road delineation while producing vectorized structures closer to online maps than per-pixel masks.
- PolyMapper directly extracts topological maps from city overhead imagery with a CNN-RNN architecture.
- Its graph-sequentialization method represents different object types as closed polygons, unifying their shape representations.
- Experiments across varied datasets show high performance for delineating building footprints and road networks from raw aerial images.
- PolyMapper performs better or on par with state-of-the-art methods specialized for building or road extraction.
- Its topological outputs are closer to online map structures and more natural and less redundant than conventional per-pixel masks.