Source-linked AI summary
From Google Maps to a Fine-Grained Catalog of Street trees
Steve Branson, Jan Dirk Wegner, David Hall, Nico Lang, Konrad Schindler, Pietro Perona
TL;DR
Municipalities need up-to-date urban tree inventories, but trees are still often mapped manually and prior automation commonly relies on dedicated sensing campaigns. The paper uses CNNs on publicly available aerial and street-view imagery to detect trees, recognize fine-grained species, and track changes over time. In Pasadena, it detects more than 70% of street trees, recognizes more than 80% of 40 species, and correctly detects and classifies changes in more than 90% of cases.
Problem
Municipalities need up-to-date urban tree inventories, while trees remain mostly mapped manually and existing automated methods often rely on dedicated airborne LiDAR or hyperspectral campaigns.
Method
The pipeline uses CNNs with publicly available aerial images and street-level panoramas for multi-view tree detection, fine-grained species recognition, and temporal change tracking.
Results
The Pasadena system detects > 70% of street trees, achieves > 80% species recognition accuracy for 40 species, and correctly detects and classifies changes in > 90% of cases.
Takeaways & Limitations
Publicly available RGB imagery can support automated urban tree inventory generation, species recognition, and city-scale updates across many thousands of trees.
Takeaways & Limitations
The change tracker is vulnerable to inaccurate street-view heading metadata, and rare replacement cases lacked enough training data for a dedicated class.
Abstract
from arXiv · showhide
Up-to-date catalogs of the urban tree population are important for municipalities to monitor and improve quality of life in cities. Despite much research on automation of tree mapping, mainly relying on dedicated airborne LiDAR or hyperspectral campaigns, trees are still mostly mapped manually in practice. We present a fully automated tree detection and species recognition pipeline to process thousands of trees within a few hours using publicly available aerial and street view images of Google MapsTM. These data provide rich information (viewpoints, scales) from global tree shapes to bark textures. Our work-flow is built around a supervised classification that automatically learns the most discriminative features from thousands of trees and corresponding, public tree inventory data. In addition, we introduce a change tracker to keep urban tree inventories up-to-date. Changes of individual trees are recognized at city-scale by comparing street-level images of the same tree location at two different times. Drawing on recent advances in computer vision and machine learning, we apply convolutional neural networks (CNN) for all classification tasks. We propose the following pipeline: download all available panoramas and overhead images of an area of interest, detect trees per image and combine multi-view detections in a probabilistic framework, adding prior knowledge; recognize fine-grained species of detected trees. In a later, separate module, track trees over time and identify the type of change. We believe this is the first work to exploit publicly available image data for fine-grained tree mapping at city-scale, respectively over many thousands of trees. Experiments in the city of Pasadena, California, USA show that we can detect > 70% of the street trees, assign correct species to > 80% for 40 different species, and correctly detect and classify changes in > 90% of the cases.
1. Introduction
The paper targets the costly, incomplete manual maintenance of urban tree inventories with an automated system using publicly available aerial and street-level imagery. It combines learned tree detection, species recognition, and inventory updating to support large-scale coverage.
- California has 9.1 million street trees, whose ecosystem services provide substantial community benefits beyond their estimated $2.5 billion replacement value.Reported benefits include reduced energy use, improved air and water quality, and increased carbon capture and storage.
- The proposed system uses publicly available aerial images and street-level panoramas to detect trees and recognize their species automatically at large scale.It uses supervised deep CNN classification learned from existing tree inventories.
- Existing inventory efforts remain incomplete because few citizen scientists participate, leaving only about 1000 of more than 80,000 Pasadena trees mapped in the last three years.
- Public imagery could enable low-cost, immediate inventory generation and help cities assess urban-forest diversity for management decisions.The motivation includes identifying species relevant to risks such as pest arrival.
- The workflow downloads imagery, separately trains a tree detector and species classifier where ground truth exists, and applies them to new images.Nearby, reasonably recent inventories can provide training data when available.
2. Related work
Prior urban tree-mapping methods commonly depend on specialized sensors, hand-designed procedures, or limited datasets. The paper instead extends RGB-image, deep-learning-based mapping with integrated species recognition and temporal change tracking.
- Single-aerial-image approaches such as marked point processes work best for separated trees but are brittle, difficult to tune, and computationally expensive.
- Earlier urban methods segment aerial imagery or point clouds and classify regions using scale-space analysis, shape priors, active contours, or related rules.
- A recent RGB-only urban detector used CRFs, tree templates, and hierarchical rules, but addressed no species recognition and was demonstrated on limited data.Its ability to scale remained unclear.
- Most remote-sensing pipelines require dedicated LiDAR, hyperspectral, or RGB-NIR campaigns, limiting sensor generalization, dataset size, and temporal coverage.They also require new tree detection and species ground truth for each test site, leaving large-scale practical impact unclear.
- This paper relies on publicly available standard RGB imagery, integrates detection with species recognition, and adds Siamese-CNN change tracking across time.The system is evaluated on a large-scale dataset with detailed failure analysis.
3. Detection of trees
The detector converts per-view tree proposals into geographic coordinates, reassesses them across aerial and street views, and combines evidence with learned spatial and map priors. A CRF and greedy inference produce geographic tree detections robust to missing single-view detections.
- 3. Detection of trees: A Faster R-CNN detector generates liberal region proposals and scores from CNN features for each test image, using pretrained Pascal VOC weights to initialize training.
- 3.1. Multi-view detection: Multi-view detection collects proposals geographically, reprojects them into every view, recomputes scores, and suppresses overlaps to obtain tree locations.This avoids directly comparing incompatible bounding-box coordinates across aerial and street views.
- 3.1. Multi-view detection: The workflow remains robust when a tree is initially missed in one view because its geographic proposal is rescored in views lacking an initial detection.
- 3.2. Probabilistic model: A CRF combines aerial and street-view detection scores with priors for road distance and neighboring-tree spacing, using learned potential parameters and weights.The model selects geographic tree objects using image evidence, semantic map data, and spatial context.
- 3.3. Training and inference of the full model: Learned priors penalize trees closer than 2m or farther than 32m from neighbors and closer than 0.25m or farther than 8m from roads.The learned potential weights indicate that street-view and aerial imagery are the most important CRF inputs.
4. Tree species classification
The paper treats tree species recognition as fine-grained object recognition and combines multiple image views with CNN-derived features to classify species.
- The system recognizes tree species as subclasses within the broader object category of trees, framing the task as fine-grained object recognition.
- Given tree detections, it uses one aerial image and three increasingly zoomed street-view crops from the closest panorama for each tree.
- Four separately trained VGG16 CNNs extract view-specific features that are concatenated and classified with a linear SVM.
5. Tree change tracking
The change-tracking module addresses errors from repeated detection and uneven street-view revisits by comparing images for individual trees over time with a Siamese CNN.
- Naively comparing two detector runs requires manual filtering because position errors can make one tree appear at two locations and produce false changes.The reported position discrepancies are approximately 1m–8m.
- Uneven street-view revisit frequency prevents city-wide comparisons at identical times, so the method compares each tree whenever new imagery becomes available.Main roads are mapped frequently, whereas side roads have few images.
- The tracker uses a Siamese CNN with identical weight-sharing branches to extract features from multiple input images for change assessment.Shared weights encode the assumption that the inputs have identical statistics while reducing learnable parameters.
- The first tracker version requires geographic positions for accurate per-tree image comparison, but can use positions supplied by GIS data.
6. Experiments
Experiments in Pasadena evaluate the pipeline on large-scale imagery and tree inventories, measuring detection, species recognition, and change-tracking performance. Results show that combining aerial and street-view evidence improves detection, while species recognition depends on sufficient training examples and change classification is affected by image-alignment issues.
- 6.3. Tree detection results: 0.706 mAP for the full detector exceeds 0.42 mAP using aerial images alone and 0.581 mAP using street-view images alone.The full model combines multiple views; aerial imagery helps identify trees, while street views better localize trunks, especially under dense canopy.
- 6.3. Tree detection results: At least 5% mAP could be recovered by excluding private-land trees from evaluation with a public/private GIS layer.Many apparent false positives are trees near roads that are absent from the public-street-tree inventory.
- 6.3.1. Detection Error Analysis: 520 manually inspected detection errors show that outdated inventories, private-land omissions, localization thresholds, and georeferencing errors materially affect evaluation.Some correct detections are penalized because the reference inventory is incomplete or older than the Google Maps imagery; 40 errors (7.7%) were near ground-truth trees but missed the distance threshold.
- 6.4. Tree species classification results: Average class precision exceeds 0.80 for the 40 most frequent species, while detected locations reach 0.88 versus 0.83 for ground-truth locations across 30 species.Detected locations can be better centered than inventory coordinates, but class precision drops when species have few training examples.
- 6.4. Tree species classification results: Species performance drops sharply below 10 training samples per class, and the authors estimate that more than 100 instances per species are needed in this setup.The estimate assumes four images per tree, a limited number of species, and a 70/20/10 train-validation-test split.
- 6.5. Tree change tracking results: Change-classification errors commonly arise from inaccurate panorama headings, which can shift the same tree away from the image center between 2011 and 2016.A replaced old tree may also reasonably receive either the removed or new label, suggesting a missing replacement category.
7. Conclusions
The pipeline automates urban tree inventory generation, species recognition, and change tracking, with encouraging Pasadena results and planned evaluation across more cities and datasets.
- >70% of Pasadena trees were detected, species recognition exceeded 80% for 40 frequent species, and change classification exceeded 90% across three categories.The change categories were unchanged, removed, and new tree planted.
- The approach requires at least 100 training instances per tree species for satisfying CNN performance, making training-data availability a practical constraint.
- The authors plan to test generalizability on more cities and geographic areas using additional datasets beyond Pasadena.They specifically motivate further research on more datasets, cities, and geographic areas based on standard RGB images.
- Species recognition was on par with citizen scientists, although automated detection performed worse than manual collection in the reported comparison.Citizen scientists missed 1.2% of trees, counted 1.0% extra trees, and matched expert species annotations for 84.8% of trees.
- Future extensions could estimate trunk diameter and tree stress to support age, biomass, pest, resource-management, and replanting assessments.The text presents these capabilities as possible extensions beyond detection and species recognition.
- Google Street View coverage can be incomplete, so crowd-sourced or national mapping imagery may be needed to fill geographic gaps.The authors cite Zurich, where only 60% of the city is covered by Google street-view panoramas because of legal issues.
Appendix A. Projections between Images and geographic coordinates
The appendix defines projections that convert geographic tree locations into image coordinates and detections back into geographic coordinates across street-view, aerial, and map imagery.
- Projection functions map geographic positions to image coordinates using camera or map parameters, while inverse aerial projection recovers latitude and longitude from detections.Street-view projection depends on camera position and orientation; aerial projection uses Web Mercator coordinates and zoom.
- Point-wise latitude/longitude tree annotations are converted into fixed-size training boxes, including 100×100 aerial-image boxes corresponding to approximately 12×12 meters.For street-view imagery, the appendix also describes boxes based on the projected extent of an 8 × 12 meter object.
- Map images share pixel alignment with aerial imagery and provide rasterized scene information for probabilistic spatial priors in the CRF framework.
- Street-view panoramas are 360° equidistant-cylindrical images recorded with camera geographic coordinates and heading, enabling local geometric projection.
- The street-view geometry assumes locally flat terrain and represents objects in East, North, Up coordinates relative to the camera.Tree locations are then related to camera-relative angles, height, distance, and heading.