Source-linked AI summary
SegmentAnyTree: A sensor and platform agnostic deep learning model for tree segmentation using laser scanning data
Maciej Wielgosz, Stefano Puliti, Binbin Xiang, Konrad Schindler, Rasmus Astrup
TL;DR
Individual tree crown segmentation methods are often tailored to particular lidar platforms, limiting transfer across sensor types and data densities. The study trains a sensor-agnostic model using ULS, MLS, and synthetically sparsified point clouds. It reports improved detection and segmentation, including for trees under the canopy, alongside reduced computational demands compared with prior methods.
Problem
Many individual tree crown methods are platform-specific, and above-canopy ALS methods have had difficulty detecting understory trees and transferring to sparser data.
Source: §1 · pp. 2–3Method
Results
SegmentAnyTree advances the state of the art through improved detection and segmentation of under-canopy trees while reducing computing resources compared with previous methods.
Source: §4 · p. 23Takeaways & Limitations
The study demonstrates feasibility of a sensor-agnostic model across airborne and terrestrial lidar resolutions and suggests convergence between traditionally separate segmentation approaches.
Source: §4 · p. 23Performance is poorer in multilayered broadleaved forests, and further improvement may require training data from survey-grade TLS and more complex forest types.
Source: §4.1 · p. 15, §4 · p. 23
Abstract
from arXiv · showhide
This research advances individual tree crown (ITC) segmentation in lidar data, using a deep learning model applicable to various laser scanning types: airborne (ULS), terrestrial (TLS), and mobile (MLS). It addresses the challenge of transferability across different data characteristics in 3D forest scene analysis. The study evaluates the model's performance based on platform (ULS, MLS) and data density, testing five scenarios with varying input data, including sparse versions, to gauge adaptability and canopy layer efficacy. The model, based on PointGroup architecture, is a 3D CNN with separate heads for semantic and instance segmentation, validated on diverse point cloud datasets. Results show point cloud sparsification enhances performance, aiding sparse data handling and improving detection in dense forests. The model performs well with >50 points per sq. m densities but less so at 10 points per sq. m due to higher omission rates. It outperforms existing methods (e.g., Point2Tree, TLS2trees) in detection, omission, commission rates, and F1 score, setting new benchmarks on LAUTx, Wytham Woods, and TreeLearn datasets. In conclusion, this study shows the feasibility of a sensor-agnostic model for diverse lidar data, surpassing sensor-specific approaches and setting new standards in tree segmentation, particularly in complex forests. This contributes to future ecological modeling and forest management advancements.
1. Introduction
Existing individual tree crown methods are often sensor-specific, while transfer across lidar platforms remains limited. This study proposes training with synthetically sparsified proximal scans to extend tree segmentation across airborne and terrestrial data.
1. Introduction
- Most ITC methods are tailored to particular lidar platforms, leaving comparatively little work on models transferable across sensors without dataset-specific retuning.The proposed sensor-agnostic approach aims to apply consistently across lidar datasets.Source: §1 · pp. 2, 4
- Traditional ALS methods often use canopy height models that miss understory trees, while seed-point window sizes require empirical tuning for new datasets.The cited benchmark discussion reports a 52% average detection rate and identifies poor detection of co-dominant and understory trees as a key limitation.Source: §1 · pp. 2–3
- Dense TLS, MLS, and ULS data support detailed annotations, but methods tailored to proximal scans are not directly transferable to above-canopy ALS.The paper notes that manual editing of proximal-scan crown segments can also bottleneck use of those data.Source: §1 · p. 3
- Deep-learning ITC methods for ALS remained relatively few and relied primarily on very dense data, while applicability to sparser ALS was identified as necessary.The cited ULS model achieved an 82.3% detection rate versus 69.6% for the compared model, but results were limited to data above 1000 points/m2.Source: §1 · p. 3
- The study trains and validates a sensor-agnostic tree-segmentation model by augmenting labeled TLS/MLS-derived data with synthetic sparsification.The authors propose this as a way to extend proximal-scan methods to lower-density ALS data.Source: §1 · pp. 3–4
2. Materials
The study combines annotated ULS and MLS training data with a test collection spanning airborne, mobile, and terrestrial scans. Randomly subsampled ULS and MLS point clouds provide test cases at lower densities.
2. Materials
- The training and evaluation collection comprises in-house and public forest scans with manually annotated unique tree identifiers.The FOR-instance ULS data use a fixed 70% development and 30% test split, while MLS test data follow the split used by Wielgosz et al. (2023).Source: §2 · p. 4
- ULS and MLS point clouds were randomly subsampled to 1000, 500, 100, and 10 points/m2 to represent a range of airborne densities.The described range spans helicopter-based high-density ALS through conventional airborne capture and standard ALS.Source: §2 · p. 4
- The test data include MLS plots from NIBIO and public TLS/MLS benchmarks including Wytham Woods, TreeLearn, and LAUTx.Wytham Woods represents a structurally diverse broadleaved forest; LAUTx includes broadleaved, coniferous, and mixed forest types.Source: §2 · pp. 5–6
- FOR-instance test results are evaluated separately across five regional subsets to represent performance across different forest types.The subsets are NIBIO, CULS, TU_WIEN, RMIT, and SCION.Source: §2 · p. 6
3. Methods
The methods hold model form and hyperparameters constant while varying training inputs across ULS, MLS, and sparsified data. Evaluation measures transfer across platforms, densities, and canopy height layers using annotated test datasets.
3. Methods
- The study keeps model form and hyperparameters constant while training models on ULS, MLS, and combinations of their sparsified versions.Five scenarios range from a single platform to combined data with progressively broader sparsification.Source: §3 · pp. 6, 8, 9
- Platform comparisons contrast models trained only on ULS or only on MLS, while density comparisons assess sparsification as training augmentation.The density analysis targets transfer to ALS-like data, including densities of 10 points/m2.Source: §3 · pp. 7–8
- The best-performing model is tested on FOR-instance and its sparsified versions to assess transfer across decreasing point densities.The resolution analysis is limited to these data because comparable sparsification is not considered meaningful for naturally dense terrestrial datasets.Source: §3 · p. 8
- Canopy-profile performance is assessed by grouping trees into 5 m height bins and calculating metrics for each bin.Tree height is used to define layers because canopy social-status information is unavailable.Source: §3 · p. 8
- The PointGroup-based network uses a 3D CNN with separate semantic and instance-segmentation prediction heads.Instance branches predict tree-centering offsets and embeddings; the embeddings are clustered and refined by ScoreNet.Source: §3 · pp. 8–9
- Tree instances are matched to ground truth by point-level IoU, with a detection requiring IoU > 0.5; evaluation includes detection, omission, and commission rates.Tree-height RMSE complements detection and F1 metrics by assessing segmentation along the tree's full length.Source: §3 · pp. 9–10
4. Results and discussion
The study finds that sparsification improves cross-platform tree segmentation, while performance remains sensitive to forest structure and very low point density. Its strongest benchmark gains are on LAUTx and NIBIO_MLS, with poorer results on some complex forests.
4.1.1 Comparison between airborne and terrestrial data as source for model training
- ULS-trained models had higher detection, lower commission, and higher F1 than MLS-trained models, although differences were marginal on most datasets.On TUWIEN, ULS training yielded approximately 20% higher detection and approximately 20% lower commission errors than MLS training.Source: §4.1 · pp. 10–11, Figure 3 · p. 11
4.1.2 Comparison of augmentation strategies
- Scenarios 4 and 5 averaged detection rates of 0.76 and 0.77, above scenarios 1–3 at 0.70, 0.67, and 0.72, respectively.Scenario 5 increased detection by up to 14 percentage points on TUWIEN versus scenario 3; aggressive sparsification also boosted detection on several dense terrestrial datasets.Source: Table 1 · p. 12, §4.1 · p. 12
4.2 Performance for different point cloud resolutions
- Across FOR-instance, performance was stable above 50 points m-2 but declined at 10 points m-2, mainly through an approximately 20% increase in omission.Commission errors and F1 score were less affected by lower point density.Source: Figure 4 · p. 14, §4.1 · p. 14
4.3 Performance across canopy profile
- The model segmented trees throughout the canopy profile, including understory trees under 10 m with detection rates of 0.2–0.8, though results varied by forest type and canopy structure.Performance was generally poorer in multilayered broadleaved forests; Wytham Woods’ dense understory and survey-grade TLS data absent from training were relevant scope constraints.Source: §4.1 · p. 15, Figure 5 · p. 16
4.4 Computational efficiency
- The proposed method’s computational-efficiency metric was an order of magnitude better than TLS2trees, under assumptions of one GPU and 64 versus 200 CPU cores.The comparison assumes both approaches use one GPU and does not account for GPU architecture details.Source: Table 2 · p. 17, §4.1 · p. 17
4.6 Advancing the state-of-the-art
- On LAUTx, the method outperformed Point2Tree and TLS2trees by approximately 20 percentage points across detection, omission, commission, and F1; on NIBIO_MLS, detection improved 17–20% and F1 26–27%.The LAUTx gains were approximately 25–30% against TLS2trees; NIBIO_MLS improvements in commission errors and tree-height RMSE were more modest.Source: §4.1 · pp. 20–21, Table 3 · p. 21
- Wytham Woods was among the poorest-performing datasets, with detection 0.53, omission 0.47, commission 0.27, RMSE H 4.2, and F1-score 0.88.The authors identify complex forest structure and the absence of labeled survey-grade TLS from training as relevant constraints; some understory and intertwined co-dominant trees were missed in Wytham Woods and TUWIEN.Source: §4.1 · pp. 18, 20, Table 3 · p. 21
- Across benchmark datasets, the method matched or exceeded prior FOR-instance results and substantially surpassed Point2Tree and TLS2trees on LAUTx and NIBIO_MLS.It had the highest detection rates across tested datasets, while TreeLearn F1 was 0.92 versus 0.98 for Henrich et al. (2023).Source: §4.1 · pp. 20, 22
4. Conclusion
SegmentAnyTree demonstrates the feasibility of a sensor-agnostic model spanning airborne and terrestrial laser-scanning resolutions, with improved performance from diverse data and augmentation. The authors report a state-of-the-art advance, especially for detecting and segmenting trees beneath the canopy, while identifying training-data gaps for further improvement.
4. Conclusion
- Training with diverse platform and density data, together with sparsification augmentation, improved performance and supports sensor-agnostic use across airborne and terrestrial laser-scanning resolutions.The study suggests this may align approaches that have traditionally treated airborne and terrestrial segmentation separately.Source: §4 · p. 23
- Open-source benchmarking datasets provide a transparent way to assess future improvements in tree-segmentation methods, unlike studies based on separate regional datasets.The authors describe this as a new way of working for the forest remote-sensing community.Source: §4 · p. 23
- The model advances the state of the art, particularly in detecting and segmenting dominated trees beneath the canopy, while reducing computing resources relative to previous methods.The results also provide a benchmark for prominent open-source datasets.Source: §4 · p. 23
- Adding survey-grade TLS data and more complex forest types to training could further improve SegmentAnyTree; performance is already described as very good in coniferous-dominated forest.The authors identify these data and forest types as areas for expanding training coverage.Source: §4 · p. 23
APPENDIX
The appendix shows scenario 5 predictions on unseen dense airborne laser-scanning data and presents model performance across sparsified FOR-instance datasets. The examples highlight segmentation of understory or dominated trees within dense groups.
APPENDIX
- Table A.1 reports scenario 5 model performance across the different sparsified FOR-instance datasets.The caption gives the table's scope but not its performance values.Source: Table A.1 · p. 25
- Scenario 5 predictions are illustrated on unseen dense airborne laser-scanning data from helicopter-, manned-aircraft-, and consumer-drone-mounted systems.The examples include survey-grade and consumer-grade data.Source: Figure A.1 · p. 26
- The examples emphasize segmentation of understory or dominated trees even when they occur in dense groups.The caption identifies this as the model's main improvement over previous methods applied to ALS data.Source: Figure A.1 · p. 26