PaperlayerView exact revision on arXiv

Source-linked AI summary

Mapping Woody Vegetation from Multi-Source Imagery and Prediction Fusion for Enhanced Data Efficiency and Accuracy

Kal Backman, Jared Wood, Adam Roff

arXiv:2608.26471v1cs.CV

TL;DR

Large-scale woody vegetation mapping must address both deep learning’s need for extensive labels and remote sensing imagery defects. This paper combines label transfer, multi-source image composition, and prediction fusion in a New South Wales framework, reducing errors by up to 76.2% and improving consistency across image dates.

  • Problem

    Deep learning offers spatially consistent vegetation maps but requires large amounts of labeled data, while clouds, shadows, lighting, and weather defects compromise imagery quality.

  • Method

    The framework transfers labels across multiple SPOT image sources and combines normalized multi-date imagery with multi-image prediction fusion for woody vegetation segmentation.

  • Results

    Training on multiple image sources reduced errors by up to 76.2% and reduced performance standard deviation across image dates by 13×; prediction fusion reduced overall accuracy error by 53.6%.

  • Takeaways & Limitations

    Multi-source learning produced the best-performing models while improving consistency across image dates and reducing the labels required relative to prior deep learning CNN approaches.

Abstract

from arXiv · show

Tree cover maps are a fundamental remote sensing product, used to derive ecological insights about the landscape and are essential to change detection, vegetation mapping and fire monitoring programs. However, comprehensive tree cover mapping requires reliable and high-quality imagery, free of cloud and weather defects to ensure accurate model outputs. Deep learning approaches can generate high quality maps with minimal human intervention but require large amounts of human annotated data to be successful. In this work we propose a framework consisting of methods that aim to improve the data efficiency and robustness of deep learning models using data fusion techniques to segment woody vegetation defined as vegetation over the height of 2m across the state of New South Wales, Australia. To improve robustness against varying image quality, we propose an image composition method that normalizes the imagery and removes defects, whilst also minimizing the reliance on individual image quality by proposing a prediction fusion method. The two methods resulted in an error reduction of 38.2% and 53.6% respectively compared to single-source imagery. To address deep learning approaches' limitation of requiring large amounts of data, we apply label transfer to multiple sources of imagery as a form of data augmentation to improve data efficiency. Learning from multiple image sources was shown to be the biggest improvement in performance, resulting in an error reduction between 28.1% to 76.2% across the different validation experiments, whilst reducing the standard deviation of performance across image dates by a factor of 13.

I. INTRODUCTION

The paper addresses deep learning’s data demands and imagery-quality limitations in large-scale woody vegetation mapping. It proposes label transfer, multi-date image composition, and prediction fusion within a statewide framework.

  • Research motivation: Deep learning improves spatial consistency but requires orders of magnitude more examples than traditional approaches.Traditional methods are more data-efficient but can produce speckled holes and isolated predictions because they use limited contextual fields.
  • Research motivation: Clouds, shadows, lighting, flooding, and snow make consistent high-quality imagery difficult to obtain across large study areas.Existing mitigation approaches include cloud masks, recaptured composites, time series imagery, and manual quality intervention.
  • Proposed framework: The framework segments woody vegetation defined as vegetation over 2 meters using a CNN encoder-decoder in a U-Net configuration.Its objectives are improved data efficiency and robustness to varying image quality.
  • Proposed framework: 801,137 km2 of New South Wales is covered by the proposed deep-learning woody vegetation segmentation framework.The framework is intended to map the entire state.
  • Proposed framework: Label transfer to multiple image sources is used as data augmentation to increase data efficiency and performance.The approach transfers partially labelled images across image sources.
  • Proposed framework: 53.6% error reduction is reported when multi-image composition and prediction fusion are coupled.The paper presents both methods as improving accuracy and robustness against image defects.

II. METHODOLOGY

The methodology normalizes and combines yearly SPOT imagery, then uses iterative uncertainty-guided labeling to expand training data. Operators label selected patches with confidence weighting and model-assisted pre-labels.

  • Multi-date image composition: 2019–2022 SPOT 6/7 imagery provides four yearly image sources across 801,137 km2 of mainland New South Wales.The 1.5m pansharpened imagery contains blue, green, red, and NIR bands and is divided into 343 scenes.
  • Multi-date image composition: A multi-date composite applies per-band normalization followed by a discrete median across four yearly images.The method requires no additional cloud masks or manual normalization-parameter intervention.
  • Multi-date image composition: Gaussian histogram fitting selects the lower-error single- or double-Gaussian model to normalize each image band.Each band is divided by the selected Gaussian mean; for double Gaussians, the lower mean is used.
  • Multi-date image composition: The discrete median chooses an actual normalized pixel value rather than averaging, preserving association with a specific image source.For an even number of values, it selects the lower of the two middle values.
  • Woody extent labels: Operators assign binary woody or non-woody labels to 512 × 512 patches, supported by Google Earth and Google Street View.An additional weighting mask records label confidence and supports partially completed patches.
  • Woody extent labels: 100 most uncertain patches per scene are proposed for labeling when their uncertainty exceeds a specified threshold.Uncertainty is computed from thresholded confidence values, with woody predictions weighted by 2; operators receive model predictions as pre-labels.
  • Woody extent labels: 1,365 labeled image patches were collected from 86 scenes through repeated training and uncertainty-guided patch selection.The process continued until satisfactory model performance was reached.

4) Training and validation dataset:

The dataset combines yearly SPOT imagery and a SPOT composite with transferred composite labels, then separates patches into training and validation sets while preventing corresponding temporal patches from crossing splits.

  • Dataset construction: Input bands are clipped at the lowest and highest 2.5% of values and rescaled to [-1, 1].This normalization is applied before dataset construction.
  • Dataset construction: The dataset includes 2019–2022 yearly SPOT imagery and the SPOT image composite to maximize data efficiency.Composite labels are transferred to the individual image sources.
  • Training and validation split: Approximately 55% of image patches are assigned to training and 45% to validation.Scenes with more than two samples are guaranteed at least one validation sample.
  • Training and validation split: Validation patches are applied across all input image sources, preventing corresponding patches from another year from entering training.For example, a validation patch from 2019 cannot have its corresponding 2020 patch in training.

B. Model architecture

The woody vegetation model uses a U-Net encoder-decoder to produce pixel-level confidence scores, trained on normalized, augmented imagery with a combined binary cross-entropy and Dice loss.

  • B. Model architecture: The model comprises an encoder, decoder, and head in a U-Net configuration for woody vegetation segmentation.The head outputs confidence scores for woody vegetation at each pixel.
  • B. Model architecture: The encoder generates five multiscale feature maps, while the decoder progressively upsamples and fuses them with corresponding encoder features.Feature-map channels increase from 64 to 1028 as resolution decreases from 1/2 to 1/32 of the input.
  • B. Model architecture: Training uses patches from composite and individual-date imagery with stochastic augmentation, including gamma corrections and band shifts.These transformations diversify the training examples across image sources and appearance conditions.
  • B. Model architecture: The loss is a weighted sum of binary cross-entropy and Dice loss, with weighting coefficients of 0.2 and 1.0 respectively.The label-derived weight mask incorporates operator confidence and ignores unlabeled pixels.

D. Inference & prediction fusion

Inference combines overlapping predictions and multiple imagery sources by weighting confidence according to spatial position, source quality, and cross-source class consistency.

  • D. Inference & prediction fusion: 1024 × 1024 patches are swept across scenes with 50% overlap, and overlapping predictions are fused using center-weighted confidence averages.Predictions near patch centers receive higher weights than predictions near edges.
  • D. Inference & prediction fusion: The class consistency coefficient is a weighted average of remapped classes multiplied by confidence across image sources.Weights derive from cloud masks, with an additional factor of 2 for SPOT composite imagery.
  • D. Inference & prediction fusion: The class consistency coefficient ranges from -1 to 1, representing high-confidence consistency for non-woody and woody vegetation at its extremes.A binary pixel classification is assigned by thresholding this coefficient.
  • D. Inference & prediction fusion: Final confidence is a weighted average of remapped source confidences multiplied by the squared class consistency coefficient.Disagreement between predicted classes causes greater confidence decay.

III. EXPERIMENT DESIGN

The experiment compares models trained on different combinations of imagery sources across multiple validation datasets using weighted overall accuracy.

  • III. EXPERIMENT DESIGN: Seven models were trained with varying source-image combinations, including four models trained separately on 2019, 2020, 2021, or 2022 SPOT imagery.The remaining models used broader combinations of the available image sources.
  • III. EXPERIMENT DESIGN: Each model was evaluated on each of seven validation datasets to compare cross-source generalization.This design tests how training-source composition affects performance across validation datasets.
  • III. EXPERIMENT DESIGN: Performance was ranked using weighted overall accuracy, defined from weighted true positives, true negatives, false positives, and false negatives.The metric accounts for incomplete labels and uncertainty in distinguishing vegetation above or below 2m.

B. Fisher et al. dataset evaluation

The Fisher et al. evaluation tests the trained models on a manually classified 2011 point dataset using woody, non-woody, and overall accuracy.

  • B. Fisher et al. dataset evaluation: Each trained model was evaluated on the Fisher et al. dataset using the image source on which that model was trained.The evaluation compares model predictions against the labeled point dataset.
  • B. Fisher et al. dataset evaluation: Evaluation reports woody accuracy, non-woody accuracy, and overall accuracy for the models and the Fisher et al. model.The results are presented in Table IV.

C. Independent point dataset evaluation

The independent point evaluation used temporally aligned MAXAR imagery to assess woody vegetation mapping, finding that models trained on multiple image sources performed best and most consistently.

  • The independent dataset contained 4,491 randomly sampled points across 12 scenes, manually classified as woody or non-woody using MAXAR 0.6m imagery.It was gathered to address temporal misalignment in the Fisher et al. dataset relative to SPOT 6/7 imagery.
  • 97.9% weighted overall accuracy was achieved by the model trained on all data sources, while models trained on all single-date sources achieved 97.6% against all data sources.These models most often attained the best weighted overall accuracy across validation datasets.
  • 76.2% average error reduction was obtained by training on multiple image sources compared with single-date imagery models averaging 90.7% weighted overall accuracy.The result was reported across the validation datasets.
  • Single-date models performed best on matching image dates but degraded on alternative dates, with the 2019 model dropping from 97.9% to 81.0%.This represented a 9× increase in average error across single-date images.
  • 13× lower performance standard deviation was observed for models trained on multiple sources than for single-source models.Average standard deviation decreased from 0.050 to 0.004 across the datasets.

B. Fisher et al. dataset evaluation

Evaluation on the Fisher et al. dataset and other validation settings showed that multi-source training and fused imagery improved accuracy and robustness, while introducing temporal and computational considerations.

  • Fisher et al. dataset evaluation: 92.8% overall accuracy was achieved by models trained on all data sources and all single-date image sources on the Fisher et al. dataset.Single-date models averaged 89.9% overall accuracy, corresponding to a 28.1% average error reduction from multi-source training.
  • Fisher et al. dataset evaluation: All trained models outperformed the Fisher et al. model despite an 11-year difference between the source imagery and evaluation points.
  • Image source and fusion evaluation: 97.3% overall accuracy was obtained by fusing predictions from all image sources, reducing average overall error by 53.6% versus individual sources.This was a further 25% error reduction relative to the SPOT composite.
  • Image source and fusion evaluation: 96.4% overall accuracy was achieved by the SPOT composite, versus 94.2% average overall accuracy across individual single-date sources.The composite reduced total error by 38.2%.
  • Reliance on single-date imagery: 13× lower performance standard deviation was observed for multi-source models across datasets than for single-date models.Sharing handcrafted composite labels with individual dates improved robustness across annual imagery.
  • Image composition: 7.07% to 16.53% of cloud-impacted pixels were recovered by multi-date image composition compared with single-date datasets.Relying on single-source imagery would leave an average 12.91% of pixels missing because of potential cloud cover.
  • Limitations: 5× more computational time was required to generate initial predictions for five image sources using prediction fusion than for one source.Image composition instead required additional preprocessing and artifact storage, but no additional inference processing.
  • Limitations: Label transfer across image sources can introduce label noise when clouds or land clearing make single-date imagery differ from the composite.

B. Data efficiency

The framework improves deep-learning data efficiency through label transfer across imagery sources and active-learning patch selection, while retaining a scope boundary against non-deep-learning methods. It produces a woody vegetation map for New South Wales.

  • Data augmentation: Label transfer from SPOT composite imagery to alternative sources multiplied training-image diversity and made the model the most efficient among prior deep-learning methods.All models trained on multiple input imagery sources routinely outperformed models trained on a single input source.
  • Active learning: Active learning proposed uncertain image patches for labeling, targeting difficult or unfamiliar conditions and stopping when few patches from unseen scenes remained.Its quantitative impact was not evaluated against stratified random sampling because independent baseline datasets were costly and stopping criteria were difficult to determine objectively.
  • Remaining data demands: The proposed work required orders of magnitude more human-annotated examples than non-deep-learning methods despite being the most data-efficient compared with prior deep-learning approaches.The paper attributes this requirement to the larger number of trainable parameters in deep-learning CNNs.
  • Mapping output: The framework generated a woody vegetation extent map covering New South Wales, Australia.The map was reprojected at 240m pixel resolution using average resampling to obtain woody percentage values.

D. Historical woody vegetation map comparison

The proposed 1.5m woody vegetation map was compared with the historical 5m Fisher et al. map across New South Wales and IBRA bioregions. Disagreements concentrated in woodland regions and also arose around urban areas, fire-related temporal mismatches, and damp coastal vegetation near the 2m threshold.

  • Map agreement: 85.4% classification similarity was found between the proposed 1.5m map and the historical 5m Fisher et al. map after mode-resampling alignment.Agreement counts pixels sharing the same woody classification between the two maps.
  • IBRA-region disagreement: 31.4%, 27.5%, and 21.1% of disagreement occurred in the Cobar Peneplain, Murray Darling Depression, and Mulga Lands, respectively.Across these three regions, 96% of total disagreement was non-woody disagreement.
  • IBRA-region disagreement: Woodland spacing produced non-woody disagreement because the proposed framework segmented individual tree crowns while the historical model aggregated them into homogeneous regions.The proposed output’s finer detail was attributed to 1.5m imagery and stronger spatial information pooling.
  • Other disagreement sources: Other disagreements arose around urban areas, after fires because of temporal misalignment, and in damp coastal regions containing vegetation near the 2m woody-definition threshold.The comparison figures identify these as distinct regions of interest for visual inspection.
Loading 2608.26471v1…