PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep learning on edge: extracting field boundaries from satellite images with a convolutional neural network

François Waldner, Foivos I. Diakogiannis

arXiv:1910.12023v2cs.CV

TL;DR

Digital agriculture needs precise field boundaries, but existing methods can require costly preprocessing and locally arbitrary tuning. This paper uses multitask ResUNet-a segmentation to predict field extent, boundaries, distances, and reconstructed imagery, achieving accurate extraction and generalisation across imagery, space, and time; seasonal averaging improves robustness to temporal variation.

  • Problem

    Manual boundary records are time-consuming, while existing methods can require expensive preprocessing, arbitrary local tuning, and ancillary data.

  • Method

    The paper trains ResUNet-a as a multitask semantic-segmentation network predicting field extent, boundary probability, nearest-boundary distance, and reconstructed imagery.

  • Results

    The model achieved high pixel- and field-level accuracy and generalised without retraining across single-date Sentinel-2, Landsat-8, five cropping systems, and acquisition dates.

  • Takeaways & Limitations

    Single-date imagery can replace composites with marginal accuracy loss, while averaging predictions from at least four seasonal images addresses temporal accuracy variation.

  • Takeaways & Limitations

    Single-image prediction can still struggle to distinguish certain classes, and wider evaluation is needed for smallholder systems where very high resolution may be required.

Abstract

from arXiv · show

Applications of digital agricultural services often require either farmers or their advisers to provide digital records of their field boundaries. Automatic extraction of field boundaries from satellite imagery would reduce the reliance on manual input of these records which is time consuming and error-prone, and would underpin the provision of remote products and services. The lack of current field boundary data sets seems to indicate low uptake of existing methods,presumably because of expensive image preprocessing requirements and local, often arbitrary, tuning. In this paper, we address the problem of field boundary extraction from satellite images as a multitask semantic segmentation problem. We used ResUNet-a, a deep convolutional neural network with a fully connected UNet backbone that features dilated convolutions and conditioned inference, to assign three labels to each pixel: 1) the probability of belonging to a field; 2) the probability of being part of a boundary; and 3) the distance to the closest boundary. These labels can then be combined to obtain closed field boundaries. Using a single composite image from Sentinel-2, the model was highly accurate in mapping field extent, field boundaries, and, consequently, individual fields. Replacing the monthly composite with a single-date image close to the compositing period only marginally decreased accuracy. We then showed in a series of experiments that our model generalised well across resolutions, sensors, space and time without recalibration. Building consensus by averaging model predictions from at least four images acquired across the season is the key to coping with the temporal variations of accuracy. By minimising image preprocessing requirements and replacing local arbitrary decisions by data-driven ones, our approach is expected to facilitate the extraction of individual crop fields at scale.

1. Introduction

The paper targets scalable field-boundary extraction for digital agriculture, replacing manual records and locally tuned, preprocessing-heavy methods with a data-driven deep-learning approach. It formulates extraction as multitask segmentation and evaluates generalisation across imagery, locations, and acquisition conditions.

  • Manual field-boundary records are time-consuming and error-prone, creating disincentives for digital agricultural services.
  • Existing extraction methods often require expensive preprocessing, arbitrary local tuning, and ancillary maps that do not generalise across locations.
  • The proposed approach uses ResUNet-a for multitask pixel prediction of field extent, boundary probability, and distance to the nearest boundary.
  • The paper aims to support routine field-boundary extraction at scale by reducing preprocessing workload and replacing context-specific arbitrary decisions.
  • The model achieved high thematic and geometric accuracy using a monthly Sentinel-2 composite and generalised across single-date imagery, Landsat, locations, and acquisition dates without recalibration.

2. Extracting field boundaries with multi-task semantic segmentation

Field-boundary extraction is framed as correlated multitask semantic segmentation, using shared representations and conditioned outputs to support closed individual-field contours.

  • Multitask learning shares representations across related outputs, potentially improving generalisation over single-task boundary prediction.
  • The task predicts multiple class labels rather than only whether each pixel is a boundary.
  • ResUNet-a performs four correlated tasks: field extent mapping, boundary identification, nearest-boundary distance estimation, and input-image reconstruction.
  • The extent, boundary, and distance predictions provide information for post-processing open contours into closed contours and individual fields.

2.1. Boundary detection with a deep convolutional neural network

The model combines an encoder-decoder backbone with residual, dilated, pooling, and conditioned multitask components. Training uses transformed imagery and multitask loss, while inference produces several masks for field-boundary extraction.

  • Model architecture: ResUNet-a combines a UNet encoder-decoder, residual blocks, atrous convolutions, pyramid scene parsing pooling, and conditioned multitasking.These components capture context, preserve localisation, enlarge receptive fields, aggregate multiscale information, and connect output tasks.
  • Conditioned inference: The reconstruction branch uses Hue-Saturation-Value imagery to provide information about colour variation and object extent.
  • Model architecture: The encoder compresses image information, while the symmetric decoder restores resolution and localises classes using residual units with multiple dilation rates.
  • Conditioned inference: ResUNet-a produces extent, boundary, distance, and reconstructed-image outputs through conditioned inference.The distance mask is generated first, then informs boundary prediction; distance and boundary outputs inform extent prediction, while reconstruction is parallel.
  • Training: Horizontal and vertical flips plus random brightness changes augment training data, while rotations and zooms are avoided because they may break field symmetry.
  • Training: The loss averages the Tanimoto distance and its complement, and the multitask loss averages losses across tasks.
  • Inference: Once trained, the model processes any 256×256 image through a single forward pass, although contextual classification may vary near image edges.

2.2. Extraction of individual fields

The method converts ResUNet-a’s multiple segmentation outputs into closed boundaries and individual fields through two data-driven post-processing approaches: cutoff and watershed.

  • ResUNet-a predicts extent, boundary, and distance masks that support extraction of closed boundaries and individual fields.Both post-processing methods use multiple semantic segmentation outputs and can be automatically optimised using reference data.
  • The cutoff method thresholds and combines the extent and boundary masks to delineate individual fields with closed boundaries.The symmetric difference of the two binary layers produces the extracted fields.
  • The watershed method applies seeded watershed segmentation to all three masks, using the distance mask to derive field-centre seeds.Catchment basins grow from these seeds until they meet, forming boundaries between adjacent fields.
  • Threshold optimisation: Threshold optimisation first maximises extent agreement with a reference cropland map using MCC, then jointly tunes boundary and distance thresholds.The procedure requires reference data to identify optimal threshold values.
  • Threshold optimisation: The final candidate balances oversegmentation and undersegmentation by selecting the Pareto-optimal solution closest to the 1:1 line.The rates range from 0 to 1, with values closer to 1 indicating less segmentation error.

3. Data and study sites

The study uses a large South African main site and five secondary agricultural sites, with Sentinel-2 imagery and reference data assembled for training, validation, testing, and generalisation experiments.

  • Main study site: The main study area covers 120,000 km2 in South Africa’s maize-producing region, where fields average 17 ha and range from 1 to 830 ha.The site includes flat undulating plains to mountainous terrain and maize, wheat, sunflower, and soybean production.
  • Satellite data: Sentinel-2 was selected for its 5-day revisit frequency, 10-m resolution, and free availability for large-scale applications.The revisit frequency supports cloud-free acquisition and consistent multi-temporal composites.
  • Satellite data: The main site used a March 2017 monthly cloud-free Sentinel-2 composite, with 93% of composited pixels cloud-free.Images were converted to surface reflectance before compositing.
  • Satellite data: Preprocessing consisted of selecting blue, green, red, and near-infrared bands and standardising each band to zero mean and unit variance.Standardisation supports neural-network optimisation because the model is sensitive to input scale.
  • Reference data: The main-site reference boundaries were manually digitised from 2.5-m panmerged SPOT imagery and rasterised at 10 m for validation.Reference layers represented field extent, 10-m-buffered boundaries, and normalised distance to the closest boundary.
  • Data partitioning: Twelve Sentinel-2 tiles were split into 10 training tiles, one validation tile, and one test tile, then partitioned into 256×256-pixel input images.Five secondary sites used single Sentinel-2 tiles and ancillary cropland and manually digitised field data.

4. Methods

The experiments evaluate ResUNet-a training, satellite-image generalisation, and field-extraction accuracy using pixel- and object-based metrics, with statistical comparison of post-processing methods.

  • Model training: ResUNet-a models were trained with Adam, and weight-decay models used 10^-4, 10^-5, or 10^-6 over 100 epochs.Adam uses adaptive learning rates derived from first- and second-moment estimates of gradients.
  • Post-processing comparison: The best model’s outputs were post-processed into individual fields, and cutoff and watershed methods were compared with paired Wilcoxon tests.Cutoff thresholds were searched from 0.01 to 0.99 in increments of 0.01.
  • Accuracy assessment: Accuracy assessment combined pixel-based metrics, including MCC, overall accuracy, and class-wise F-score, with object-based field metrics.Object-based metrics capture differences in field shape, size, and location.
  • Accuracy assessment: For experiments outside South Africa, hit rate was preferred because the available cropland maps were not accurate enough to serve as validation data.Hit rate is the ratio of successfully detected validation fields to total validation fields.
  • Accuracy assessment: Object-based evaluation measured oversegmentation, undersegmentation, eccentricity, and location shift between extracted and reference fields.Location shift was expressed in pixels and area-weighted when one reference field matched multiple extracted fields.

4.2. Comparison with conventional edge detection

The study compares the model’s predicted boundary mask with edges detected by a Scharr filter using sampled boundary and interior pixels.

  • The Scharr baseline detects edge magnitudes from horizontal and vertical edges in each spectral band, which are then averaged.The filter was selected for better rotation invariance than Sobel or Prewitt operators.
  • The comparison samples 1,000 boundary and interior pixels and evaluates their correspondence with the labels “boundary” and “not boundary”.

4.3. Generalisation experiments

The experiments tested whether the model generalised across image dates, spatial resolutions, sensors, and locations without retraining. They also evaluated averaging predictions across seasonal images to reduce temporal variability.

  • The model was evaluated on single-date imagery, coarser-resolution imagery, another sensor, and images acquired across sites and the growing season.
  • Single-date inference was tested as a less preprocessing-intensive alternative to monthly composites, without assuming a significant accuracy loss.
  • Consensus was formed by averaging segmentation masks from multiple observation dates and was progressively evaluated across the season.

5. Results

The selected ResUNet-a model achieved high pixel- and field-level accuracy and produced clearer boundaries than conventional edge detection. Accuracy was generally retained across single dates, resolutions, sensors, and sites, although acquisition date and resolution affected performance.

  • 5.1. Model selection: MCC=0.82 was achieved by the deeper ResUNet-a D7 model, which was selected for subsequent analyses.The best D6 model reached MCC=0.81 with the highest weight decay, while training curves showed overfitting after 80 epochs.
  • 5.2. Assessment of the baseline model: 92% overall accuracy and 82% MCC were achieved for the baseline extent map, with F-scores of 89% for cropland and 93% for non-cropland.Optimising the extent threshold changed results by less than 1% relative to a 50% threshold.
  • 5.2. Assessment of the baseline model: 55,720 fields were extracted, and 99% of reference fields were identified with shape and position accuracy metrics above 0.85 and a 7-pixel location shift.The simpler cutoff approach achieved similar results to the watershed approach.
  • 5.3. Comparison with edge detection: Scharr filtering produced significantly weaker boundaries and noisier interiors than ResUNet-a, with P <0.001 for both comparisons.The network learned sensitivity to relevant edge types, producing clearer retrieved edges.
  • 5.4. Generalisation to single-date imagery: Single-date imagery marginally changed performance: over- and under-segmentation rates dropped by 0.02-0.05, while the offset remained unchanged.Only the undersegmentation difference was statistically significant (P < 0.001).
  • 5.5. Generalisation to resolutions and sensors: Resampling to 30 m reduced the hit rate by only 0.06, while individual-field extraction was more sensitive to resolution than sensor changes.For 30-m versus 10-m Sentinel-2 data, the hit rate changed significantly from 0.88 to 0.79.
  • 5.6. Generalisation across time and space: Acquisition date changed the hit rate from 0.75 to 0.99 and location shift from 7 to 17 pixels, while consensus reduced variability and its improvement rate decreased after four images.
  • 5.6. Generalisation across time and space: Secondary sites showed larger temporal variation, including a Canadian hit-rate range of 0.21 to 0.97, and required local threshold optimisation for good generalisation.

6. Discussion

The ResUNet-a multitask approach achieved high field-boundary accuracy and generalised across sensors, resolutions, locations, and acquisition dates without recalibration. Time-averaged consensus improved robustness to temporal variability while retaining relatively light preprocessing requirements.

  • The model generalised across space, time, resolutions, and sensors without retraining, including single-date Sentinel-2 and Landsat-8 imagery.Applying the model to Landsat-8 reduced geometric accuracy to approximately 70% and hit rate to approximately 80%.
  • More than 85% accuracy was achieved at both pixel and object levels for field-boundary extraction.
  • Approximately 90% accuracy was achieved for mapping the main-site cropping extent from a single monthly Sentinel-2 composite.The result was comparable to state-of-the-art cropland-classification methods using multitemporal features.
  • Accuracy varied substantially with acquisition date, with differences as large as 75%, particularly in previously unseen areas.The training set used images from a single month, and broader seasonal or geographic training could reduce this sensitivity.
  • Averaging predictions from four images spread across the growing season delivered most of consensus’s accuracy benefits and halved location errors.Consensus was generally at least as accurate as single-date predictions and required lighter processing than alternatives using temporal features.
  • The approach supports automated, data-driven field-boundary extraction at scale while minimising image preprocessing and context-specific tuning.The authors recommend organising reference data in continuous blocks and using monthly composites across the season.

7. Conclusion

The paper presents a multitask deep-learning approach that extracts individual field boundaries from satellite imagery and generalises across multiple imaging conditions. It proposes evidence-based procedures for applying the method at scale, including composite-based training, single-date prediction, seasonal averaging, and data-driven combination.

  • The model uses multitasking and conditioned inference to predict field extent, boundary probability, and distance to the nearest boundary before extracting individual fields.
  • The neural network achieved state-of-the-art field boundary detection and generalised across space, time, resolutions, and sensors.
  • Training on monthly cloud-free composites is recommended to maximise use of training data.
  • Single-date images can replace prediction composites with marginal loss of accuracy, while averaging at least four seasonal predictions addresses temporal accuracy variation.
  • The paper recommends data-driven procedures for combining model outputs when extracting field boundaries at scale.
Loading 1910.12023v2…