PaperlayerView exact revision on arXiv

Source-linked AI summary

Satellite Image Time Series Analysis for Big Earth Observation Data

Rolf Simoes, Gilberto Camara, Gilberto Queiroz, Felipe Souza, Pedro R. Andrade, Lorena Santos, Alexandre Carvalho, Karine Ferreira

arXiv:2204.11301v1cs.LG

TL;DR

Big-EO software must balance flexibility, interoperability, efficiency, and ease of use. sits is an open-source R package for time-series land classification that combines machine learning with spatial smoothing and training-data quality assessment. A Cerrado case study classified about 200 million hectares with good performance using an advanced deep-learning model.

  • Problem

    Big-EO software must balance flexibility, interoperability, efficiency, and ease of use.

    Source: §5 · p. 18

  • Method

    sits combines satellite image time-series classification with training-data quality checks, machine learning, spatial smoothing, and accuracy assessment in an open-source R workflow.

    Source: §1 · p. 2, §3.8 · p. 11

  • Results

    About 200 million ha of the Cerrado were classified with good performance using an advanced deep-learning model on time series.

    Source: §4.6 · p. 17

  • Takeaways & Limitations

    The Cerrado case study demonstrates an integrated classification workflow executable with six R commands and no specific parallel-processing knowledge.

    Source: §4.6 · p. 17

    sits uses fixed time intervals and pixel-based time series, and requires ARD image collections to be converted into data cubes.

    Source: §6 · p. 19

Abstract

from arXiv · show

The development of analytical software for big Earth observation data faces several challenges. Designers need to balance between conflicting factors. Solutions that are efficient for specific hardware architectures can not be used in other environments. Packages that work on generic hardware and open standards will not have the same performance as dedicated solutions. Software that assumes that its users are computer programmers are flexible but may be difficult to learn for a wide audience. This paper describes sits, an open-source R package for satellite image time series analysis using machine learning. To allow experts to use satellite imagery to the fullest extent, sits adopts a time-first, space-later approach. It supports the complete cycle of data analysis for land classification. Its API provides a simple but powerful set of functions. The software works in different cloud computing environments. Satellite image time series are input to machine learning classifiers, and the results are post-processed using spatial smoothing. Since machine learning methods need accurate training data, sits includes methods for quality assessment of training samples. The software also provides methods for validation and accuracy measurement. The package thus comprises a production environment for big EO data analysis. We show that this approach produces high accuracy for land use and land cover maps through a case study in the Cerrado biome, one of the world's fast moving agricultural frontiers for the year 2018.

1. Introduction

The paper introduces sits, an open-source R package for machine-learning analysis of satellite image time series, using a time-first, space-later approach. It aims to support a complete land-classification workflow for big Earth observation data.

1. Introduction

  • Satellite-based land-cover products provide consistent, periodic, global coverage to help quantify land change and support evidence-based sustainability policies.Land-cover conversion is linked to greenhouse-gas emissions and biodiversity loss.Source: §1 · p. 1
  • Big-EO analysis software must address how researchers and developers can analyse the growing volume of publicly available satellite data.Open-data policies have made unprecedented amounts of satellite data publicly available.Source: §1 · pp. 1–2
  • Satellite image time series provide calibrated, comparable observations across dates, and frequent revisits can capture land-use and land-cover change.The series may come from one sensor or combine sensors.Source: §1 · p. 2
  • The time-first, space-later approach classifies each location’s time series before spatial post-processing, unlike comparing classified images from different dates.Authors argue this approach is better suited to tracking change continuously [8].Source: §1 · p. 2
  • sits provides an integrated workflow for big-EO land classification, from cloud-hosted data cubes and training-data quality checks to machine learning and spatial smoothing.It also includes validation and accuracy assessment methods.Source: §1 · p. 2
  • The package adds cloud image-catalogue support, deep-learning methods, training-data quality control, and data-cube-specific parallel processing beyond existing R packages.The authors cite recently published deep-learning methods not available as R packages [8] and new training-data quality methods.Source: §1 · p. 2

2. Earth Observation Data Cubes

The section defines EO data cubes by their space-time structure and data completeness, then explains why satellite image collections may need processing to meet those requirements. It also describes sits’ decision to support both regular cubes and irregular image collections.

2. Earth Observation Data Cubes

  • An EO data cube is a georeferenced, regular spatial grid over continuous time intervals, with shared attributes and no missing values across its extent.The definitions specify a field function over space-time positions, a two-dimensional spatial component, and consecutive non-overlapping temporal intervals.Source: §2 · p. 3
  • ARD collections provide comparable, reprojected images but may contain cloudy or missing pixels, so they do not guarantee valid values at every location.Cloudy or missing values conflict with the data-cube field-function requirement.Source: §2 · p. 3
  • The Sentinel-2 tile 20LKP image composites illustrate that images from different dates can contain substantial cloud cover.Software supporting the cube abstraction must replace cloudy or missing pixels with valid values.Source: Figure 2 · p. 4, §2 · p. 4
  • Neighbouring Sentinel-2 tiles can have different timelines—144 instances for 20LLP versus 71 for 20LKP—so common timelines are needed to use one model across large areas without reprocessing.The example covers 13 July 2018 to 28 July 2019.Source: §2 · p. 4
  • sits supports both regular data cubes and irregular image collections rather than requiring all imagery to be reprocessed into regular collections.The paper names Brazil Data Cube and gdalcubes as examples of tools that produce consistent data cubes.Source: §2 · p. 4

3. Software Design and Analysis Methods

sits is designed as a production environment for land classification, with an R API that covers the workflow from data-cube selection through classification and smoothing. Its design goals include cloud-data access, training-data quality control, multiple learning methods, and efficient large-area processing.

3.1. Requirements and Design Choices

  • R was chosen for its established data-analysis use, spatial and time-series abstractions, machine-learning support, and CRAN’s quality standards and cross-platform compatibility.The package relies on many third-party packages as an integrated environment for the full land-classification cycle.Source: §3.1 · pp. 4–5
  • The design goals cover a concise classification API, cloud access, training-data quality control, multiple machine-learning and deep-learning methods, large-area parallel processing, and spatial post-processing.The package targets specialists who understand remote sensing and can write R scripts.Source: §3.1 · pp. 4–5

3.2. Workflow and API

  • The workflow selects an image collection and space-time extent, obtains and quality-checks labelled training samples, trains a model, classifies the cube, and spatially smooths the result.Changing cube dimensions requires retraining; a trained model can be reused when another cube has the same bands and number of temporal intervals.Source: §3.2 · pp. 5–6
  • The API assigns workflow tasks to functions, including cube creation, data extraction, training, classification, smoothing, and production of a final labelled image.The sits_train() function encapsulates differences among methods, from random forests to convolutional neural networks; convenient defaults support novice users while allowing fine-tuning.Source: §3.2 · p. 6

3.3. Handling Data Cubes

sits accepts imagery from cloud services and local files, while using STAC endpoints to access current catalogue information. Its cube definition stores metadata and retrieves data as needed.

3.3. Handling Data Cubes

  • sits supports ARD collections and data cubes from several cloud services, the Brazil Data Cube, gdalcubes, and local files.User-transparent functions enforce the data-cube properties for ARD inputs.Source: §3.3 · p. 7
  • sits uses STAC endpoints to access up-to-date information from frequently updated cloud datasets.STAC is a geospatial information specification adopted by providers of large image collections.Source: §3.3 · p. 7
  • Users define a cube by selecting an ARD collection and specifying its spatial and temporal extent; the cube contains metadata, with data accessed on demand.The example selects an AWS Sentinel-2 collection, tile, and date range.Source: §3.3 · p. 7

3.4. Handling Time Series

sits stores satellite time series with spatial, temporal, label, and source-cube metadata, and retrieves them from data cubes using expert-provided samples. It also assesses sample quality with SOMs, where mixed-label neurons can flag likely noisy samples.

3.4. Handling Time Series

  • A sits time series table stores time series data alongside metadata for spatial and temporal information, sample label, and source data cube.Locations use longitude and latitude coordinates for the WGS84 ellipsoid.Source: §3.4 · p. 7
  • Experts supply samples with locations, labels, and dates; sits retrieves the corresponding time series from the data cube for classification training.Samples can be provided as CSV files or shapefiles.Source: §3.4 · p. 8

3.5. Sample Quality Control

  • SOM-based quality assessment maps high-dimensional training samples to two dimensions while preserving topological relations among similar patterns.Neighbouring neurons provide information about intraclass and interclass variability.Source: §3.5 · p. 8
  • Heterogeneous SOM neurons with multiple labels at significant probability are likely to contain noisy samples, while homogeneous neurons usually contain good-quality samples.The function also provides quality information per sample and a 2D map for visualizing class noise.Source: §3.5 · p. 8
  • In a Cerrado SOM of 50,160 MODIS MOD13Q1 samples across 12 labels from 2000–2017, some Rocky Savanna neurons mixed with Dense Woodland; removing outliers improves classification results.The map also shows Pasture neurons forming a cluster.Source: §3.5 · p. 8, Figure 5 · p. 9

3.6. Training Machine Learning Models

sits lets users choose training labels and supports both feature-space classifiers and deep learning models that use temporal order. Defaults serve novice users, while model parameters remain adjustable.

3.6. Training Machine Learning Models

  • Users choose training samples and labels for their region and classification scheme; the package extracts the associated time series for model training.The paper states that training-sample quality can affect classification-map accuracy more than the choice of machine-learning method.Source: §3.6 · p. 9
  • sits supports feature-space methods including random forests, support vector machines, extreme gradient boosting, and multi-layer perceptrons, as well as temporal deep-learning models.Supported 1D-CNN algorithms include TempCNN, ResNet, and InceptionTime; convolution cascades capture features at different time scales.Source: §3.6 · pp. 9–10
  • Default model settings support novice users, while experienced users can adjust all model parameters.The defaults were chosen based on the authors’ testing, and the documentation describes model tuning parameters.Source: §3.6 · p. 10

3.7. Data Cube Classification

sits uses customized parallel processing to handle big EO data with independent workers and minimal communication. It divides cubes into spatial chunks, combines the resulting subimages, and supports recovery from worker failures.

3.7. Data Cube Classification

  • sits implements customized parallel processing because R’s memory behavior is unpredictable when parallel tasks use dependencies such as Keras, Python, and TensorFlow.The package targets virtual machines close to the data repository and runs in any environment supporting R.Source: §3.7 · p. 10
  • Independent workers read and store large payloads separately, minimizing direct interaction with the main process and avoiding worker-to-worker communication.The authors report that no existing R parallel-processing package met their needs for this use.Source: §3.7 · p. 10
  • The method divides a cube into spatial chunks based on cube size, core count, and available memory, then assigns chunks to worker cores.Each chunk includes all temporal intervals; workers produce output subimages that are joined into the result.Source: §3.7 · p. 10
  • Failure of one worker does not halt processing; sits can recover a failed block and resume classification from the last processed chunk.The paper names memory exhaustion, power interruption, and network breakdown as possible disruptions.Source: §3.7 · p. 10

3.8. Post-Processing

sits smooths classification probabilities using Bayesian estimation informed by neighbouring pixels. The update balances a pixel’s original value with the neighbourhood mean, with the balance shaped by covariance and local uncertainty.

3.8. Post-Processing

  • Bayesian smoothing uses pixel-neighbourhood information to reduce uncertainty in labels, addressing errors more likely near class transitions or in mixed pixels.It applies a mathematical transformation to classifier probabilities before smoothing.Source: §3.8 · p. 11
  • The method assumes the transformed class-probability vector follows a multivariate normal distribution with an unknown mean and an a priori covariance matrix controlling smoothness.The covariance matrix represents prior belief about class variability and possible confusion between classes.Source: §3.8 · p. 11
  • The posterior estimate combines the pixel value with the neighbourhood mean, weighted by covariance matrices; the neighbourhood parameters are estimated from nearby pixels.The resulting estimator is computed per pixel to produce a smoothed map.Source: §3.8 · p. 11
  • Neighbourhood influence is strongest when the original classification odds ratio is low; diverse nearby values shift weight toward the local mean, while similar values favor the original pixel.The user supplies the smoothing factor a priori, and the procedure is intended to incorporate spatial effects while handling outliers.Source: §3.8 · p. 12

3.9. Validation and Accuracy Assessment

sits supports model validation and area-weighted accuracy assessment, while warning that validation estimates are best-case and unsuitable as accuracy measures. Its extensible design lets users add classifiers and integrate sits with other tools and languages.

3.9. Validation and Accuracy Assessment

  • Validation estimates are a “best-case” scenario subject to training-sample selection bias and should compare models on the same training data, not serve as accuracy measures.sits uses k-fold validation, testing each partition with a model trained on the others and averaging results to estimate prediction error.Source: §3.9 · p. 12
  • Area-weighted accuracy assessment adjusts mapped areas for classification error and provides error-adjusted area estimates with confidence intervals.The sits_accuracy_area() function follows the best practices proposed by Olofsson et al..Source: §3.9 · p. 12

3.10. Extensibility

  • The sits_train() closure encapsulates classifier-specific details, allowing users to add models without changing other package components.Training and classification are independent and extensible through the overloaded R predict function.Source: §3.10 · pp. 12–13
  • sits time series can be exported in tabular formats, images can be exported locally, TensorFlow models can be saved, and Python users can access the API through rpy2.These options broaden the potential community of contributors and users.Source: §3.10 · p. 13

4. Results and Discussion

The study applies sits to map Cerrado land use and cover from Landsat-8 time series for September 2017–August 2018. The area-weighted overall accuracy was 0.86, while the authors identify low-sample classes as a challenge.

4. Results and Discussion

  • The study mapped the Cerrado, a fast-moving agricultural frontier, using a one-year Landsat-8 land use and cover classification.The biome is Brazil’s second largest, covering 1.9 million km2.Source: §4 · p. 13

4.1. Input Data

  • The input covered 51 Brazil Data Cube Landsat-8 tiles from September 2017 to August 2018, with 24 images per tile and about 8 TB of data.Inputs included seven spectral bands, NDVI, EVI, and cloud-cover information.Source: §4.1 · p. 13

4.2. Training Samples

  • Systematic 5 × 5 km sampling yielded 48,850 training points after retaining only samples whose labels agreed across three source maps.The initial 85,026 samples came from a grid across the Cerrado; class counts varied considerably, including 16,251 Cerrado and 38 Natural Non Vegetated samples.Source: §4.2 · p. 13

4.3. Training and Classification

  • A TempCNN trained on the samples used all nine spectral and vegetation-index attributes, followed by Bayesian smoothing and most-likely-class labelling.Classification averaged 30 minutes per tile and took about 24 hours overall on a 24-core server with 128 GB memory.Source: §4.3 · p. 14

4.4. Code

  • The Cerrado workflow is expressed in six R commands, without requiring specific parallel-processing knowledge.The commands cover data-cube access, time-series retrieval, model training, classification, smoothing, and map generation.Source: §4.4 · pp. 15–16

4.5. Classification Accuracy

  • The independent accuracy assessment used specialist-interpreted reference samples and an area-weighted estimator for overall accuracy and class-area totals.The 5,286 evaluation samples were independent of training data; “Cerradao”, “Cerrado”, and “Open Cerrado” were merged into one assessment label.Source: §4.5 · p. 16
  • 0.86 overall accuracy was achieved for the Cerrado classification using an area-weighted estimator based on reference samples.Table 1 presents area-weighted classification accuracy.Source: §4.5 · p. 16, Table 1 · p. 17

4.6. Discussion of Results

  • Producer’s accuracy was 51% for Perennial Crop and 63% for Natural Non Vegetated; both classes had few training samples, motivating more balanced and representative sampling.The respective available training counts were 68 and 38.Source: §4.6 · p. 17

5. Comparative Analysis

The comparison contrasts segmentation-based time-series methods with sits’ classification of predefined-size series using all observations. It describes trade-offs among platforms and notes that comparative benchmarks across method families are not available to the authors.

5. Comparative Analysis

  • BFAST, CCDC, and TWDTW segment multiyear series before classification, whereas sits classifies predefined-size time series using all their values.BFAST detects trends and breaks, CCDC extracts segment metrics, and TWDTW matches segments to predefined patterns.Source: §5 · p. 17
  • The authors report no known benchmarks comparing segmentation-based methods with feature-space or multiresolution methods.They anticipate that more evidence will help users choose a method suited to their needs.Source: §5 · p. 18
  • sits provides ready-to-use deep-learning models for image time series, while GEE requires users to develop and run such models outside Earth Engine using TensorFlow.The authors say this currently makes sits easier to use than GEE for deep-learning image-time-series tasks.Source: §5 · p. 18
  • ODC offers flexibility through Python tools but relies on users’ programming skills and does not handle image-collection tiles with different timelines.sits instead emphasizes a simple API and large-image-collection processing without parallel-programming knowledge.Source: §5 · p. 18
  • The authors describe a trade-off among flexibility, interoperability, efficiency, and ease of use, and say sits currently supports land classification rather than other EO applications.They frame the platforms as having different benefits and constraints for potential users.Source: §5 · p. 18

6. Conclusions

Big EO software faces trade-offs across hardware-specific efficiency, portability, and accessibility. sits adopts a time-series approach to land classification while acknowledging limits on continuous-change monitoring and data preparation.

6. Conclusions

  • Big EO software involves trade-offs among hardware-specific efficiency, portability, and ease of learning, and no single approach is likely to be the complete best solution.The authors identify interoperability, reusability across cloud services, and simple APIs as common ground for software design.Source: §6 · p. 19
  • sits prioritizes time-series analysis and supplies all available data to machine-learning methods to find patterns in multidimensional spaces rather than using time-series metrics.Its data structures and API follow from this land-classification design choice.Source: §6 · p. 19
  • Fixed intervals reduce continuous-change monitoring, sits assigns each time series to a pixel, and using it requires converting ARD collections to data cubes.Object-based time-series approaches can perform better than pixel-based approaches.Source: §6 · p. 19
  • Planned sits developments include data-cube operations, active and semi-supervised learning, spatial objects, new classifiers, and rule-based multiyear comparison.The authors also plan to provide community guidance and documentation.Source: §6 · p. 19

Related papers

Loading 2204.11301v1…