PaperlayerView exact revision on arXiv

Source-linked AI summary

TorchGeo: Deep Learning With Geospatial Data

Adam J. Stewart, Caleb Robinson, Isaac A. Corley, Anthony Ortiz, Juan M. Lavista Ferres, Arindam Banerjee

arXiv:2111.08872v4cs.CVcs.LG

TL;DR

Remote-sensing imagery and geospatial layers vary in bands, resolutions, and coordinate systems, while common machine-learning tooling is not designed to handle these differences. TorchGeo integrates geospatial datasets and multispectral tools into PyTorch workflows; its benchmarks achieve close-to-state-of-the-art results, and ImageNet pretraining improves spatial generalization in land-cover mapping.

  • Problem

    Remote-sensing data vary across sensors and geospatial layers, while common machine-learning tools offer limited support for multispectral imagery and geospatial data handling.

    Source: §1 · p. 1

  • Method

    TorchGeo integrates benchmark and generic geospatial datasets, samplers, multispectral transforms, and pretrained models into PyTorch workflows.

    Source: §1 · pp. 2–3

  • Results

    TorchGeo achieves close-to-state-of-the-art results on all experimental datasets, and ImageNet pretraining significantly improves spatial generalization in land-cover mapping.

    Source: §1 · p. 3

  • Takeaways & Limitations

    TorchGeo’s benchmark catalog and results make it easier to compare new ideas without repeating expensive computations.

    Source: §5 · p. 8

Abstract

from arXiv · show

Remotely sensed geospatial data are critical for applications including precision agriculture, urban planning, disaster monitoring and response, and climate change research, among others. Deep learning methods are particularly promising for modeling many remote sensing tasks given the success of deep neural networks in similar computer vision tasks and the sheer volume of remotely sensed imagery available. However, the variance in data collection methods and handling of geospatial metadata make the application of deep learning methodology to remotely sensed data nontrivial. For example, satellite imagery often includes additional spectral bands beyond red, green, and blue and must be joined to other geospatial data sources that can have differing coordinate systems, bounds, and resolutions. To help realize the potential of deep learning for remote sensing applications, we introduce TorchGeo, a Python library for integrating geospatial data into the PyTorch deep learning ecosystem. TorchGeo provides data loaders for a variety of benchmark datasets, composable datasets for generic geospatial data sources, samplers for geospatial data, and transforms that work with multispectral imagery. TorchGeo is also the first library to provide pre-trained models for multispectral satellite imagery (e.g., models that use all bands from the Sentinel-2 satellites), allowing for advances in transfer learning on downstream remote sensing tasks with limited labeled data. We use TorchGeo to create reproducible benchmark results on existing datasets and benchmark our proposed method for preprocessing geospatial imagery on the fly. TorchGeo is open source and available on GitHub: https://github.com/microsoft/torchgeo.

1 INTRODUCTION

Remote-sensing data vary substantially across sensors and geospatial layers, while common machine-learning tools do not readily handle these differences. TorchGeo integrates geospatial data into PyTorch workflows, and its benchmarks show competitive results, including improved spatial generalization with ImageNet pretraining.

1 INTRODUCTION

  • Remote-sensing imagery varies in spectral bands, spatial resolutions, and collection patterns, limiting whether models developed for one platform generalize to others.The paper notes that a single set of pretrained weights cannot span imaging platforms.Source: §1 · pp. 1–2
  • Layers with different coordinate reference systems produce non-pixel-aligned crops, so TorchGeo reprojects and resamples them during data loading.Figure 2 contrasts unaligned raster layers with aligned layers prepared for model inputs or masks.Source: Figure 1 · p. 2, Figure 2 · p. 3
  • TorchGeo provides benchmark and generic geospatial data loaders, geospatial samplers, multispectral transforms, and pretrained models for remotely sensed imagery.Its loaders can combine raster and vector layers and sample pixel-aligned patches on the fly.Source: §1 · pp. 2–3
  • TorchGeo achieves close-to-state-of-the-art results on all experimental datasets, and ImageNet pretraining significantly improves spatial generalization in land-cover mapping.The authors position these results as reproducible baselines for future work.Source: §1 · p. 3

2 DESIGN

Deep-learning training needs pixel-aligned imagery and masks, but geospatial layers commonly differ in coordinate systems, bounds, and resolution. TorchGeo aligns queried patches on the fly and composes geospatial datasets for sampling and training.

2 DESIGN

  • Training segmentation models requires pixel-aligned imagery and masks, but geospatial datasets are often misaligned because their bounds, coordinate systems, and resolutions differ.Conventional preprocessing requires reprojection, cropping, and resampling, and can require domain knowledge and duplicate data layers.Source: §2 · p. 3
  • TorchGeo datasets return reprojected and resampled data for spatiotemporal queries at a requested destination CRS and spatial resolution.Geospatial samplers can generate queries randomly or in a regular grid pattern.Source: §2 · p. 4
  • Users can compose layers over their union or intersection, including imagery-label intersections for multimodal learning and data fusion.Generic dataset classes support building multimodal datasets without custom code.Source: §2 · p. 4
  • TorchGeo datasets and samplers work with standard PyTorch data loaders to produce fixed-size batches, trading storage space for loading time without requiring GIS-tool knowledge.The paper benchmarks this data-loading approach in Section 4.3.Source: §2 · p. 4

3 IMPLEMENTATION

TorchGeo’s implementation combines datasets, samplers, models, transforms, and training routines within a PyTorch-oriented design. It supports both community benchmarks and generic geospatial layers, with samplers that generate spatially meaningful bounding boxes.

3 IMPLEMENTATION

  • TorchGeo implements variable-input deep-learning architectures with pretrained weights, multispectral augmentations, geospatial transforms, and training routines including BYOL.The implementation follows other PyTorch domain libraries to reduce the new concepts users need to learn.Source: §3 · p. 4

3.1 Datasets

  • Benchmark datasets provide inputs and target labels for specific tasks, while generic datasets represent reusable layers such as Landsat imagery and Cropland Data Layer labels.TorchGeo supports benchmark-dataset access options such as automatic downloading.Source: §3.1 · p. 4

3.2 Samplers

  • RandomGeoSampler, RandomBatchGeoSampler, and GridGeoSampler generate fixed-sized bounding boxes from valid extents, random scenes, or grid patterns, respectively.The samplers support different training or inference patterns and can be used with PyTorch data loaders.Source: §3.2 · pp. 4–5

4 EXPERIMENTS AND RESULTS

The experiments assess TorchGeo datasets and samplers, report reproducible benchmarks, and test how ImageNet pre-training affects generalization across geographic splits.

4.1 Datasets

  • The experiments include classification, regression, and segmentation benchmarks, alongside 114 Landsat scenes and Cropland Data Layer data for loader benchmarking.The benchmark datasets include multispectral So2Sat imagery and land-cover datasets.Source: §4.1 · pp. 5–6

4.2 Data loader benchmarks

  • GridGeoSampler is fastest; at larger batches RandomBatchGeoSampler beats RandomGeoSampler through better cache use, while preprocessing can speed sampling when storage permits.Figure 3 reports average sampling rates and, in panel (a), seed-based minimum and maximum performance.Source: §4.2 · p. 6, Figure 3 · p. 7

4.3 Dataset benchmarks

  • TorchGeo’s simple training setup is competitive across several datasets: ImageNet initialization performs best on RESISC45, EuroSAT, and UC Merced; So2Sat parity requires all Sentinel-2 bands; LandCover.ai comes within 0.75% mIoU of its reported baseline.The paper also reports its first basic benchmark results for Chesapeake Land Cover.Source: §4.3 · p. 7

4.4 Effect of ImageNet pre-training on generalization performance

  • 12 mIoU points in Virginia was the largest Chesapeake gain; ImageNet improved every out-of-state split except Maryland, while So2Sat accuracy gains were +7% and +10%.In Delaware, the ImageNet-pretrained and scratch models had the same in-domain performance.Source: §4.4 · p. 7

5 DISCUSSION

TorchGeo brings geospatial data workflows and benchmarks into PyTorch, while providing a platform for future geospatial machine-learning research.

5 DISCUSSION

  • TorchGeo combines common dataset loaders, composable raster and vector data loaders, geospatial samplers, multispectral transforms, pretrained satellite models, and trainers.It performs alignment preprocessing on the fly, avoiding the need for users to run those steps separately.Source: §5 · p. 7
  • The platform supports research directions including self-supervised learning, geospatial augmentation, spatial inductive biases, data fusion, and shape-based models.The paper notes that existing self-supervised work uses spatiotemporal metadata and can be implemented and scaled in TorchGeo.Source: §5 · pp. 7–8
  • A shared dataset catalog and released benchmark results let researchers compare new methods without repeating expensive computations.The paper presents the benchmarks as a starting point for those comparisons.Source: §5 · p. 8

A DATA LOADER BENCHMARKING

The loader benchmark measures sampling from a large Landsat and Cropland Data Layer collection while varying batch size and random seed.

A DATA LOADER BENCHMARKING

  • The benchmark uses 114 Landsat 8 scenes and one 2019 Cropland Data Layer file, stored as 151 GB of uncompressed Cloud Optimized GeoTIFFs.The files remain in their original UTM and Albers Equal Area coordinate reference systems.Source: A DATA LOADER BENCHMARKING · p. 10
  • The experiments run on a six-core Intel Xeon E5-2690 Azure machine with the data on a local SSD.Six workers are used for parallel data loading.Source: A DATA LOADER BENCHMARKING · p. 10
  • Batch size and random seed vary; each epoch samples 4,096 patches of size 224 with stride 112.The remaining hyperparameters are held fixed.Source: A DATA LOADER BENCHMARKING · p. 10

B PRE-PROCESSING ALIGNMENT WITH GDAL

The appendix illustrates manual GDAL reprojection for aligning a Cropland Data Layer raster to a Landsat scene.

B PRE-PROCESSING ALIGNMENT WITH GDAL

C TORCHGEO CODE EXAMPLE

TorchGeo composes geospatial datasets using unions or intersections and samples image patches for a standard PyTorch data-loading workflow. The example combines Landsat imagery with CDL labels while matching their CRS and resolution.

C TORCHGEO CODE EXAMPLE

  • The composed dataset and sampler can be used with a standard PyTorch DataLoader in a familiar API.The example presents this as a way to work with geospatial data and sample patches from combined Landsat and CDL data.Source: C TORCHGEO CODE EXAMPLE · p. 11
  • The example unions Landsat 7 and 8 using only their common spectral bands, then intersects that dataset with CDL to retain regions covered by both.TorchGeo automatically matches the datasets' CRS and resolution.Source: C TORCHGEO CODE EXAMPLE · p. 11
  • A random geospatial sampler returns 10,000 256 × 256-pixel patches per epoch from the intersected datasets.A custom collation function combines sample dictionaries into mini-batches for the training or evaluation pipeline.Source: C TORCHGEO CODE EXAMPLE · p. 11
  • TorchGeo composes datasets through union and intersection operators, supporting both equivalent data sources and image-label pairs.Unions accommodate datasets with disparate locations; intersections require queries to be present in both datasets.Source: C TORCHGEO CODE EXAMPLE · pp. 11–12

D BENCHMARK DATASET EXPERIMENTS

The experiments use predefined dataset splits and a small hyperparameter search, then evaluate the selected configurations across repeated training runs. The setup also compares RGB-only and multispectral Sentinel-2 inputs on So2Sat and EuroSAT.

D BENCHMARK DATASET EXPERIMENTS

  • A single fixed random seed is used for a validation-based grid search over ResNet18 or ResNet50, random or ImageNet initialization, three learning rates, and cross-entropy or Jaccard loss.The learning rates are 0.01, 0.001, and 0.0001.Source: D BENCHMARK DATASET EXPERIMENTS · p. 12
  • For So2Sat and EuroSAT, experiments compare all Sentinel-2 bands with RGB-only inputs; ImageNet (+ random) randomly initializes non-RGB kernels in the first convolutional layer.Classification and regression use ResNets directly, while semantic segmentation uses them as U-Net encoders.Source: D BENCHMARK DATASET EXPERIMENTS · p. 12
  • The selected hyperparameter configuration is retrained with 10 different seeds, and test performance is reported across those models.Training uses AdamW, reduces the learning rate on validation-loss plateaus, and early-stops based on validation loss.Source: D BENCHMARK DATASET EXPERIMENTS · p. 12
Loading 2111.08872v4…