DeepSat - A Learning framework for Satellite Imagery

Saikat Basu; Sangram Ganguly; Supratik Mukhopadhyay; Robert DiBiano; Manohar Karki; Ramakrishna Nemani

Source-linked AI summary

DeepSat - A Learning framework for Satellite Imagery

Saikat Basu, Sangram Ganguly, Supratik Mukhopadhyay, Robert DiBiano, Manohar Karki, Ramakrishna Nemani

arXiv:1509.03602v1Sep 11, 2015cs.CV

TL;DR

Satellite-image classification is hindered by high variability and a shortage of labeled, high-resolution, multi-class datasets. The paper introduces SAT-4 and SAT-6 and combines feature extraction and normalization with a Deep Belief Network, achieving 97.95% on SAT-4 and 93.9% on SAT-6 while outperforming comparison algorithms.

Problem
Satellite-image classification involves high variability, limited labeled data, and a shortage of high-resolution datasets with multiple class labels.
Method
The framework extracts handcrafted features, normalizes them, and feeds the resulting vectors to a Deep Belief Network for classification.
Results
97.95% accuracy on SAT-4 and 93.9% on SAT-6 outperform three state-of-the-art object-recognition algorithms by approximately 11% and 15%, respectively.
Takeaways & Limitations
Feature extraction improves discriminative power for satellite datasets, while SAT-4 and SAT-6 provide benchmarks for learning representations and classifying satellite imagery.
Takeaways & Limitations
The framework treats satellite and airborne imagery interchangeably because its features and learning algorithms are intended to handle both datasets.

Abstract

from arXiv · show

Satellite image classification is a challenging problem that lies at the crossroads of remote sensing, computer vision, and machine learning. Due to the high variability inherent in satellite data, most of the current object classification approaches are not suitable for handling satellite datasets. The progress of satellite image analytics has also been inhibited by the lack of a single labeled high-resolution dataset with multiple class labels. The contributions of this paper are twofold - (1) first, we present two new satellite datasets called SAT-4 and SAT-6, and (2) then, we propose a classification framework that extracts features from an input image, normalizes them and feeds the normalized feature vectors to a Deep Belief Network for classification. On the SAT-4 dataset, our best network produces a classification accuracy of 97.95% and outperforms three state-of-the-art object recognition algorithms, namely - Deep Belief Networks, Convolutional Neural Networks and Stacked Denoising Autoencoders by ~11%. On SAT-6, it produces a classification accuracy of 93.9% and outperforms the other algorithms by ~15%. Comparative studies with a Random Forest classifier show the advantage of an unsupervised learning approach over traditional supervised learning techniques. A statistical analysis based on Distribution Separability Criterion and Intrinsic Dimensionality Estimation substantiates the effectiveness of our approach in learning better representations for satellite imagery.

1 Introduction

The paper addresses satellite-image classification amid high variability and limited labeled data by introducing labeled datasets and a feature-normalization framework using a Deep Belief Network.

Problem: Satellite image classification is difficult because acquisition conditions and land-cover types create substantial variability, while existing approaches do not generalize well at scale.Land-cover classes such as trees, grasslands, barren lands, and water bodies exhibit high intra-class variability; the paper limits contextual windows to 28m×28m.
Problem: Limited labeled satellite datasets have constrained research, with NLCD 2006 offering global coverage at 30m resolution but insufficient detail for distinguishing land-cover types.Prior high-resolution datasets covered smaller areas or provided labels only for roads or binary tree-cover delineation.
Contributions: The paper introduces SAT-4 and SAT-6, two labeled airborne-image datasets covering approximately 800 square kilometers for evaluating learning models.The source imagery spans the continental United States and supports applications including land-cover mapping, carbon-content measurement, and rooftop-area estimation.
Approach: The proposed framework combines handcrafted feature extraction and feature normalization with an unsupervised Deep Belief Network trained for classification.The approach is motivated by the need to learn representations from large amounts of unlabeled data while retaining informative handcrafted features.
Results: 97.95% accuracy on SAT-4 and 93.9% on SAT-6 exceed three state-of-the-art object-recognition algorithms by approximately 11% and 15%, respectively.The comparison includes Deep Belief Networks, Convolutional Neural Networks, and Stacked Denoising Autoencoders.

2 Dataset2

The datasets are constructed from large-scale NAIP imagery by manually labeling uniform land-cover patches and extracting nonoverlapping 28×28 windows for classification.

Source imagery: NAIP provides 330,000 scenes across the continental United States, with approximately 65 terabytes of high-resolution imagery.The imagery comprises GeoTIFF DOQQ tiles averaging approximately 6000×7000 pixels and 200 megabytes each.
Data construction: The study uses 1500 image tiles covering varied California landscapes, including rural, urban, forested, mountainous, agricultural, and aquatic areas.An image-labeling tool was used to manually identify uniform patches belonging to individual land-cover classes.
Data construction: 28×28 nonoverlapping sliding-window blocks were extracted from labeled uniform patches and stored with their corresponding class labels.The window size was selected to provide an appropriate local context for the classification task.
SAT-4: SAT-4 contains 500,000 patches across four broad land-cover classes, split into 400,000 training and 100,000 testing examples from disjoint image tiles.Its classes are barren land, trees, grassland, and a residual class containing other land-cover types.
SAT-6: SAT-6 contains 405,000 28×28 patches across six classes, split into 324,000 training and 81,000 testing examples from disjoint NAIP tiles.The classes are barren land, trees, grassland, roads, buildings, and water bodies.

3 Investigation of various Deep Learning Models

The paper investigates DBN, CNN, and SDAE architectures for satellite-image classification, varying their depth and width to assess performance on SAT-4 and SAT-6. Results show that network size and architecture materially affect classification accuracy.

3.1 Deep Belief Network: DBNs use unsupervised, layer-wise RBM pre-training followed by supervised feedforward backpropagation.The training objective maximizes expected log probability on the training dataset, and trained RBMs initialize the supervised neural network.
3.1 Deep Belief Network: DBN experiments vary neurons per layer and total layers to test whether deeper networks learn discriminative higher-order texture features.The architectures are evaluated on both SAT-4 and SAT-6.
3.2 Convolutional Neural Network: CNNs combine convolutional and subsampling layers with pooling, followed by a fully connected layer and dataset-specific softmax output.Experiments vary feature maps, layer counts, kernel sizes, and pooling functions.
3.3 Stacked Denoising Autoencoder: SDAEs train multiple sparse autoencoders greedily and independently, freeze each trained layer, and classify deepest-layer activations after supervised fine-tuning.The encoding and decoding stages form the two-phase training procedure.
3.3.1 SDAE Results on SAT-4 & SAT-6: SDAE networks with 5 layers and 100 neurons per layer perform best on both datasets, whereas larger configurations show a significant accuracy drop.The classifier accuracy initially improves and then declines as neurons and layers increase.

4 DeepSat - A Detailed Architectural Overview

DeepSat transforms satellite imagery into normalized, selected feature vectors before DBN classification. Its feature extraction and posterior-estimation pipeline is designed to improve discriminative representation.

Feature extraction: 150 features are extracted from image channels and color co-occurrence matrices using statistical, transform, texture, and spectral descriptors.Descriptors include mean, variance, correlation, entropy, homogeneity, contrast, and related measures across hue, saturation, intensity, and NIR channels.
Feature selection: 22 features are selected from the 150 extracted features using a ranking algorithm based on the Distribution Separability Criterion.The criterion ranks features according to class-distribution separability.
Normalization: Training and test feature vectors are separately normalized to the range [0, 1].For each feature type, Fmin and Fmax are computed over all images in the dataset.
Classification: Normalized feature descriptors are fed into a DBN trained with contrastive divergence, whose learned weights initialize a feedforward backpropagation network.The resulting neural network estimates posterior probabilities for the class labels given the feature vector.
Classification: The network output represents the posterior probability of class C1, while P(C2|x) = 1 − y(x) in binary classification.The formulation extends to multiple class labels.
Design rationale: Feature extraction improves the discriminative power of the DBN-based classifier by providing dimensionality reduction.The paper characterizes this phase as a useful dimensionality reduction technique.

5 Results and Comparative Studies

DeepSat is evaluated across network architectures on SAT-4 and SAT-6, with the experiments comparing its classification performance against Random Forest and traditional deep-learning baselines. The reported results favor DeepSat and deep architectures over Random Forest.

Random Forest comparison: 69% accuracy on SAT-4 and 54% on SAT-6 are achieved by the Random Forest classifier.The highest Random Forest accuracy uses 100 trees, with no significant improvement from adding more trees.
Comparative results: Deep architectures produce better classification accuracy than Random Forest, which relies solely on supervised learning.This comparison is used to examine the advantage of unsupervised pre-training in DBNs.
DeepSat architectures: Table 4 reports DeepSat classification accuracy across various network architectures on SAT-4 and SAT-6.The table is the paper’s architecture-level evaluation of the proposed framework.

6 Why Traditional Deep Architectures are not enough for SAT-4 & SAT-6?

Traditional deep architectures lose useful spatial and higher-order information when processing highly variable satellite imagery as raw pixel vectors. DeepSat addresses this representation problem through feature extraction, which improves distribution separability and aligns with higher classifier accuracy.

Why raw-pixel architectures struggle: Satellite imagery has high intra- and inter-class variability, limited labeled data, and important texture and spatial-context cues.Traditional DBN and SDAE inputs lose relative pixel spatial information, weakening the discriminative power of the raw pixel representation.
Distribution separability: DeepSat features have greater distances between class means and lower standard deviations than original image distributions.These distributional changes provide better class separability than raw image values.
Distribution Separability Criterion: Ds ranks features by maximizing mean distance between class distributions while minimizing their standard deviations.The criterion combines the mean distance between distribution means with the mean standard deviation of class-conditional distributions.
Activation analysis: DeepSat neuron activations exhibit distribution separability several orders of magnitude higher than DBN neurons using raw pixel values.The paper links this difference to DeepSat’s performance improvement over the raw-pixel DBN framework.
Connection to accuracy: Distribution separability is positively correlated with final classifier accuracy across the evaluated DBN and DeepSat architectures.The paper presents Ds as a measure of final classifier accuracy based on this comparison.

7 What is the difference between MNIST, CIFAR-10 and SAT-6 in terms of dimensionality?

SAT-6 has substantially higher intrinsic dimensionality than MNIST, making raw-image classification harder for deep networks. The extracted SAT-6 features occupy a lower-dimensional manifold that supports smaller architectures.

Dataset dimensionality: SAT-6 is argued to lie on a higher-dimensional manifold than MNIST and CIFAR-10.The paper uses intrinsic dimension to explain why raw-feature classification differs across these datasets.
Estimation method: DANCo estimates intrinsic dimension using normalized nearest-neighbor distances and angles between neighboring points.The estimates are averaged over 10 rounds of 1000 random samples for the compared datasets and extracted SAT-6 features.
Dataset dimensionality: SAT-6 has intrinsic dimensionality orders of magnitude higher than MNIST.The paper uses this contrast to characterize the difficulty of classifying raw SAT-6 data with deep neural networks.
Effect of feature extraction: Extracted SAT-6 features have much lower intrinsic dimensionality than raw vectors and can be classified with relatively smaller networks.The features are described as lying on a lower-dimensional manifold.
High-dimensional geometry: In high-dimensional spaces, data points become increasingly concentrated relative to the total search space, reducing classification and clustering discriminative power.This geometric effect is presented as background for the dimensionality analysis.

8 Related Work

Prior work addressed land-cover and object-recognition tasks at resolutions or sites that limit evaluation for high-resolution satellite object classification.

75% and 78% accuracies were reported for MODIS and Landsat land-cover maps, respectively.
These studies make performance difficult to assess for 1-m imagery and higher-resolution object-recognition tasks.
A deep convolutional hierarchical framework was evaluated on the 20m-resolution AVIRIS Indiana’s Indian Pines test site.

9 Conclusions and Future Directions

The framework achieves high accuracy on SAT-4 and SAT-6, while future work targets additional representations and pooling methods for satellite datasets.

Conclusions: 97.95% and 93.9% accuracy were achieved on SAT-4 and SAT-6, outperforming state-of-the-art methods by approximately 11% and 15%, respectively.
Conclusions: Feature extraction improves the framework’s discriminative power for satellite datasets with inherently high variability.
Conclusions: Distribution Separability Criterion analysis supports the effectiveness of the feature-extraction approach.
Future Directions: Future work will investigate pooling techniques, sparse representations, and hierarchical representations such as Convolutional DBN.