Unsupervised Deep Feature Extraction for Remote Sensing Image Classification

Adriana Romero; Carlo Gatta; Gustau Camps-Valls

Source-linked AI summary

Unsupervised Deep Feature Extraction for Remote Sensing Image Classification

Adriana Romero, Carlo Gatta, Gustau Camps-Valls

arXiv:1511.08131v1Nov 25, 2015cs.CV

TL;DR

Remote sensing classification is difficult because imagery is high-dimensional and labeled data are limited. The paper combines greedy layer-wise unsupervised pre-training with EPLS to learn sparse hierarchical features efficiently. Across several classification settings, deep architectures and the proposed representations outperform established alternatives, while depth can eventually be limited by spatial-resolution loss or overfitting.

Problem
High-dimensional remote sensing imagery and relatively few labeled samples make direct supervised convolutional-network training challenging.
Method
The method combines greedy layer-wise unsupervised pre-training with EPLS to learn hierarchical sparse representations for classification.
Results
The approach outperforms PCA, kPCA, and prior aerial-classification methods across aerial-scene, VHR, multispectral, and hyperspectral settings, with deep networks generally surpassing single-layer variants.
Takeaways & Limitations
Single-layer networks require receptive fields covering neighboring pixels, whereas deeper architectures capture increasingly abstract features until pooling or parameter growth limits performance.
Takeaways & Limitations
Increasing depth can reduce performance through excessive spatial-resolution loss and an increased number of parameters that promotes overfitting.

Abstract

from arXiv · show

This paper introduces the use of single layer and deep convolutional networks for remote sensing data analysis. Direct application to multi- and hyper-spectral imagery of supervised (shallow or deep) convolutional networks is very challenging given the high input data dimensionality and the relatively small amount of available labeled data. Therefore, we propose the use of greedy layer-wise unsupervised pre-training coupled with a highly efficient algorithm for unsupervised learning of sparse features. The algorithm is rooted on sparse representations and enforces both population and lifetime sparsity of the extracted features, simultaneously. We successfully illustrate the expressive power of the extracted representations in several scenarios: classification of aerial scenes, as well as land-use classification in very high resolution (VHR), or land-cover classification from multi- and hyper-spectral images. The proposed algorithm clearly outperforms standard Principal Component Analysis (PCA) and its kernel counterpart (kPCA), as well as current state-of-the-art algorithms of aerial classification, while being extremely computationally efficient at learning representations of data. Results show that single layer convolutional networks can extract powerful discriminative features only when the receptive field accounts for neighboring pixels, and are preferred when the classification requires high resolution and detailed results. However, deep architectures significantly outperform single layers variants, capturing increasing levels of abstraction and complexity throughout the feature hierarchy.

I. INTRODUCTION

Remote sensing classification must handle high-dimensional, redundant, noisy, and potentially nonlinear imagery, often with limited labeled data. Existing feature-extraction and learning approaches leave efficiency, sparsity, depth, and spatial-information challenges unresolved.

Remote sensing combines diverse multispectral and hyperspectral measurements, with classification maps among its most relevant products.
High pixel dimensionality, noise, uncertainty, spatial-spectral redundancy, collinearity, and nonlinear relations complicate automatic classification.
PCA and nonlinear feature-extraction methods are widely used for dimensionality reduction, but remote sensing data remain challenging to represent efficiently.
Existing autoencoders require multiple free parameters, rely on heuristic regularization, are mainly shallow, and often use spectral information with limited spatial integration.
Sparse unsupervised deep convolutional feature learning had not yet been systematically addressed for remote sensing data analysis.

B. Contributions

The paper addresses remote sensing’s nonlinear, high-dimensional, and computationally demanding setting with efficiently trained deep convolutional networks that learn sparse representations without relying on abundant labels.

The paper targets nonlinear spatial-spectral analysis, local redundant structure, sparse representations, computational efficiency, and limited labeled data.
Greedy layer-wise unsupervised pre-training is combined with EPLS to learn hierarchical sparse features without meta-parameter tuning.
The learned hierarchy uses lower layers for low-level features and higher layers for more abstract and complex representations supporting image and pixel classification.
The work presents a systematic study of sparse unsupervised deep convolutional networks for remote sensing analysis in a fully unsupervised setting.
Experiments compare classification accuracy and expressive power across aerial scenes and land-cover tasks involving VHR, multispectral, and hyperspectral images.

A. Deep Convolutional Neural Networks

A CNN stacks convolution, point-wise nonlinearity, and pooling to build hierarchical representations from local image features. Its main design choices include depth, outputs, receptive fields, and spatial pooling.

Deep neural networks create hierarchical representations by sequentially passing each layer’s output into the next layer.
CNNs share weights locally, applying the same filters across input locations.
Each CNN layer convolves learnable filters, applies a point-wise nonlinearity, and pools nearby feature statistics.
Pooling reduces spatial size and computational cost while providing local translational invariance.
The first CNN layer receives multispectral or hyperspectral imagery, while later layers receive feature maps with spatial and channel dimensions.
The architecture’s key meta-parameters are layer count, outputs per layer, receptive-field size, and pooling size and type.

B. The greedy layer-wise unsupervised pre-training strategy

Greedy layer-wise unsupervised pre-training learns each convolutional layer from patches and unlabeled data, then passes the resulting feature maps forward to train the next layer.

Layer-wise unsupervised criteria enable pre-training with large amounts of unlabeled data before optional supervised fine-tuning.
Patch-based training extracts random patches from images or feature maps to learn each layer’s parameters.
The process repeats sequentially until all L layers are trained and their parameters are produced.
For each layer, the procedure learns parameters from vectorized patches using an unsupervised criterion, here EPLS.
The trained layer transforms input feature maps into output maps that become the training data for the next layer.

C. Unsupervised learning criteria with sparsity

EPLS learns unsupervised sparse features by constructing targets that enforce both population and lifetime sparsity, then optimizing layer parameters against those targets.

Population sparsity limits simultaneous activations, while lifetime sparsity regulates activation frequency across the dataset.
EPLS learns discriminative features by iteratively building a sparse target from layer outputs and optimizing filters for that target.
The learned parameters are optimized using out-of-the-box minibatch stochastic gradient descent with adaptive learning rates.
For each minibatch, EPLS normalizes layer outputs, selects maximally activated outputs after inhibition, and remaps the target to active or inactive values.

D. Feature Extraction

Feature extraction applies learned filters and a selected encoding strategy to produce sparse representations, with polarity splitting optionally separating positive and negative responses.

Feature extraction maps each layer’s input feature map to a representation using a chosen post-filter nonlinearity.
Natural encoding uses the same nonlinearity employed during network training, while alternative strategies such as soft-threshold encoding can be combined with training.
Polarity splitting concatenates positive and negative code components, doubling the number of outputs when applied.
Deep architectures are trained with greedy layer-wise unsupervised pre-training using EPLS, with training stopping when relative error decrease becomes very small.

III. EXPERIMENTAL RESULTS

The experiments assess the proposed algorithm across remote-sensing classification and segmentation settings with varied input dimensionality, class counts, and labeled-data availability.

The evaluation covers land-cover and land-use classification in aerial, very-high-resolution, multispectral, and hyperspectral imagery.
The experiments examine extracted-feature capabilities across problems differing in input dimensionality, number of classes, and available labeled data.

A. Aerial Scene Classification

On UCMerced aerial scenes, EPLS-based convolutional features outperform the cited comparison methods, while deeper networks improve performance until accuracy declines at three layers.

Single-layer networks: 74.34±3.0% accuracy exceeds 62.7%±1.72% for OMP-1 with soft-threshold encoding on normalized raw pixels.
Single-layer networks: EPLS outperforms the method in across all tested output counts without training or encoding meta-parameters to tune.
Single-layer networks: The proposed method shows high sensitivity and specificity for most classes, with errors concentrated among visually similar residential subclasses.
Deep architectures: Two-layer CNNs improve the previous single-layer results for all tested output counts.
Deep architectures: 84.53% accuracy from the two-layer CNN with N_l^h = 1000 per layer outperforms all previous results.
Deep architectures: Increasing depth to three layers reduces accuracy, potentially reflecting greedy pre-training, receptive-field mismatch, or increased overfitting risk.
Sparsity analysis: With three layers, OMP-1 drops from 72.90% to 43.14%, whereas EPLS improves by 5.52% when adding a second layer.

B. Very high resolution (VHR) image classification

In two VHR images, the experiments assess CNN feature extraction for habitat classification across training-set sizes and network depths. Deeper CNNs improve performance, with gains saturating at six layers and beyond 5% training samples.

Data and setup: The study uses two QuickBird VHR images, Nayak-Middle Fork and Kol, for independent spectral-spatial habitat classification experiments.The images contain four spectral bands and represent salmon habitat classes in North Pacific Rim river flood plains.
Data and setup: CNN architectures are evaluated across varying numbers of layers and training examples to assess their ability to capture spectral-spatial structure.The experimental setup performs feature extraction and classification independently for each image.
Results: More training samples improve results, while nonlinear CNN and kPCA features outperform linear PCA in the VHR experiments.The reported comparisons use overall accuracy and kappa statistics across training fractions from 0.5% to 10%.
Results: Deep CNNs produce average kappa gains of about +10% on Nayak and +20% on Kol compared with shallower variants.Performance saturates at six layers and when more than 5% of samples are used for training.

C. Multispectral image classification

The multispectral experiments evaluate unsupervised CNN features for cloud screening across seven MERIS images. CNN features generally outperform spectral-only and kPCA representations, although gains vary by scene conditions.

Data and setup: Cloud screening is evaluated on seven 16-channel MERIS images representing diverse scenes.The images were acquired over Abracos, Ascension Island, Azores, Barcelona, Capo Verde, Longyearbyen, and Mongu.
Evaluation: CNN classification results are evaluated using overall accuracy and Cohen’s kappa as the number of labeled training pixels increases.The two measures show consistent convergence, with no large differences between OA and κ.
Results: CNN features yield average κ gains of 5–30% across scenes and clearly outperform the other feature-extraction approaches numerically.In some scenes, CNN improves over kPCA but not over raw spectral features because spatial feature extraction is less effective under snowy mountains, sunglint, or easy cloud configurations.
Model choice: Linear activation functions produce less-performing extracted features than the logistic activation used in the experiments.

D. Hyperspectral image classification

On Indian Pines hyperspectral classification, deeper CNNs with pooling and spatially informed receptive fields produced the strongest results, while excessive pooling eventually reduced spatial resolution.

Data and setup: The Indian Pines experiment used the full 614 × 2166 image with 220 spectral bands, reduced to 200 after removing noisy and water-absorption bands.Twenty of 58 land-cover classes and background pixels were excluded from classification.
Feature extraction: Single-layer networks remained near κ = 0.3 for pixel-wise classification, but improved when neighboring pixels were included through 5×5 receptive fields.The best single-layer configuration used N 1 h = 200 features and 5×5 receptive fields.
Depth and pooling: The 6-layer network reached κ = 0.84, outperforming the same-setting SVM result of κ = 0.75.Max-pooling after each intermediate layer was extremely beneficial, and deeper networks improved accuracy substantially.
Training-label robustness: A 6-layer network trained with 5% of samples per class outperformed the best single-layer network trained with 30%.This comparison highlights the deep features’ robustness to fewer labeled examples.
Pooling limitation: Adding a seventh layer decreased accuracy because repeated max-pooling reduced spatial resolution below the scale of many labeled regions.The 7-layer resolution could not capture regions smaller than 4096 pixels, while labeled regions averaged around 500 pixels.
Learned representations: Feature visualizations showed increasingly complicated and abstract representations through deeper layers, except for the seventh layer’s spatially over-regularized features.The comparison used the three features with highest mutual information with labels across layers 1–7.

IV. CONCLUSIONS

The paper presents computationally efficient unsupervised convolutional feature learning with population and lifetime sparsity. Across remote-sensing tasks, spatial information, pooling, and depth improved classification until resolution loss or overfitting limited further gains.

Method: The method combines greedy layer-wise pre-training with EPLS to learn hierarchical sparse representations using population and lifetime sparsity.The extracted features can be supplied to a simple classifier, and feature learning has computational cost equal to OMP-1 while outperforming it.
Empirical findings: The networks effectively encode spatio-spectral information across remote-sensing images with different spatial and spectral resolutions.Experiments covered very high-resolution, multispectral, and hyperspectral applications.
Architectural conclusions: Classification improved with spatial information, many output features, pooling, and added layers until repeated pooling reduced spatial resolution or model size induced overfitting.The authors identify these effects as the main boundaries on further architectural improvement.
Future work: Future work includes testing generalization across multi-temporal and multi-angular imagery and reducing reliance on labeled data for task-specific training.The paper also proposes exploring dropout and sparsity adaptation while preserving computational efficiency.