PaperlayerView exact revision on arXiv

Source-linked AI summary

Hydra: an Ensemble of Convolutional Neural Networks for Geospatial Land Classification

Rodrigo Minetto, Mauricio Pamplona Segundo, Sudeep Sarkar

arXiv:1802.03518v2cs.CV

TL;DR

Satellite land-use classification uses large datasets, but prior approaches often trained a single CNN, while ensembles can be costly to train. Hydra fine-tunes a coarsely optimized CNN multiple times to create diverse ensemble heads, reducing training time while maintaining performance. The authors report state-of-the-art performance on NWPU-RESISC45 and third place in the FMOW challenge.

  • Problem

    Prior satellite-image classification methods trained a single CNN, while ensemble training was considered costly for large datasets.

    Source: §1 · p. 2

  • Method

    Hydra coarsely optimizes a CNN body, then fine-tunes it repeatedly using varied training strategies to create diverse ensemble heads.

    Source: §1 · p. 2, §3 · p. 3

  • Results

    Hydra achieved state-of-the-art performance on NWPU-RESISC45 and third place in the FMOW challenge.

    Source: §4.1.6 · p. 9, §5 · p. 10

  • Takeaways & Limitations

    The authors conclude that Hydra is most indicated when training time is limited.

    Source: §5 · p. 10

    The FMOW setup required models to be generated from raw data within 7 days and test predictions within 24 hours.

    Source: §4.1.3 · p. 6

Abstract

from arXiv · show

We describe in this paper Hydra, an ensemble of convolutional neural networks (CNN) for geospatial land classification. The idea behind Hydra is to create an initial CNN that is coarsely optimized but provides a good starting pointing for further optimization, which will serve as the Hydra's body. Then, the obtained weights are fine-tuned multiple times with different augmentation techniques, crop styles, and classes weights to form an ensemble of CNNs that represent the Hydra's heads. By doing so, we prompt convergence to different endpoints, which is a desirable aspect for ensembles. With this framework, we were able to reduce the training time while maintaining the classification performance of the ensemble. We created ensembles for our experiments using two state-of-the-art CNN architectures, ResNet and DenseNet. We have demonstrated the application of our Hydra framework in two datasets, FMOW and NWPU-RESISC45, achieving results comparable to the state-of-the-art for the former and the best reported performance so far for the latter. Code and CNN models are available at https://github.com/maups/hydra-fmow

1 INTRODUCTION

Land-use classification from satellite imagery matters across humanitarian and military applications, but images vary in target scale, function, clutter, viewpoint, occlusion, and time. Hydra addresses the cost of ensembles by fine-tuning a coarsely optimized CNN multiple times to form diverse classifiers.

1 INTRODUCTION

  • Land-use classification assigns a target location in a satellite image to a class of interest or to none, and the resulting information serves humanitarian and military applications.Images may also carry metadata.Source: §1 · p. 1
  • Satellite-image classification is difficult because target sizes vary, similar-looking places can serve different functions, and imagery may contain clutter or be affected by viewpoint, occlusion, and temporal change.Temporal changes include illumination and movable objects in the short term, and major topographic changes over longer periods.Source: §1 · pp. 1–2
  • Prior state-of-the-art methods trained a single CNN on large satellite-image datasets, while the authors describe ensembles as combining classifiers that converge to different endpoints.They note that ensemble training cost may have discouraged its use in prior work.Source: §1 · p. 2
  • Hydra coarsely optimizes a CNN body, then fine-tunes its weights repeatedly with different augmentation, target-region sizes, and image spectra to create diverse ensemble heads.The authors report that this approach cuts training time approximately in half while maintaining ensemble classification performance.Source: §1 · p. 2

2 RELATED WORK

Geospatial land-image research spans multiple tasks, while land-use classification benchmarks have grown from small datasets to larger, more challenging ones. Prior work applied CNNs to region classification and explored CNN-based semantic segmentation, a task this paper does not evaluate.

2 RELATED WORK

  • Remote-sensing image research includes semantic segmentation, target localization, and region classification.The passage characterizes the field as broad and active.Source: §2 · p. 2
  • The Hydra diagram depicts coarse optimization of a body followed by multiple fine-tuned heads seeking different endpoints.The body is shown as a black line and the heads as red lines.Source: Figure 2 · p. 2
  • Earlier land-use datasets were small and often yielded nearly perfect scores, whereas FMOW (63 classes, 470086 images) and NWPU-RESISC45 (45 classes, 31500 images) pose harder classification problems.The passage also lists UC Merced, WHU-RS19, Brazilian Coffe Scenes, and AID as earlier datasets.Source: §2 · p. 2
  • Prior CNN studies evaluated architectures and training setups on smaller datasets, while Christie et al. [1] and Cheng et al. reported results on FMOW and NWPU-RESISC45, respectively.The authors conclude that CNN classifiers were a suitable choice for Hydra.Source: §2 · p. 2
  • Prior semantic-segmentation approaches combined hyperspectral imagery with LiDAR terrain data or used encoder-decoder networks, but this paper conducts no segmentation experiments.The authors suggest Hydra could also benefit that task.Source: §2 · pp. 2–3

3 THE HYDRA FRAMEWORK

Hydra builds CNN ensembles by coarsely optimizing a shared starting body, then refining copies into heads with varied training samples. ResNet and DenseNet heads are fused by majority voting.

3 THE HYDRA FRAMEWORK

  • Hydra first coarsely optimizes CNNs, then uses their weights to initialize repeatedly refined heads, varying training sets during optimization to encourage ensemble diversity.The body and heads use the same training set in the two stages, while heads experience different transformed versions during refinement.Source: §3 · p. 3, Figure 3 · p. 3

3.1 CNN architectures

  • ResNet and DenseNet form complementary heads: ResNet adds bypasses, while DenseNet connects layers to preceding outputs and reuses features with fewer parameters.Both architectures replace the final classifier with three hidden layers, with image metadata joined to the first hidden layer.Source: §3.1 · pp. 3–4

3.2 On-line data augmentation

  • Online augmentation applies random transformations each epoch, giving Hydra heads different training-set versions; unlike offline augmentation, the samples are not fixed before training.The transformations include flips, zooms, and shifts over different crops; photometric distortion is omitted to avoid meaningless samples.Source: §3.2 · p. 4

3.3 Fusion and decision

  • For each region, Hydra sums a head’s score vectors across images, has each head vote for its highest-scoring class, and assigns the majority label; at half or fewer votes, it marks a false detection.Each head produces scores for the classes, and the final decision is made by majority voting.Source: §3.3 · pp. 4–5, Figure 4 · p. 4

4.1 The FMOW dataset

The FMOW section outlines the dataset and competition setup, then reports Hydra’s training choices and results. Hydra finished third in the competition, with accuracy 1% below the best result.

4.1.1 Dataset

  • FMOW contains nearly half a million images across training, evaluation, and testing subsets, with 62 land-use classes plus false detections.The data include temporal views and challenges such as changing shadows, viewpoints, weather, inaccurate annotations, and variable region sizes.Source: §4.1.1 · pp. 5–6

4.1.2 Scoring

  • The challenge ranks submissions by a class-weighted average of F-measures, with class F-measure defined as the harmonic mean of precision and recall.False detections have weight zero, but misclassifications involving them still affect other classes’ false-positive or false-negative counts.Source: §4.1.2 · p. 6

4.1.3 Hardware and time restrictions

  • Finalists had to train models from raw data within seven days, predict all test images within 24 hours, and use specified Linux AWS instances.The permitted instances were m4.10xlarge and g3.16xlarge.Source: §4.1.3 · p. 6

4.1.4 Baseline

  • Hydra’s FMOW classifiers adapt the organizers’ baseline, which combines DenseNet-161 image features and metadata in a two-hidden-layer MLP.The baseline used softmax loss and Adam; its classifiers did not use augmentation.Source: §4.1.4 · pp. 6, 8

4.1.6 Results

  • For the FMOW testing subset, on-line augmentation increased accuracy by more than 2% over corresponding versions without augmentation; the ensemble improved performance by at least 3% over every individual classifier.Source: §4.1.6 · p. 9, Table 3 · p. 9
  • Third place was Hydra’s final FMOW ranking, with accuracy 1% below the best result on the sequestered dataset.The ensemble had nearly the same performance on the testing subset and sequestered dataset.Source: §4.1.6 · p. 9, Table 4 · p. 9
  • Hydra and a standard ensemble showed no significant accuracy difference when their classifiers were fused, while both outperformed their best individual head.The body gave Hydra’s heads a good starting point, and the heads ended training with slightly different accuracies.Source: §4.1.6 · p. 9
  • Against a standard ensemble with the same number of classifiers and similar accuracy, Hydra’s training was approximately two times faster.With N heads, Hydra’s training and inference costs tend to be N/2 and N times those of an individual classifier, respectively.Source: §4.1.6 · p. 9

4.2 NWPU-RESISC45 dataset

On NWPU-RESISC45, Hydra achieved the highest accuracy reported in the section’s comparison, with gains over both individual CNNs and Cheng et al.’s best reported results. The experiment used a modified Hydra configuration and five repeated train/test splits.

4.2 NWPU-RESISC45 dataset

  • NWPU-RESISC45 has 45 classes and 31,500 images but no satellite metadata, and its class labels differ in scope from FMOW’s.Some NWPU-RESISC45 land uses are split into finer classes than FMOW, while some NWPU-RESISC45 classes are objects or vegetation.Source: §4.2 · p. 9
  • The evaluation used five random splits with either 10% or 20% of each class for training, and Cheng et al.’s best prior result used ImageNet-initialized VGG16.The remaining samples were used for testing, with average accuracy and standard deviation reported.Source: §4.2 · pp. 9–10
  • The confusion matrix maps ground-truth labels to Hydra predictions, with large off-diagonal values marking classes that are hard to distinguish.The caption identifies the matrix as the Hydra ensemble after 11 training epochs.Source: Figure 12 · p. 10
  • Hydra outperformed Cheng et al.’s best reported accuracy by more than 4%, and its gain over individual CNNs was slightly greater than 1%.The section reports the ensemble as achieving the highest accuracy on NWPU-RESISC45.Source: §4.2 · p. 10
  • Online augmentation increased ResNet accuracy by about 1.0% and DenseNet accuracy by about 1.6%.The reported comparison is for NWPU-RESISC45.Source: §4.2 · p. 10
  • The experiment used four DenseNets and four ResNets for 16 epochs, split evenly between body training and head fine-tuning.The four heads per architecture used no augmentation, flips, zooming, or shifts, respectively.Source: §4.2 · p. 10

5 CONCLUSIONS

The paper presents Hydra as a CNN-ensemble framework suited to limited training time, reporting state-of-the-art performance on NWPU-RESISC45 and third-best performance in the FMOW challenge. It also frames aerial-scene classification as an important but unresolved problem for decision-making.

5 CONCLUSIONS

  • The authors describe aerial-scene classification as important for government and intelligence or military decision-making, including fast disaster response, but still far from solved.They cite the final FMOW challenge results as evidence that the problem remains unresolved.Source: §5 · p. 10
  • Hydra achieved state-of-the-art performance on NWPU-RESISC45 and third-best performance in the FMOW challenge.The FMOW challenge had more than 50 active participants, including teams and individuals from academia and industry.Source: §5 · p. 10
  • The framework automatically creates CNN ensembles and is most indicated when training time is limited.Its body is a coarsely optimized CNN whose weights are fine-tuned multiple times to form the heads.Source: §5 · p. 10
  • The paper includes a figure showing regions mislabeled by the Hydra ensemble.Source: Figure 13 · p. 11
  • The NWPU-RESISC45 accuracy table reports results for 10%/90% and 20%/80% training/testing splits, averaged with standard deviations across five random splits.The caption notes that classifiers may use online augmentation and/or ImageNet weights.Source: Table 5 · p. 11
Loading 1802.03518v2…