Source-linked AI summary
Deep Transfer Learning for Land Use and Land Cover Classification: A Comparative Study
Raoof Naushad, Tarunpreet Kaur, Ebrahim Ghaderpour
TL;DR
LULC classification is difficult because scene semantics are challenging to capture and CNNs are tedious to train on small datasets. The study fine-tunes pretrained VGG16 and Wide ResNet-50 models on RGB EuroSAT data using training-enhancement techniques, achieving a 99.17% benchmark with Wide ResNet-50. The results also compare augmentation and optimization strategies for performance and computational efficiency.
Problem
LULC classification remains challenging because complex spatial distributions make low-level features insufficient, while CNNs are tedious to train on smaller datasets.
Method
The study fine-tunes pretrained VGG16 and Wide ResNet-50 models on RGB EuroSAT data, using early stopping, gradient clipping, learning-rate optimization, and data augmentation.
Results
99.17% accuracy was achieved with Wide ResNet-50 on RGB EuroSAT, exceeding the reported 98.74% DDRL-AM result and VGG16's 98.55%.
Takeaways & Limitations
The proposed transfer-learning methodology provides a 99.17% RGB EuroSAT benchmark and Wide ResNet-50 performed better than VGG16.
Takeaways & Limitations
The conclusion identifies the need for datasets with higher inter- and intra-class variability and greater data quality and quantity.
Abstract
from arXiv · showhide
Efficiently implementing remote sensing image classification with high spatial resolution imagery can provide a significant value in Land Use and Land Cover (LULC) classification. The new advances in remote sensing and deep learning technologies have facilitated the extraction of spatiotemporal information for LULC classification. Moreover, the diverse disciplines of science, including remote sensing, have utilised tremendous improvements in image classification by Convolutional Neural Networks (CNNs) with transfer learning. In this study, instead of training CNNs from scratch, the transfer learning is applied to fine-tune pre-trained networks Visual Geometry Group (VGG16) and Wide Residual Networks (WRNs), by replacing the final layer with additional layers, for LULC classification using the red-green-blue version of the EuroSAT dataset. Moreover, the performance and computational time are compared and optimised with techniques, such as early stopping, gradient clipping, adaptive learning rates, and data augmentation. The proposed approaches have addressed the limited-data problem, and very good accuracies are achieved. The results show that the proposed method based on the WRNs performs better than the previous best-stated results in terms of the computational efficiency and accuracy from 98.57% to 99.17%.
1. Introduction
LULC classification remains challenging because low-level pixel or object features do not adequately capture scene semantics, while CNNs are costly to train on small datasets. The study therefore evaluates transfer learning with VGG16 and Wide ResNet-50 on RGB EuroSAT imagery.
- LULC classification remains challenging as complex spatial distributions and increasing abstraction from pixels to scenes make low-level features insufficient for capturing scene semantics.
- CNNs extract high-level feature representations, but training them on smaller datasets is tedious.
- Transfer learning reuses general features from a primary CNN task as a starting point or feature extractor for a secondary task.
- The study applies VGG16 and Wide ResNet-50 transfer learning to RGB EuroSAT LULC classification and evaluates data augmentation and model-enhancement techniques.
2. Related Works
Prior EuroSAT studies used diverse deep-learning and transfer-learning architectures, with performance varying across models and spectral-band combinations. The paper positions its comparison against established RGB results, including a 98.74% DDRL-AM benchmark.
- Related work spans PCA-based methods, self-organizing networks, hybrid deep-learning frameworks, stacked autoencoders, and deep belief networks for remote-sensing classification.
- Attention-based and fine-tuning approaches have also been evaluated across LULC datasets and aerial-image classification tasks.
- 98.74% accuracy was reported by DDRL-AM using RGB bands on the EuroSAT dataset, providing a comparison benchmark.
- The section frames Table 1 as a comparative analysis of LULC classification studies using EuroSAT.
3. Materials and Methods
The methods fine-tune pretrained VGG16 and Wide ResNet-50 models on RGB EuroSAT data, replacing final classification layers while freezing initial layers. Training is enhanced with augmentation, gradient clipping, early stopping, and adaptive learning-rate reduction.
- 3.2. Transfer Learning Methods: VGG16 and Wide ResNet-50 are fine-tuned from pretrained models by replacing final layers, freezing initial layers, and training modified layers on EuroSAT.
- 3.3. Data Augmentation: Data augmentation uses Gaussian blurring, horizontal and vertical flips, rotation, and resizing to increase visual variability.
- 3.3.2. Gradient Clipping: Gradient clipping reduces gradient norms exceeding a maximum norm of 0.1 to help prevent vanishing and exploding gradient issues.
- 3.3.3. Early Stopping: Early stopping terminates training when validation performance stops improving, retaining the best model weights and reducing overfitting.
- 3.3.4. Learning rate optimisation: ReduceLROnPlateau lowers the learning rate by a factor of 0.1 after two epochs without improvement, with Adam capped at 0.0001.
4. Results
The study evaluates fine-tuned VGG16 and Wide ResNet-50 models for RGB EuroSAT classification, comparing data augmentation, training behavior, computational efficiency, and class-wise performance. Wide ResNet-50 achieves the strongest reported validation accuracy and improves performance across most classes, while confusion remains among visually similar land-cover categories.
- VGG16 results: 98.55% validation accuracy was achieved with VGG16 and data augmentation, compared with 98.14% without augmentation.Early stopping stopped training at the 21st epoch with augmentation and the 18th epoch without it.
- Wide ResNet-50 results: Wide ResNet-50 training showed steady initial learning, later smaller improvements, and three learning-rate changes during training.The reduced learning rates supported continued optimization between the 15th and 23rd epochs.
- Class-wise performance: The confusion matrices compare class-wise validation performance for VGG16 and Wide ResNet-50, highlighting errors among Annual Crop, Permanent Crop, Pasture, and Herbaceous Vegetation.High performance was reported for VGG16 on Forest, Highway, Residential, and Sea/Lake, each above 99% accuracy.
- Class-wise performance: Wide ResNet-50 classified Forest and Sea/Lake most accurately at 99.86%, while Permanent Crop had the lowest accuracy at 97.39%.Accuracy and misclassification rates improved for all classes except River; similar topological features continued to confuse several vegetation and crop classes.
5. Discussion
The study compares fine-tuned VGG16 and Wide ResNet-50 for RGB EuroSAT LULC classification, finding Wide ResNet-50 more accurate and computationally feasible. Data augmentation and training enhancements further affected accuracy, convergence, and computational efficiency.
- 99.17% accuracy was achieved by Wide ResNet-50 on RGB EuroSAT classification.
- Data augmentation produced more training epochs and better accuracy by providing additional feature examples for generalisation.
- Forest and Sea/Lake were least misclassified, while vegetation classes and Highway versus River were confused because of similar topological features.
- Wide ResNet-50 exceeded VGG16 by at least 0.6 percentage points, reaching 99.17% versus 98.55%.
6. Conclusions
The paper concludes that transfer learning with VGG16 and Wide ResNet-50 provides a benchmark for RGB EuroSAT LULC classification. Data augmentation and model-enhancement techniques improved the reported results, while broader data variability remains a relevant boundary.
- 99.17% accuracy established a benchmark for RGB EuroSAT LULC classification and improved the state of the art.
- Data augmentation outperformed training on the original dataset, while regularisation, early stopping, gradient clipping, and learning-rate optimisation improved training efficiency.
- Wide ResNet-50 produced better results than VGG16 under the same data-augmentation approaches.
- Higher-quality data, greater dataset quantity, and increased inter- and intra-class variability are identified as ways to address the remaining classification problem.
Abbreviations
The abbreviations section defines terms used for colour bands, neural-network architectures, learning methods, datasets, and remote-sensing indices.
- CNNs means Convolutional Neural Networks, DL means Deep Learning, and TL means Transfer Learning.
- LULC means Land Use and Land Cover, while RGB means Red-Green-Blue.
- VGG means Visual Geometry Group, ResNet means Residual Networks, and WRNs means Wide Residual Networks.
- SWIR means Short-Wave Infrared, and WDA means Without Data Augmentation.