PaperlayerView exact revision on arXiv

Source-linked AI summary

Balanced Multi-Task Attention for Satellite Image Classification: A Systematic Approach to Achieving 97.23% Accuracy on EuroSAT Without Pre-Training

Aditya Vir

arXiv:2510.15527v1cs.CV

TL;DR

Satellite classification often relies on ImageNet transfer learning, leaving the performance achievable through architectural design alone an open question. This work develops a from-scratch CNN with learnable spatial-spectral attention and reports 97.23% EuroSAT test accuracy, within 1.34% of fine-tuned ResNet-50.

  • Problem

    Because labeled satellite datasets are relatively scarce, ImageNet transfer learning is widely used, leaving the from-scratch performance ceiling on EuroSAT an open question.

    Source: §1 · p. 1, §1.2 · p. 2

  • Method

    The study iteratively develops custom CNNs and combines Coordinate Attention for spatial features with Squeeze-Excitation blocks for spectral features through learnable fusion.

    Source: §6 · p. 6

  • Results

    97.23% test accuracy on EuroSAT without pre-training, within 1.34% of fine-tuned ResNet-50.

    Source: §6 · p. 6

  • Takeaways & Limitations

    Systematic architectural design can achieve competitive performance in domain-specific applications where pre-training is unavailable.

    Source: §6 · p. 6

    Evaluation is limited to RGB-only analysis although full EuroSAT has 13 spectral bands, and persistent vegetation confusion remains (PermanentCrop: 94.46%).

    Source: §5.3 · p. 6

Abstract

from arXiv · show

This work presents a systematic investigation of custom convolutional neural network architectures for satellite land use classification, achieving 97.23% test accuracy on the EuroSAT dataset without reliance on pre-trained models. Through three progressive architectural iterations (baseline: 94.30%, CBAM-enhanced: 95.98%, and balanced multi-task attention: 97.23%) we identify and address specific failure modes in satellite imagery classification. Our principal contribution is a novel balanced multi-task attention mechanism that combines Coordinate Attention for spatial feature extraction with Squeeze-Excitation blocks for spectral feature extraction, unified through a learnable fusion parameter. Experimental results demonstrate that this learnable parameter autonomously converges to alpha approximately 0.57, indicating near-equal importance of spatial and spectral modalities for satellite imagery. We employ progressive DropBlock regularization (5-20% by network depth) and class-balanced loss weighting to address overfitting and confusion pattern imbalance. The final 12-layer architecture achieves Cohen's Kappa of 0.9692 with all classes exceeding 94.46% accuracy, demonstrating confidence calibration with a 24.25% gap between correct and incorrect predictions. Our approach achieves performance within 1.34% of fine-tuned ResNet-50 (98.57%) while requiring no external data, validating the efficacy of systematic architectural design for domain-specific applications. Complete code, trained models, and evaluation scripts are publicly available.

1 Introduction

The introduction frames from-scratch satellite classification as a way to study architectural requirements beyond ImageNet transfer learning. It proposes a staged investigation culminating in learnable spatial-spectral attention.

1.1 Motivation

  • Limited large-scale labeled data has encouraged ImageNet transfer learning, which may not capture satellite imagery’s distinctive spectral and spatial characteristics.Systematic studies of EuroSAT training from scratch remain limited, despite potential use where pre-training is unavailable.Source: §1 · p. 1, §1.1 · p. 1

1.2 Research Questions

  • The research asks what from-scratch performance is achievable, which failure modes arise, and whether satellite imagery benefits from learnable balance between spatial and spectral attention.It also asks how targeted architectural innovations can address identified failure modes.Source: §1.2 · p. 2

1.3 Contributions

  • The three-stage progression raises EuroSAT accuracy from 94.30% to 95.98% to 97.23%, through baseline, attention-enhanced, and balanced multi-task attention models.The progression is presented as iterative problem identification and solution design.Source: §1.3 · p. 2
  • The proposed attention mechanism combines Coordinate Attention for spatial features and Squeeze-Excitation blocks for spectral features, with learnable α converging to approximately 0.57.The reported value indicates near-equal importance of the two modalities.Source: §1.3 · p. 2
  • CBAM resolves targeted River–Highway confusion but introduces vegetation-classification failures, motivating a balanced dual-path design.The paper frames this as a trade-off of single-modality attention.Source: §1.3 · p. 2
  • Evaluation tracks per-class metrics, confusion patterns, and a 24.25% confidence-calibration gap; implementations, trained models, hyperparameters, and scripts are released.The reported evaluation quantifies individual component contributions.Source: §1.3 · p. 2

2 Related Work

Prior EuroSAT systems commonly use ImageNet pre-training, and established attention mechanisms address channel or spatial features in different ways. This work evaluates from-scratch performance and combines two attention types through learnable fusion.

2.1 Satellite Image Classification Benchmarks

  • Reported EuroSAT accuracies are 98.57% for ImageNet-fine-tuned ResNet-50, 98.1% for EfficientNets [6], 99.19% for Vision Transformers [5], and 99.41% for ensembles; all rely on ImageNet pre-training. From scratch, the model reaches 97.23%, within 1.34% of fine-tuned ResNet-50.Source: §2.1 · p. 2, §2.3 · p. 2

2.2 Attention Mechanisms in CNNs

  • SENet [4], CBAM [2], and Coordinate Attention [3] provide channel, sequential channel-spatial, and factorized spatial attention; this work instead learns fusion of SE and Coordinate Attention.The fusion is intended to let the model discover a spatial-spectral balance.Source: §2.2 · p. 2

2.3 From-Scratch Training versus Transfer Learning

  • He et al. [8] showed scratch training can match ImageNet pre-training for object detection; here, 97.23% on EuroSAT is within 1.34% of fine-tuned ResNet-50.The paper describes this as empirical evidence of practical viability for satellite classification.Source: §2.3 · p. 2

3.1 Experimental Setup

The experiments use a reproducible train-validation-test split of EuroSAT RGB imagery, apply image augmentation, and train with mixed precision on a GPU.

Dataset

  • EuroSAT RGB contains 27,000 64×64-pixel, three-channel images across 10 classes, split 70/15/15 into 18,900 training, 4,050 validation, and 4,050 test samples.The split uses random seed 42 for reproducibility.Source: §3.1 · p. 2, Figure 1 · p. 3

Data Augmentation

  • Training augmentation includes 90°-increment rotations, horizontal and vertical flips, color jitter, Gaussian blur, and random erasing; images are normalized with ImageNet statistics.Color jitter is ±30%, blur uses a 3×3 kernel with σ=0.1–2.0, and random erasing has p=0.3.Source: §3.1 · pp. 2–3

Training Infrastructure

  • Training uses PyTorch 2.0, an NVIDIA Tesla T4 GPU, mixed-precision FP16, and batch size 64.Source: §3.1 · p. 3

3.2 Baseline Architecture (Model 1)

The baseline uses three convolutional blocks and achieves 94.30% test accuracy, with River–Highway confusion identified as its central weakness.

3.2.1 Design

  • The baseline has three convolutional blocks with channels [32, 64, 128], followed by global average pooling, FC(512), dropout, and a 10-class output.The model has 2.1M parameters.Source: §3.2.1 · p. 3

3.2.2 Training Configuration

  • Training uses Adam with lr=1e-3 and ReduceLROnPlateau with patience 3 and factor 0.5 for 30 epochs.Training takes 1.5 hours.Source: §3.2.2 · p. 3

3.2.3 Results and Analysis

  • 94.30% test accuracy; Forest reached 99.1% and SeaLake 99.0%, while River was 88.0% and Highway 93.1%, with 27 River–Highway errors (~12% of total errors).Source: §3.2.3 · p. 3
  • At 64×64 resolution, rivers and highways share low-contrast, elongated gray forms, while the deepest baseline layer has an 8×8 receptive field and no attention mechanism.The paper identifies these constraints as limiting discrimination of subtle contextual cues.Source: §3.2.3 · p. 3

3.3 CBAM-Enhanced Architecture (Model 2)

The CBAM-enhanced model targets River–Highway confusion with channel and spatial attention, improving overall and class-level results. The gains also expose a trade-off: spatial attention helps infrastructure classes while vegetation classification declines.

3.3.1 Design

  • A 7-layer network adds CBAM blocks in layers 2–7, applying sequential channel and spatial attention.The design has 7.4M parameters; its channel attention uses a 16:1 MLP reduction ratio.Source: §3.3.1 · p. 3

3.3.3 Results and Trade-off Discovery

  • 95.98% test accuracy (+1.68%), with River-Highway confusions reduced 70% (27→8).River accuracy rises from 88.0% to 95.5% (+7.5%), and Highway from 93.1% to 95.4% (+2.3%).Source: §3.3.3 · p. 3
  • CBAM’s spatial attention captures directional patterns useful for rivers and highways, but vegetation accuracy falls from 93.0% to 92.6% (-0.4%).The analysis attributes vegetation vulnerability to spatial-attention bias away from spectral and textural information used to distinguish crop types.Source: §3.3.3 · p. 3
  • The results identify orthogonal feature needs: infrastructure classes benefit from spatial attention, while land-cover classes require spectral attention.Optimizing for one modality degrades performance on the other.Source: §3.3.3 · p. 3

3.4 Balanced Multi-Task Attention Architecture (Model 3)

The architecture combines parallel spatial and spectral attention paths through learnable fusion, within a four-stage residual network. Progressive DropBlock and class-balanced loss address overfitting and confusion-pattern imbalance, with training configured using AdamW and cosine warm restarts.

3.4.1 Core Innovation

  • The core innovation combines Coordinate Attention for spatial features and Squeeze-Excitation blocks for spectral features, joined by a learnable fusion parameter.The fusion weight is learned rather than fixed.Source: §3.4.1 · p. 4

3.4.2 Architecture

  • The model uses a four-stage ResNet-style structure with [3] [2] residual blocks and channel progression [64, 128, 256, 512].Each residual block incorporates balanced multi-task attention; the network has 11.2M parameters.Source: §3.4.2 · p. 4

3.4.3 Balanced Multi-Task Attention Mechanism

  • Coordinate Attention factorizes global pooling into height and width encodings for directional patterns, while Squeeze-Excitation channel gating emphasizes spectral variations.The respective reduction ratios are 8:1 and 16:1.Source: §3.4.3 · p. 4

3.4.4 Regularization Strategy

  • DropBlock rates increase from 0.05 to 0.20 across stages, while class-balanced loss assigns higher weights to frequently confused classes.DropBlock uses 7×7 blocks; loss weights are 1.3 for frequently confused classes, 1.0 for moderate-difficulty classes, and 0.8 for consistently high-accuracy classes.Source: §3.4.4 · p. 4

3.4.5 Training Configuration

  • Training uses AdamW, cosine annealing with warm restarts, mixed precision, and early stopping; it terminated at epoch 30 with the best checkpoint at epoch 15.The best checkpoint reached 97.09% validation accuracy.Source: §3.4.5 · p. 4

4 Results

The balanced model achieves 97.23% accuracy, with every class at or above 94.46%. Results also show fewer confusion errors, lower confidence on incorrect predictions, and a learned near-equal balance between spatial and spectral attention.

4.1 Overall Performance

  • 97.23% accuracy places the balanced architecture within 1.34% of fine-tuned ResNet-50 without pre-training.Table 1 presents the comparison across models on the EuroSAT RGB test set.Source: Table 1 · p. 4, §4.1 · p. 4

4.2 Per-Class Metrics

  • All classes achieve ≥94.46% accuracy, with Cohen’s Kappa and Matthews Correlation Coefficient both at 0.9692.Table 2 reports per-class metrics for the balanced 12-layer model.Source: §4.2 · p. 4, Table 2 · p. 5

4.3 Confusion Pattern Analysis

  • The balanced model reaches River 96.27% and 5 River-Highway errors, with 112/4,050 total errors (2.77%).The comparison traces River accuracy across the baseline (88.0%), CBAM (95.5%), and balanced (96.27%) models.Source: §4.3 · p. 5, Figure 2 · p. 5

4.4 Confidence Calibration

  • Correct predictions average 90.14% confidence, versus 65.89% for incorrect predictions, a gap the authors describe as uncertainty awareness.The paper notes that low-confidence predictions can be flagged for human review in production deployment.Source: §4.4 · p. 5

4.5 Attention Balance Analysis

  • α converges to approximately 0.57 by epoch 20, corresponding to 57% Coordinate Attention and 43% SE weighting.The authors characterize this as near-equal spatial and spectral importance, with a slight spatial bias.Source: §4.2 · p. 4, §4.5 · p. 5

5 Discussion

The discussion argues that satellite imagery benefits from attention to both spatial and spectral features, with learnable fusion and progressive DropBlock contributing to the architecture. The from-scratch model approaches fine-tuned ResNet-50 performance, while the evaluation and future work leave several scope boundaries.

5.1 Architectural Insights

  • Infrastructure classes depend on spatial patterns, while land-cover classes depend on spectral signatures; balanced dual-path attention addresses both feature requirements.The discussion contrasts shape, layout, and directional structure with color, texture, and spectral indices.Source: §5.1 · pp. 5–6
  • Learnable fusion reaches α ≈0.57 and outperforms fixed combination by +0.27% in ablation.The authors associate the slight spatial bias with the presence of multiple infrastructure classes.Source: §5.1 · p. 6
  • Progressive DropBlock rates of 5–20% by depth produce the largest reported ablation gain, +0.65%.The discussion assigns lower-level feature preservation to early layers and overfitting risk to deeper, class-specific features.Source: §5.1 · p. 6

5.2 Comparison to State-of-the-Art

  • 97.23% accuracy is within 1.34% of fine-tuned ResNet-50 (98.57%), while eliminating pre-training dependency.The authors describe systematic design as substantially closing the gap and making from-scratch training viable when pre-training is unavailable or suboptimal.Source: §5.2 · p. 6

5.3 Limitations and Future Work

  • Evaluation is RGB-only although full EuroSAT contains 13 spectral bands, uses a single model, and retains PermanentCrop accuracy of 94.46%.The discussion lists ensembles, multispectral extension, architecture search, transfer to UC Merced and AID, and temporal fusion as future directions.Source: §5.3 · p. 6
  • Proposed next steps include multispectral inputs, ensembles, architecture search, transfer to other datasets, and temporal fusion.Named transfer targets are UC Merced and AID.Source: §5.3 · p. 6

6 Conclusion

The study concludes that balanced multi-task attention enables a custom CNN to achieve competitive satellite classification without pre-training. Its learned fusion supports near-equal spatial and spectral weighting, and the authors present systematic architecture design as a template for specialized domains.

6 Conclusion

  • Learnable fusion converges to α ≈0.57, while balanced dual-path attention outperforms single-modality approaches for heterogeneous imagery.The mechanism combines Coordinate Attention for spatial features with Squeeze-Excitation blocks for spectral features.Source: §6 · p. 6
  • The study reports performance within 1.34% of pre-trained ResNet-50 while requiring no external data, and finds balanced dual-path attention outperforms single-modality approaches for heterogeneous imagery.Source: §6 · p. 6
  • Systematic architectural design can achieve competitive performance where pre-training is unavailable, offering a template for iterative ML engineering in specialized domains.Source: §6 · p. 6

Related papers

Loading 2510.15527v1…