PaperlayerView exact revision on arXiv

Source-linked AI summary

Dual Contrastive Network for Few-Shot Remote Sensing Image Scene Classification

Zhong Ji, Liyuan Hou, Xuan Wang, Gang Wang, Yanwei Pang

arXiv:2603.23161v1cs.CV

TL;DR

FS-RSISC must classify remote sensing scenes from few labeled samples despite small inter-class and large intra-class variances. DCN addresses this with complementary context- and detail-guided supervised contrastive branches, and reports competitive results across four benchmark datasets, including gains over prior methods in several settings.

  • Problem

    FS-RSISC seeks to classify remote sensing images with few labeled samples, while small inter-class and large intra-class variances complicate feature learning.

  • Method

    DCN is a transfer-based network with CCL and DCL branches that use Condenser and Smelter Networks to learn discriminative context and invariant local-detail features.

  • Results

    DCN achieves competitive performance on four benchmark datasets, including at least 3.56% and 5.38% improvements over state-of-the-art approaches on NWPU-RESISC45 and AID, respectively, for 5-way 5-shot.

  • Takeaways & Limitations

    The dual contrastive framework learns complementary context and detail representations that support adaptation to different novel remote sensing scene classification tasks.

Abstract

from arXiv · show

Few-shot remote sensing image scene classification (FS-RSISC) aims at classifying remote sensing images with only a few labeled samples. The main challenges lie in small inter-class variances and large intra-class variances, which are the inherent property of remote sensing images. To address these challenges, we propose a transfer-based Dual Contrastive Network (DCN), which incorporates two auxiliary supervised contrastive learning branches during the training process. Specifically, one is a Context-guided Contrastive Learning (CCL) branch and the other is a Detail-guided Contrastive Learning (DCL) branch, which focus on inter-class discriminability and intra-class invariance, respectively. In the CCL branch, we first devise a Condenser Network to capture context features, and then leverage a supervised contrastive learning on top of the obtained context features to facilitate the model to learn more discriminative features. In the DCL branch, a Smelter Network is designed to highlight the significant local detail information. And then we construct a supervised contrastive learning based on the detail feature maps to fully exploit the spatial information in each map, enabling the model to concentrate on invariant detail features. Extensive experiments on four public benchmark remote sensing datasets demonstrate the competitive performance of our proposed DCN.

I. INTRODUCTION

FS-RSISC addresses remote sensing scene classification with few labeled samples, where small inter-class and large intra-class variances make feature learning difficult. DCN uses complementary context- and detail-guided contrastive branches to learn discriminative and invariant features, achieving competitive benchmark performance.

  • FS-RSISC targets remote sensing scene classification when only a few labeled samples are available.
  • Small inter-class variances and large intra-class variances make remote sensing scenes difficult to distinguish and represent consistently.
  • DCN combines Context-guided Contrastive Learning and Detail-guided Contrastive Learning to learn complementary discriminative and detail features.
  • The CCL branch uses a Condenser Network and supervised contrastive learning to exploit global context and improve inter-class discrimination.
  • The DCL branch uses a Smelter Network and feature-map contrastive learning to capture invariant local details across intra-class samples.
  • DCN achieves competitive performance on four benchmark datasets, outperforming state-of-the-art methods by at least 3.56% on NWPU-RESISC45 and 5.38% on AID in 5-way 5-shot settings.

II. RELATED WORK

Related work covers remote sensing scene classification, transfer learning, and few-shot learning, while positioning DCN as a transfer-based framework that jointly exploits global context and local detail features.

  • Remote Sensing Image Scene Classification: Remote sensing scene classification has used CNNs, covariance-based models, and hybrid CNN-CapsNet architectures to learn representative image features.
  • Remote Sensing Image Scene Classification: Transfer learning commonly adapts large-scale pretrained CNNs from natural images to remote sensing imagery.
  • Remote Sensing Image Scene Classification: Recent remote sensing methods address small inter-class and large intra-class variances through texture, saliency, hierarchical metric learning, and related feature strategies.
  • Few-Shot Learning: Few-shot learning classifies samples from unseen classes using few labels and has been applied to remote sensing scene classification through approaches including meta-learning.
  • Proposed Framework: DCN employs complementary CCL and DCL branches to explore global contexts and local details for different scene classification tasks.
  • Problem Setting: The transfer-learning pipeline pretrains an encoder on base classes and transfers it to novel tasks with disjoint classes and few-shot support and query sets.

B. Context-guided Contrastive Learning

The CCL branch uses a Condenser Network to transform encoder feature maps into context features by emphasizing scene-relevant global information and filtering irrelevant content.

  • CCL addresses small inter-class variances by exploiting global context to extract discriminative features that better reflect class differences.
  • Condenser Network: The Condenser Network reduces background and irrelevant-object interference when scene-relevant information is otherwise inapparent.
  • Squeeze Operation: A 1 × 1 convolution squeezes encoder feature maps Z ∈ R^h×w×c along channels into a spatial map Z_s ∈ R^h×w×1.
  • Spatial Weighting: Softmax weights spatial locations in Z_s, and the resulting weights are multiplied with Z to produce generated feature maps Z_g.
  • Expand Operation: The network squeezes Z_g again, applies Layer Normalization and ReLU, then expands the representation across channels to emphasize vital regions.
  • Context Features: Global average pooling converts the refined feature map into context features Z_c ∈ R^1×1×c.

2) Contrastive Learning:

The CCL branch uses augmented batches and supervised contrastive learning to improve context-feature discrimination, while DCL targets invariant local details through a Smelter Network and contrastive module.

  • CCL propagates two augmented batch expansions through the Feature Encoder and Condenser Network to obtain context features for contrastive learning.
  • Samples with the same label are positives, while samples with different labels are negatives relative to each anchor.
  • The CCL loss uses a normalized dot product as its distance function and includes a temperature parameter τ.
  • DCL combines a Smelter Network with contrastive learning to emphasize invariant local details in intra-class samples.

1) Smelter Network:

The Smelter Network extracts channel- and spatially informative attention to enhance significant local detail regions in feature maps.

  • The Smelter Network adaptively locates and enhances significant local detail information in input images.
  • Channel and spatial squeeze operations use average pooling on feature maps to produce channel-squeezed and spatial-squeezed maps.
  • The channel-squeezed map passes through 3 × 3 convolution, upsampling, and 1 × 1 convolution to derive spatial attention.
  • The spatial-squeezed map passes through two 1 × 1 convolutions to obtain channel attention.
  • Multiplying the attention maps, applying a 1 × 1 convolution and Sigmoid activation, and combining them with Z enhances significant local detail regions.

2) Contrastive Learning:

DCL aligns detail representations with self-attention before measuring pairwise distances and applying a supervised contrastive objective.

  • DCL propagates augmented batches through the Feature Encoder and Smelter Network to obtain detail feature maps Zd.
  • A self-attention-based feature alignment mechanism is introduced to capture correlations among detail features and measure their distance.
  • The projection network maps each detail feature map into query Q, key K, and value V vectors.
  • Values from paired samples are aligned in both directions using the other sample’s query and key representations.
  • The DCL loss contributes to the total contrastive learning loss, with α balancing the two contrastive components.

D. Total Loss

The training objective combines cross-entropy and contrastive losses, while meta-testing uses frozen feature modules and class prototypes for query prediction.

  • Cross-entropy loss optimizes the distributions of both context and detail feature spaces.
  • The logits o and ˜o come from the Condenser Network and Smelter Network, respectively, with β balancing their loss terms.
  • The full pre-training objective minimizes the cross-entropy loss and contrastive learning loss, balanced by γ.
  • During meta-testing, a support set from novel classes provides two possible prototypes based on context features and detail feature maps.
  • A query sample is assigned class probabilities using similarities to the context and detail prototypes.
  • The Feature Encoder, Condenser Network, and Smelter Network are frozen during meta-testing, and contrastive modules are discarded.

IV. EXPERIMENTS

The experiments evaluate DCN on four benchmark remote sensing datasets using disjoint training, validation, and test classes. ResNet-12 is used consistently as the backbone, with fixed preprocessing and optimization settings.

  • Datasets: DCN is evaluated on WHU-RS19, UC Merced, NWPU-RESISC45, and AID, covering both small and large datasets.The datasets use disjoint training, validation, and test classes following the few-shot learning protocol.
  • Datasets: WHU-RS19 contains 1,005 images across 19 classes and is split into 9 training, 5 validation, and 5 testing classes.
  • Datasets: UC Merced contains 100 images across 21 classes and is divided into 10 training, 6 validation, and 5 testing classes.
  • Datasets: NWPU-RESISC45 contains 31,500 256 × 256 images across 45 classes, split into 25 training, 10 validation, and 10 testing classes.
  • Datasets: AID contains 10,000 images across 30 classes and is split into 16 training, 7 validation, and 7 testing classes.
  • Implementation: ResNet-12 serves as the backbone for all four datasets, while images are cropped to 84 × 84 before training.The projection networks output 128-dimensional vectors, and pre-training uses Momentum-SGD.

C. Comparison with State-of-the-Art Methods

DCN is compared with established few-shot learning and FS-RSISC methods on small and large remote sensing datasets. It achieves the strongest reported results across the small datasets and substantial 5-shot gains on the large datasets, while ablations support both branches.

  • Comparison setup: The comparison includes nine typical few-shot learning methods and six state-of-the-art FS-RSISC methods, using ResNet-12 backbones for fairness.
  • Small datasets: 81.74% on WHU-RS19 and 58.64% on UC Merced are achieved by DCN in the 5-way 1-shot setting, leading both small-dataset comparisons.These results are at least 0.68% and 1% higher than competing methods, respectively.
  • Large datasets: 3.56% and 4.1% gains over the second-best approach are achieved on NWPU-RESISC45 and AID, respectively, in the 5-way 5-shot setting.On NWPU-RESISC45 5-way 1-shot, DCN is 0.85% below IDLN's best result.
  • Ablation studies: 3.14% ∼14.09% gains on 1-shot and 3.03% ∼8.5% gains on 5-shot are reported for DCN over the baseline across the ablation datasets.
  • Ablation studies: CCL improves over CCL without the Condenser Network, while DCL improves over DCL without the Smelter Network.The reported gains are 0.54% ∼1.9% and 0.46% ∼1.42% for CCL, while DCL gains about 1% on NWPU-RESISC45 and UC Merced.

E. Inter-class and Intra-class Variances Analysis

The variance and visualization analyses examine how the two DCN branches alter feature representations and how weighting coefficients affect performance. The Condenser Network emphasizes inter-class differences, while the Smelter Network reduces intra-class variation through local details.

  • Variance analysis: ResNet+CN produces larger inter-class embedding variance than the baseline, increasing distances between samples from different categories.
  • Variance analysis: ResNet+SN produces smaller intra-class variance than ResNet, indicating greater focus on local details across samples of the same class.The authors connect this behavior to variation in appearances and backgrounds within a class.
  • Visualization analysis: Grad-CAM visualizations compare baseline and branch-integrated networks for inter-class and intra-class images, highlighting regions associated with scene categories and local details.
  • Weighting coefficients: Performance peaks at α = 1 when balancing DCL and CCL, then decreases when α is either larger or smaller.
  • Weighting coefficients: β = 0.1 gives maximum performance, while β = 0 causes a small degradation, supporting the contribution of detail-feature classification.The detail features can compensate for bias from training the classifier using only context features.
  • Weighting coefficients: Performance peaks at γ = 1, indicating that dual contrastive learning contributes positively as an auxiliary task to classification.

G. Training Time and Parameters

DCN remains practical across the four benchmark datasets, with training completed in under 6 hours despite slightly higher parameter and FLOP counts than ResNet-12. The authors report substantial performance improvement over the baseline.

  • Training time: Training DCN across four datasets required less than 6 hours, using 200 epochs for NWPU-RESISC45 and 150 epochs for the other datasets.The training-time evaluation varied the epoch count by dataset.
  • Parameter efficiency: DCN has slightly more parameters and FLOPs than the original ResNet-12 embedding network.The comparison is reported in Table VI.
  • Parameter efficiency: Despite this modest computational increase, DCN achieves a substantial performance improvement over the baseline.The authors characterize this result as demonstrating high computational efficiency.
  • Overall evaluation: Competitive experimental results on four popular benchmark remote sensing datasets demonstrate DCN's effectiveness.The conclusion summarizes the overall benchmark evaluation of the transfer-based method.
Loading 2603.23161v1…