PaperlayerView exact revision on arXiv

Source-linked AI summary

Data Leakage in Patch-Based Hyperspectral Image Classification: Quantifying the Impact of Spatial Overlap

Mohammed Q. Alkhatib

arXiv:2610.08770v1cs.CV

TL;DR

Patch-based HSI classification can leak information when random train-test sampling from one image creates overlapping patches. This paper measures that overlap and compares six classifiers across random and non-random spatial sampling; deep models show substantial OA drops under the latter setting.

  • Problem

    Same-image random sampling can create overlapping train-test patches and optimistic performance estimates, while the relationship between same-class overlap and classification performance warrants quantification.

    Source: §1 · p. 1

  • Method

    The study evaluates six classifiers under random and non-random spatial sampling and measures global overlap with OP and local severity among affected patches with AOR.

    Source: §1 · p. 1, §3.2 · p. 2

  • Results

    Under non-random spatial sampling, OA drops by 40.97 percentage points for 3D-CNN, 40.71 for ViT, and 38.81 for 2D-CNN.

    Source: §4 · p. 3

  • Takeaways & Limitations

    Random patch-based evaluation can produce optimistic performance estimates, especially for models exploiting spatial neighborhoods, while non-random spatial sampling provides a stricter assessment of generalization.

    Source: §4 · pp. 3–4

    The generality of the observed leakage effects across additional HSI datasets and training ratios remains to be assessed.

    Source: §5 · p. 5

Abstract

from arXiv · show

Patch-based learning improves hyperspectral image (HSI) classification by exploiting local spectral-spatial information, but random train-test sampling from the same image can cause spatial patch overlap, leading to data leakage and optimistic performance estimates. This paper investigates same-class train-test spatial overlap in patch-based HSI classification using two measures: overlap percentage (OP), which quantifies the global amount of overlapped testing patch pixels, and average overlap ratio (AOR), which measures the local severity among affected testing patches. Experiments on the Pavia University dataset compare random and non-random spatial sampling using SVM, MLP, 2D-CNN, 3D-CNN, ViT, and MorpMamba. The results show that deep patch-based models achieve high accuracy under random sampling, with 3D-CNN reaching 96.17% Overall Accuracy (OA), but drop substantially under non-random spatial sampling, where 3D-CNN decreases to 55.20% and ViT and 2D-CNN drop by 40.71 and 38.81 percentage points (PP), respectively. Patch-size analysis further shows that increasing the patch size from 5x5 to 19x19 raises the random-sampling overlap percentage from 23.28% to 77.02%. These findings demonstrate that random patch-based evaluation can substantially inflate classification performance, especially for models that strongly exploit spatial context. The code associated with this paper is available at: https://github.com/mqalkhatib/Data_Leakage_in_HSI_Classification.

1. INTRODUCTION

HSI classification faces high dimensionality, spectral redundancy, within-class spatial variability, and few labeled samples. This paper examines same-class train-test patch overlap and its relationship to classification performance across six classifiers and two sampling protocols.

1. INTRODUCTION

  • HSI uses contiguous-band spectral responses to assign pixel labels, with material-specific information aiding discrimination between visually similar land-cover classes.Source: §1 · p. 1
  • HSI classification remains challenging because of high dimensionality, spectral redundancy, within-class spatial variability, and limited labeled samples.Source: §1 · p. 1
  • Traditional SVMs and MLPs mainly depend on spectral features, while deep learning learns spectral-spatial representations directly from HSI data.Model families progressed from 1D-CNNs to 2D-CNNs, 3D-CNNs, hybrid 2D-3D CNNs, and transformer- and Mamba-based models.Source: §1 · p. 1
  • Random train-test sampling from the same image can make patches share pixels, weakening train-test independence and producing overly optimistic performance estimates.Larger patches, higher training ratios, and spatial autocorrelation can further intensify this leakage; spatially disjoint sampling may reduce it but can further limit labeled training samples.Source: §1 · p. 1
  • The study jointly quantifies global overlap extent and local severity, relating same-class train-test overlap to performance across six classifiers and two sampling protocols.It evaluates SVM, MLP, 2D-CNN, 3D-CNN, ViT, and MorpMamba under random and non-random spatial sampling.Source: §1 · p. 1

2. RELATED WORK

Patch-based HSI learning uses local neighborhoods to exploit spectral-spatial correlations, but overlapping patches can correlate training and testing samples. Prior leakage-reduction strategies address this dependency, while this work jointly measures overlap extent and severity and relates them to performance across classifier families.

2.1. Patch-Based Learning in HSI

  • Patch-based HSI learning extracts a local neighborhood around each labeled pixel, letting classifiers exploit local spectral-spatial correlations.Source: §2.1 · p. 2
  • Larger patches increase spatial coverage and the probability of train-test overlap, while same-scene random sampling can place samples close together due to spatial continuity.Neighboring patches may share pixels, making samples correlated even when their center pixels differ.Source: §2.1 · p. 2

2.2. Data Leakage in Remote Sensing and Computer Vision

  • Spatially disjoint, iterative, and augmentation-based sampling strategies have been proposed to reduce patch dependency, but may reduce usable labeled samples and primarily focus on mitigation.Source: §2.2 · p. 2
  • This work addresses a gap in jointly quantifying global overlap extent and local severity and relating them to performance degradation across classifier families under matched conditions.It evaluates traditional machine-learning, CNN-, Transformer-, and Mamba-based classifiers under random and leakage-reduced spatial sampling.Source: §2.2 · p. 2

3. METHODOLOGY

The methodology describes a PCA-based patch classification pipeline, defines two measures of same-class train–test overlap, and compares random with spatially separated sampling. Experiments use six classifiers on Pavia University under matched settings.

3.1. Patch-Based HSI Classification

  • The classification pipeline applies PCA to the hyperspectral image, extracts spatial patches, and passes them to a deep model and classification head.Softmax produces the final class predictions.Source: §3.1 · p. 2

3.2. Quantifying Train-Test Spatial Overlap

  • Overlap percentage (OP) measures same-class overlap across all testing-patch pixels, while average overlap ratio (AOR) averages overlap only among affected testing patches.A sampling protocol can therefore have low OP but high AOR when few testing patches overlap substantially.Source: §3.2 · pp. 2–3

3.3. Non-Random Spatial Sampling Protocol

  • Random sampling may place training and testing patches in overlapping regions, whereas non-random spatial sampling selects samples with greater separation for a stricter, leakage-reduced evaluation.Testing pixels are not explicitly excluded after patch extraction, so the non-random protocol is not fully leakage-free; performance drops indicate possible benefit from spatial dependency under random sampling.Source: §3.3 · p. 3

3.4. Experimental Setup

  • Pavia University experiments compare six classifiers under matched sampling settings, using 9 × 9 × 15 patches and 10 independent runs.PCA reduces the data to 15 components; 94 training and 18 validation samples per class are used, with the remainder for testing.Source: §3.4 · p. 3

4. RESULTS AND DISCUSSION

Deep patch-based classifiers score highly under random sampling but lose substantial accuracy under non-random spatial sampling. Larger patches increase same-class overlap, while global overlap and local severity can diverge under non-random sampling.

4. RESULTS AND DISCUSSION

  • 96.17% OA for 3D-CNN leads random-sampling results, ahead of ViT at 91.78%, 2D-CNN at 86.97%, and MorpMamba at 81.07%.SVM and MLP obtain 62.15% and 69.50% OA, respectively.Source: §4 · p. 3, Table 1 · p. 4
  • For 3D-CNN, non-random sampling reduces accuracy on Asphalt from 94.28% to 46.27%, Meadows from 97.63% to 49.78%, Gravel from 91.59% to 43.99%, and Bare Soil from 96.38% to 47.89%.Compact, spectrally distinctive classes such as Metal Sheet and Shadow remain comparatively easier to classify under non-random sampling.Source: §4 · p. 3
  • 40.97, 40.71, and 38.81 percentage-point OA drops occur for 3D-CNN, ViT, and 2D-CNN, respectively, under non-random spatial sampling.MorpMamba drops 32.62 percentage points, while SVM and MLP drop 10.90 and 19.35 percentage points.Source: §4 · p. 3, Table 3 · p. 4

4.1. Effect of Patch Size on Same-Class Spatial Leakage

  • Under random sampling, increasing patch size from 5×5 to 19×19 raises OP from 23.28% to 77.02% and AOR from 47.29% to 79.91%.Under non-random sampling, OP rises from 0.83% to 4.66%, while AOR decreases from 45.26% to around 39% for larger patches.Source: §4.1 · p. 4, Table 4 · p. 4
  • OP measures global overlap, whereas AOR captures severity among affected testing patches, so few affected patches can still have substantial local overlap.AOR is computed only for testing patches with non-zero overlap.Source: §4.1 · pp. 4–5

5. CONCLUSION

Random sampling can substantially inflate patch-based HSI classification performance, particularly for deep spatial-context models; larger patches also increase measured overlap. Broader dataset and training-ratio evaluation remains future work.

5. CONCLUSION

  • 40.97, 40.71, and 38.81 percentage points: under non-random spatial sampling, OA fell for 3D-CNN, ViT, and 2D-CNN, respectively, while SVM and MLP had smaller reductions.Source: §5 · p. 5
  • 23.28% to 77.02%: random-sampling overlap percentage rose as patch size increased from 5 × 5 to 19 × 19.Source: §5 · p. 5
  • Random patch-based sampling can substantially inflate classification performance, especially for deep models that exploit spatial context; the study uses OP and AOR to quantify overlap.Source: §5 · p. 5
  • Future work will test additional hyperspectral datasets and training ratios to assess how general the observed leakage effects are.Source: §5 · p. 5
Loading 2610.08770v1…