PaperlayerView exact revision on arXiv

Source-linked AI summary

GETNET: A General End-to-end Two-dimensional CNN Framework for Hyperspectral Image Change Detection

Qi Wang, Zhenghang Yuan, Qian Du, Xuelong Li

arXiv:1905.01662v1cs.CVeess.IV

TL;DR

Hyperspectral image change detection offers richer information but must address high dimensionality, mixed pixels, and underused subpixel abundance information. GETNET combines mixed-affinity matrices, hybrid unmixing, and a 2-D CNN, and evaluations on existing and newly constructed datasets show its effectiveness through extensive comparisons.

  • Problem

    HSI-CD must handle high-dimensional spectral data and mixed pixels while making fuller use of detailed subpixel abundance information.

  • Method

    GETNET uses hybrid unmixing and a mixed-affinity matrix to fuse hyperspectral and abundance information before learning features with a 2-D CNN.

  • Results

    GETNET was evaluated on existing and newly constructed HSI-CD datasets and showed effectiveness through extensive comparisons and analyses.

  • Takeaways & Limitations

    The framework provides a general HSI-CD approach that combines multi-source information fusion with cross-channel gradient features and aims for robust, generalizable feature learning.

Abstract

from arXiv · show

Change detection (CD) is an important application of remote sensing, which provides timely change information about large-scale Earth surface. With the emergence of hyperspectral imagery, CD technology has been greatly promoted, as hyperspectral data with the highspectral resolution are capable of detecting finer changes than using the traditional multispectral imagery. Nevertheless, the high dimension of hyperspectral data makes it difficult to implement traditional CD algorithms. Besides, endmember abundance information at subpixel level is often not fully utilized. In order to better handle high dimension problem and explore abundance information, this paper presents a General End-to-end Two-dimensional CNN (GETNET) framework for hyperspectral image change detection (HSI-CD). The main contributions of this work are threefold: 1) Mixed-affinity matrix that integrates subpixel representation is introduced to mine more cross-channel gradient features and fuse multi-source information; 2) 2-D CNN is designed to learn the discriminative features effectively from multi-source data at a higher level and enhance the generalization ability of the proposed CD algorithm; 3) A new HSI-CD data set is designed for the objective comparison of different methods. Experimental results on real hyperspectral data sets demonstrate the proposed method outperforms most of the state-of-the-arts.

I. INTRODUCTION

HSI change detection benefits from rich spectral information but remains challenged by mixed pixels, high dimensionality, limited datasets, and incomplete use of subpixel and cross-spectral information. GETNET addresses these challenges with mixed-affinity matrices, an end-to-end 2-D CNN, and a new evaluation dataset.

  • HSIs can identify finer changes because their high spectral resolution reflects detailed object composition.
  • Mixed pixels combine multiple substances, so assigning each pixel to one substance may represent it inaccurately.
  • Hyperspectral data are difficult for some CD algorithms to handle, while dimensionality reduction can lose detailed information.
  • HSI-CD datasets remain limited because ground-truth construction requires time-consuming and expensive field work and manual labeling.
  • Spectral unmixing decomposes mixed pixels into endmembers and abundance maps that provide subpixel composition and interpretable change information.
  • GETNET combines a mixed-affinity matrix, an end-to-end 2-D CNN, and a new HSI-CD dataset for multi-source feature learning and objective comparison.

II. RELATED WORK

HSI-CD research includes arithmetic, transformation, classification, and advanced deep-learning methods, while unmixing adds subpixel information but remains challenging and underexplored.

  • Classical multi-band CD methods fall into image arithmetic, image transformation, image classification, and other advanced approaches.
  • CVA detects change using magnitude and direction derived from spectral-vector subtraction.
  • Transformation methods emphasize changed pixels in feature spaces, but PCA depends on image statistics and is vulnerable to unbalanced data.
  • Classification methods compare multitemporal classified images or classify differential and compound features to identify changed regions.
  • Deep-learning methods learn representations from multitemporal vectors or reshaped pixels for multi-band image analysis.
  • Unmixing provides subpixel information by analyzing abundance changes, but HSI-CD unmixing remains challenging and insufficiently studied.

III. METHODOLOGY

GETNET combines hybrid spectral unmixing, mixed-source data construction, and a 2-D CNN for hyperspectral change detection. The method uses linear and nonlinear abundance information alongside hyperspectral data to mine subpixel-level content.

  • III. METHODOLOGY: The proposed pipeline applies spectral unmixing to two preprocessed hyperspectral images, then combines the resulting abundance maps with the original imagery.The final deep network produces the change-detection result.
  • III. METHODOLOGY: ATGP extracts potential target endmembers, after which abundance estimation produces linear and nonlinear abundance maps for each pixel.The same endmembers are used for the linear and nonlinear unmixing procedures.
  • III. METHODOLOGY: Hybrid-unmixing combines linear and nonlinear mixture models to use their respective advantages for hyperspectral change detection.The paper notes that few prior works combine both model types in HSI-CD.
  • III. METHODOLOGY: The HSI and both abundance-map cubes are stacked along the spectral dimension into multi-source data with size w × h × (b + 2m).The construction preserves hyperspectral information together with linear and nonlinear abundance information.

B. Mixed-affinity Matrix Generation for Information Fusion

The mixed-affinity matrix converts corresponding multitemporal pixel vectors and abundance representations into a 2-D representation for information fusion. Its regions encode pixel-level, spectral, and subpixel-level relationships before 2-D CNN processing.

  • B. Mixed-affinity Matrix Generation for Information Fusion: The mixed-affinity matrix represents relationships between corresponding pixels across wavelengths and abundance maps.It is an n × n matrix with n = b + 2m.
  • B. Mixed-affinity Matrix Generation for Information Fusion: Parts A, B, D, and E encode spectral, linear-abundance, cross-abundance, and nonlinear-abundance differences between corresponding observations.Part A represents pixel-level differences, while Parts B, D, and E reveal subpixel-level differences.
  • B. Mixed-affinity Matrix Generation for Information Fusion: The matrix values are passed to a 2-D CNN, whose output is a binary changed-versus-unchanged classification.The value of Kij indicates similarity between corresponding pixels, with larger values denoting greater similarity.
  • B. Mixed-affinity Matrix Generation for Information Fusion: The matrix simultaneously fuses preprocessed hyperspectral data with linear and nonlinear abundance maps, naturally representing pixel- and subpixel-level information.This design processes multiple information sources together rather than relying only on spectral differences.
  • B. Mixed-affinity Matrix Generation for Information Fusion: Mapping two 1-D pixel vectors to a 2-D matrix exposes cross-channel differences among b spectral channels and 2m abundance maps.The authors describe this as maximizing multi-source information utilization.

C. GETNET with “Separation and Combination”

GETNET treats hyperspectral change detection as binary classification and designs an end-to-end 2-D CNN specifically for mixed-affinity representations. The architecture differs from conventional image-classification networks because its input has structured spectral and abundance regions.

  • C. GETNET with “Separation and Combination”: HSI-CD is formulated as binary classification of changed and unchanged pixels within the proposed end-to-end 2-D CNN.The network operates on mixed-affinity matrices rather than ordinary image inputs.
  • C. GETNET with “Separation and Combination”: The architecture is designed for the specific change-detection task instead of simply adopting a deeper conventional CNN used for general image classification.The passage contrasts this design with networks using residual layers or inception modules.

1) “Separation and Combination”:

GETNET separates spectral and abundance processing in early convolutional layers, then combines their learned higher-level features for classification. Training uses balanced positive and negative samples, while experiments evaluate the method on four real HSI datasets.

  • 1) “Separation and Combination”:: Different convolution kernels are used for distinct spectral and abundance regions because these feature types have different natures.The matrix places spectral affinity in the upper-left and abundance affinity in the lower-right regions.
  • 1) “Separation and Combination”:: Locally sharing convolution and max-pooling layers learn spectral and abundance features separately before fully connected layers classify them.The fully connected layers serve as the classifier for the learned higher-level representations.
  • 1) “Separation and Combination”:: Mixed-affinity matrices enter the 2-D CNN and its outputs are changed/unchanged pixel labels from which a change-difference image is obtained.Ground-truth maps are used for evaluation rather than training in the described HSI analysis.
  • 1) “Separation and Combination”:: The initial ratio of positive to negative training samples is set to 1:1 to reduce bias from class imbalance.The passage also notes that deeper networks require more training samples when trained without pretraining.
  • 1) “Separation and Combination”:: Experiments compare the proposed method and state-of-the-art methods on four real hyperspectral image change-detection datasets.The section introduces datasets, evaluation measures, and detailed performance analysis.

A. Datasets Description

The experiments use four real HSI datasets, including the newly constructed river dataset and three existing datasets, with ground-truth maps supporting evaluation.

  • Ground truth: Each dataset includes two multitemporal HSIs acquired at the same position and a ground-truth map of changed and unchanged classes.Ground-truth construction uses field investigation, accurate algorithms, and spectral-value queries for uncertain pixels.
  • Data sources: The study uses EO-1 Hyperion imagery, with selected high-SNR bands after noise elimination and existing ground-truth maps from prior work.Hyperion covers 0.4–2.5 microns with approximately 242 bands, 10 nm spectral resolution, and 30 m spatial resolution.
  • Existing datasets: The farmland dataset contains 450 × 140 pixels and 155 selected bands from images acquired in 2006 and 2007.Its primary visual change is farmland size, and 20.95% of pixels are labeled samples.
  • Existing datasets: The countryside dataset contains 633 × 201 pixels and 166 selected bands from images acquired in 2007 and 2008.Its primary visual change is the size of rural areas, with approximately 7.55% labeled pixels.
  • Existing datasets: The Poyang lake dataset contains 394 × 200 pixels and 158 available bands from images acquired in 2002 and 2004.Its primary visual change is land change, with approximately 2.17% labeled pixels.

2) Newly Constructed Data Set:

The newly constructed river dataset captures two Hyperion observations from 2013, while evaluation compares detected changes against ground truth using pixel-level metrics.

  • Newly constructed data set: The river dataset comprises HSIs acquired in Jiangsu province on May 3 and December 31, 2013, with 463 × 241 pixels and 198 usable bands.Its main change type is disappearance of substance in the river.
  • Evaluation measures: The evaluation uses overall accuracy, Kappa coefficient, and confusion matrices to assess pixel-level change detection.Overall accuracy is computed from true positives and true negatives divided by all four confusion-matrix counts.
  • Evaluation measures: Kappa coefficient measures agreement between the change difference image and ground-truth map, with higher values indicating better results.
  • Evaluation measures: A confusion matrix describes the distribution of predicted and actual changed or unchanged classifications.

C. Experimental results

Experiments compare GETNET with conventional, transformation-based, SVM, and patch-based CNN methods across four real HSI datasets. GETNET is reported as effective across the datasets, using mixed-affinity inputs and abundance information.

  • Experimental setup: The comparison includes CVA, PCA-CVA, IR-MAD, SVM, patch-based CNN, and GETNET across four real HSI datasets.These methods represent image arithmetic, image transformation, support-vector classification, and deep-learning approaches.
  • Experimental setup: GETNET is trained from scratch with the same parameter setup across all datasets, without parameter adjustment or pretrained models.Training uses 30,000 steps, batch size 96, Adagrad, learning rate 10^-4, and epsilon 10^-8.
  • Farmland results: GETNET provides the best reported farmland performance after jointly considering overall accuracy and Kappa coefficient.GETNET without unmixing achieves the second-best performance, while its Kappa indicates almost perfect agreement with ground truth.

2) Experiments on the Countryside Data Set:

On the countryside dataset, GETNET performs best among the compared methods, while results on Poyang lake and river show advantages and scope boundaries across different change patterns.

  • Countryside results: GETNET achieves the best countryside performance, with an OA approximately 8% higher than the patch-based CNN.GETNET and CVA both achieve OA above 98%, whereas GETNET without unmixing performs below CVA.
  • Countryside results: Abundance information improves GETNET over its version without unmixing across all four datasets.The paper attributes this to subpixel-level representation being useful for change detection.
  • Poyang lake results: GETNET identifies dispersed Poyang lake changes and changes in spacer width that patch-based CNN does not recognize well.GETNET produces the best overall Poyang lake performance, while GETNET without unmixing ranks second in OA.

4) Experiments on the River Data Set:

On the River data set, GETNET generally performs strongly, although CVA slightly leads it in this case. Ablation and comparative analyses support the value of its end-to-end design and multisource fusion.

  • River data set results: CVA achieves the best result on the River data set, while CVA and GETNET both reach OA up to 95%.PCA-CVA and CNN have similar accuracy of about 94%, whereas IR-MAD performs lower.
  • Ablation study: GETNET is almost always better than GETNET without unmixing, supporting the use of abundance maps in the mixed-affinity input.The comparison evaluates GETNET with and without abundance maps to validate the effectiveness of unmixing information.
  • Method comparison: GETNET outperforms CNN because end-to-end training avoids error accumulation and multisource fusion mines spectral-gradient characteristics.The fused inputs include hyperspectral data and unmixing abundance information.
  • Overall evaluation: GETNET outperforms other CD methods in most cases and remains robust across different hyperspectral images.The experiments cover multiple change types and attribute these merits to the mixed-affinity matrix and GETNET design.
  • Framework design: The framework uses a mixed-affinity matrix with a 2-D CNN to process multisource information and extract more abundant cross-channel gradient features.The framework is designed to learn significant features with generalization and robustness, and is evaluated on existing and newly constructed data sets.
Loading 1905.01662v1…