PaperlayerView exact revision on arXiv

Source-linked AI summary

Unsupervised Deep Slow Feature Analysis for Change Detection in Multi-Temporal Remote Sensing Images

Bo Du, Lixiang Ru, Chen Wu, Liangpei Zhang

arXiv:1812.00645v2cs.CV

TL;DR

Multi-temporal remote sensing images contain complex changes, while unchanged pixels can show slight differences from atmospheric, illumination, and sensor effects. DSFA uses deep networks with SFA to project bi-temporal imagery, suppress invariant components, and highlight changes; experiments across three datasets show it outperforms state-of-the-art methods.

  • Problem

    Existing change detection methods face complex multi-temporal changes, while unchanged pixels can exhibit slight feature differences from atmospheric conditions, illumination, and sensor calibration.

  • Method

    DSFA uses two deep networks to project bi-temporal data into a new feature space, then applies SFA constraints to suppress invariant components and highlight changed components.

  • Results

    DSFA outperformed other state-of-the-art methods, including SFA-based and deep network algorithms, on visual and quantitative evaluations across two multi-spectral and one hyperspectral dataset.

  • Takeaways & Limitations

    The proposed algorithm provides an unsupervised approach that does not require prior labeled pixels for training.

Abstract

from arXiv · show

Change detection has been a hotspot in remote sensing technology for a long time. With the increasing availability of multi-temporal remote sensing images, numerous change detection algorithms have been proposed. Among these methods, image transformation methods with feature extraction and mapping could effectively highlight the changed information and thus has better change detection performance. However, changes of multi-temporal images are usually complex, existing methods are not effective enough. In recent years, deep network has shown its brilliant performance in many fields including feature extraction and projection. Therefore, in this paper, based on deep network and slow feature analysis (SFA) theory, we proposed a new change detection algorithm for multi-temporal remotes sensing images called Deep Slow Feature Analysis (DSFA). In DSFA model, two symmetric deep networks are utilized for projecting the input data of bi-temporal imagery. Then, the SFA module is deployed to suppress the unchanged components and highlight the changed components of the transformed features. The CVA pre-detection is employed to find unchanged pixels with high confidence as training samples. Finally, the change intensity is calculated with chi-square distance and the changes are determined by threshold algorithms. The experiments are performed on two real-world datasets and a public hyperspectral dataset. The visual comparison and quantitative evaluation have both shown that DSFA could outperform the other state-of-the-art algorithms, including other SFA-based and deep learning methods.

I. INTRODUCTION

Multi-temporal remote sensing enables change detection, but complex changes and residual differences among unchanged pixels limit existing methods. The paper proposes DSFA, combining deep nonlinear feature projection with SFA to suppress unchanged components and highlight changes.

  • Multi-temporal imagery supports land-cover and land-use change detection for applications including deforestation, damage, vegetation, and disaster monitoring.
  • Existing change detection methods include image algebra and image transformation approaches that extract or combine effective features.
  • Unchanged pixels can show slight, generally consistent differences from atmospheric, illumination, and sensor effects, whereas changed pixels have diverse feature differences.
  • SFA minimizes transformed-signal variation to suppress slowly varying unchanged components and highlight quickly varying changed pixels.
  • DSFA uses two deep networks to learn nonlinear projections before SFA, and uses CVA pre-detection to select high-confidence unchanged pixel pairs without labeled training data.

III. METHODOLOGY

DSFA extends linear SFA-based change detection with two symmetric deep networks that project bi-temporal pixels into a complex feature space. SFA then suppresses invariant transformed components, while a loss function enables gradient-based parameter optimization.

  • DSFA addresses the limited feature representation of existing linear SFA methods by introducing deep nonlinear projections.
  • The model accepts pairwise pixels from multi-temporal imagery and contains a Deep Network module plus an SFA constraint.
  • Two symmetric fully connected networks project the original inputs into a complex high-dimensional feature space.
  • The SFA constraint suppresses invariant components and highlights changed components of the transformed features.
  • DSFA formulates a loss function so network parameters can be learned with gradient-based optimization.

A. Formulation

The DSFA formulation maps bi-temporal images through symmetric networks, centers their transformed features, and solves a regularized generalized eigenvalue problem. Minimizing the squared eigenvalues suppresses unchanged-pixel variance and facilitates change detection.

  • DSFA represents bi-temporal images X and Y as matrices with m feature bands and n pixels, then computes symmetric network outputs.
  • The transformed outputs are centralized before calculating covariance matrices for the DSFA optimization.
  • A positive regularization constant r makes the covariance matrices positive definite and invertible for the generalized eigenvalue problem.
  • The DSFA objective minimizes the total squared eigenvalues, using SFA’s smallest-eigenvalue principle to suppress unchanged-pixel variance.
  • The resulting loss is used to obtain network parameters through gradient-based optimization.

B. Optimization

DSFA optimizes symmetric deep networks so SFA can extract invariant components from transformed bi-temporal features. The resulting mapped-feature differences are converted into change intensities and thresholded into binary maps.

  • Optimization: Gradient descent optimizes the deep-network parameters by minimizing the DSFA loss.The loss gradients are computed for both network instances.
  • Optimization: Using unchanged pairwise pixels as training samples makes the learned nonlinear projection target an invariant difference feature space.CVA pre-detection supplies candidate unchanged pixels for training.
  • Optimization: The trained networks transform both temporal images, after which a generalized eigenvalue problem yields the SFA projecting matrix.The transformed features are denoted Xφ and Yφ, and the projecting matrix is wφ.
  • Optimization: Chi-square distance measures mapped-feature differences while accounting for differing feature-band scales.The distance uses the number of feature bands and their statistically estimated variances.
  • Optimization: OTSU or Kmeans thresholding converts the change intensity into the final binary change map.The complete training and map-generation procedure is summarized in Algorithm 1.

IV. EXPERIMENT

The study evaluates DSFA on three multi-temporal remote sensing datasets, including two real-world ETM datasets and a public hyperspectral change-detection dataset.

  • IV. EXPERIMENT: Experiments use two Enhanced Thematic Mapper datasets and one public hyperspectral change-detection dataset.The datasets are used to evaluate DSFA in TensorFlow.

A. Experiment settings

The experiments compare DSFA with conventional, SFA-based, and deep-learning change-detection methods under specified training, thresholding, and regularization settings.

  • Experiment settings: The algorithm description identifies the binary change map D as the output generated from multi-temporal input images I1 and I2.The listed process includes standardization, training, feature mapping, SFA projection, difference calculation, and thresholding.
  • Experiment settings: Algorithm 1 standardizes both input images, uses CVA pre-detection for training samples, and iteratively updates network parameters by gradient descent.The procedure then solves the SFA problem and computes the difference map.
  • Experiment settings: The DSFA regularization parameter is tuned over [10^-8, 10^-1] and selected as 10^-4.Its influence is discussed separately in the paper.
  • Experiment settings: The comparison includes CVA, PCA, MAD, IRMAD, USFA, ISFA, PCANet, and SDPCANet, all treated as unsupervised algorithms.PCA, MAD, SFA-based methods, and patch-based deep models provide different comparison families.
  • Experiment settings: All methods use their complete output feature bands to calculate change intensity.This setting applies across the evaluated algorithms.

B. Experiments on Taizhou ETM dataset

On the Taizhou ETM dataset, DSFA-128-2 generally provides the strongest overall change-detection performance, while visual and threshold-based evaluations reveal differences among DSFA variants and competing methods.

  • B. Experiments on Taizhou ETM dataset: The Taizhou dataset contains 400×400 imagery with sampled changed and unchanged regions shown alongside pseudo-color images and ground truth.The dataset covers Taizhou city and includes 4227 changed pixels in the sampled ground truth.
  • B. Experiments on Taizhou ETM dataset: 4000 unchanged pixels, about 2.5% of all pixels, are randomly selected from CVA pre-detection for DSFA training.Ten independent runs are summed to form the final DSFA change intensity map because of random initialization.
  • B. Experiments on Taizhou ETM dataset: Visually, PCA, ISFA, and DSFA-128-2 produce the strongest Taizhou change-intensity maps among the compared methods.Brighter regions represent larger change probabilities; PCANet and SDPCANet have no intensity maps because they are classification-based.
  • B. Experiments on Taizhou ETM dataset: DSFA-128-2 achieves Kappa 0.9227 and F1 score 0.9372, while outperforming the other methods on overall accuracy.The reported advantage covers both changed and unchanged sampled regions.
  • B. Experiments on Taizhou ETM dataset: With Kmeans thresholding, DSFA-128-2 reaches 97.64% accuracy for both changed and unchanged regions and remains the best overall method.It also has the highest Kappa coefficient and F1 score in that evaluation.
  • B. Experiments on Taizhou ETM dataset: Across traversed thresholds, DSFA-128-2 performs best among DSFA variants, and DSFA results remain close across OTSU and Kmeans thresholding.The best DSFA-based results outperform most other algorithms, excluding CVA and ISFA.
  • B. Experiments on Taizhou ETM dataset: The binary maps show distinct error patterns: MAD has more false positives, while CVA, PCA, SFA methods, and some DSFA variants miss changed regions.PCANet has more false negatives and SDPCANet more false positives than DSFA-128-2 in the cited comparison.

C. Experiments on Nanjing ETM dataset

The Nanjing experiments compare DSFA with conventional, SFA-based, and deep-learning methods using visual maps, thresholded maps, and evaluation tables. DSFA, especially DSFA-128-2, generally provides the strongest separation of changed and unchanged regions and the best overall metrics.

  • Dataset: Nanjing imagery comprises two 6-band remote-sensing images acquired in 2000 and 2002, with sampled changed and unchanged regions for evaluation.The images are 800×800; the sampled changed region contains 2363 pixels and the unchanged region 12393 pixels.
  • Change intensity maps: DSFA-128-2 and DSFA-256-2 produce closely matching intensity maps with strong visual discrimination between changed and unchanged regions.DSFA-128-2 is judged visually best for calculating change intensity, while PCA is close to DSFA-64-2 but separates regions less clearly.
  • Quantitative evaluation: DSFA-128-2 outperforms other algorithms on OA UN, OA, Kappa coefficient, and F1 score when Nanjing results are segmented with OTSU.MAD and IRMAD lead on OA CHG, whereas USFA and ISFA perform well on unchanged pixels but have the lowest OA CHG.
  • Quantitative evaluation: With Kmeans segmentation, DSFA methods retain higher overall accuracies, Kappa values, and F1 scores than competing methods.PCANet-based methods achieve higher OA CHG but perform worse on the other criteria and are described as the second-best group overall.
  • Threshold robustness: DSFA methods remain best across all criteria under the best-threshold evaluation, and their results are close to those from OTSU and Kmeans.This closeness is presented as evidence that DSFA-transformed features discriminate changed and unchanged pixels effectively.
  • Binary change maps: Binary maps show similar DSFA network structures, while MAD and IRMAD generate more false positives and USFA and ISFA more false negatives.CVA and PCA are closer to DSFA but still contain fewer true negatives and more false positives; PCANet and SDPCANet also show higher false-error rates in comparison.

D. Experiments on River dataset

The River experiments evaluate DSFA and competing methods on a high-dimensional hyperspectral dataset using change maps, thresholded results, and best-threshold comparisons. DSFA methods show the strongest overall visual and numerical performance, with DSFA-128-2 leading several metrics.

  • Dataset: The River dataset contains two 198-band images of size 463 × 241, with 12566 changed pixels and 99017 unchanged pixels.The accompanying ground-truth map marks changed regions white and unchanged regions black.
  • Change intensity maps: DSFA-based intensity maps have better discriminability than CVA, PCA, MAD, IRMAD, and USFA, while PCANet and SDPCANet provide no intensity maps.The latter two methods are classification-based, so they are excluded from the intensity-map comparison.
  • Thresholded evaluation: DSFA-based methods achieve better OA UN, OA, Kappa, and F1 score performance in the River evaluation using OTSU or Kmeans.DSFA-128-2 has the best OA, Kappa, and F1 score, while DSFA-64-2 and SDPCANet share the highest OA UN.
  • Best-threshold evaluation: DSFA methods retain the best performance across the best-threshold evaluation, with DSFA-128-2 leading OA, Kappa, and F1 score.DSFA-64-2 and DSFA-256-2 rank second and third on OA and Kappa, respectively, and remain close to DSFA-128-2 on F1 score.
  • Binary change maps: Binary maps show DSFA performing best overall, although its accuracy is lower in the upper-right region where changes are subtle and the background is complex.Competing methods show higher false-positive or false-negative rates in specified regions, while DSFA is reported as best visually and numerically overall.

E. Runtime Analysis

The runtime analysis compares DSFA with IRMAD, ISFA, PCANet, and SDPCANet across three datasets and examines sensitivity to the regularization parameter r. DSFA is computationally acceptable, while its accuracy is stable for sufficiently small r and declines when r exceeds 10^-4.

  • Runtime setup: Runtime comparisons include IRMAD, ISFA, DSFA-128-2, PCANet, and SDPCANet under distinct MATLAB, Python, CPU, thread, and GPU configurations.DSFA-128-2 is measured as DSFA-CPU and DSFA-GPU, with GPU runs using a single NVIDIA 1080Ti card.
  • Runtime comparison: ISFA and IRMAD are fastest overall, followed by DSFA-GPU and DSFA-CPU, while PCANet-based methods are most time consuming.On River, both DSFA variants are faster than IRMAD and ISFA; on Taizhou and Nanjing, DSFA-GPU is close to them and DSFA-CPU remains acceptable.
  • Regularization sensitivity: The experiments use r = 10^-4, and r has little influence on final results when sufficiently small.The regularization parameter is used in the covariance-related equations governing the DSFA model.
  • Regularization sensitivity: When r < 10^-4, accuracy changes are negligible across all three datasets, whereas r > 10^-4 substantially lowers accuracy.The paper attributes the decline to larger r corrupting the characteristic of the covariance matrices.

B. Selection of Training Samples

The River experiment compares training-sample selection strategies for DSFA. CVA-selected unchanged samples perform nearly as well as ground-truth samples, while selecting changed samples performs poorly.

  • Training-sample strategies: CVA selects training samples from unchanged regions identified by CVA change-detection results, whereas Negative, Ground Truth, and Random use different selection sources.The experiment uses DSFA-128-2 on the River dataset.
  • Training-sample strategies: Negative strategy produces a very bad result because training on changed pixel pairs conflicts with the SFA and DSFA objective.The learned projection is inconsistent with the model’s main idea.
  • Training-sample strategies: Random selection is slightly better than CVA and Ground Truth on OA UN but much worse on the other evaluation criteria.Random selection includes changed pixel pairs, which misleads DSFA training.
  • Training-sample strategies: CVA and Ground Truth produce almost identical results, indicating that CVA pre-detection can provide effective training samples without ground-truth labels.The paper notes that ground-truth labeling is usually hard and time consuming.
  • Evaluation context: DSFA is evaluated on two multi-spectral datasets and a public hyperspectral dataset, with visual and quantitative results reported as outperforming other state-of-the-art methods.The comparison includes other SFA-based and deep-network algorithms.
  • Scope: The method currently targets distinguishing changed from unchanged regions in bi-temporal imagery, leaving multi-class change detection for future work.The paper also identifies sequential or video imagery as a future DSFA application.

APPENDIX A DERIVATION OF GRADIENT OF LOSS

The appendix derives the gradient of the DSFA loss with respect to transformed feature data. It expands the loss’s matrix terms, uses symmetry properties, and combines intermediate derivatives into the final gradient.

  • Gradient derivation: The appendix begins by deriving the gradient of L(θ1, θ2) with respect to X̂φ using equations based on prior derivations.The derivation is organized around intermediate matrix quantities and their derivatives.
  • Gradient derivation: The derivation uses the fact that Aφ and Bφ, along with their gradients where stated, are symmetric matrices.These symmetry properties simplify the matrix-gradient expressions.
  • Intermediate derivatives: The derivative ∇B = ∂L(θ1, θ2)/∂Bφ is calculated as an intermediate step in the loss-gradient derivation.This term is later integrated with the other matrix derivative expressions.
  • Intermediate derivatives: The appendix expands Aφ and separately derives the corresponding expression with respect to Bφ.The two matrix components are handled in parallel before being combined.
  • Final gradient: The appendix combines equations (44) and (45) to compute the gradient of L(θ1, θ2) with respect to X̂ijφ.This is presented as the final component of the displayed derivation.
Loading 1812.00645v2…