PaperlayerView exact revision on arXiv

Source-linked AI summary

TINYCD: A (Not So) Deep Learning Model For Change Detection

Andrea Codegoni, Gabriele Lombardi, Alessandro Ferrari

arXiv:2207.13159v2cs.CVcs.LGeess.IV

TL;DR

Industrial change detection needs models with lower size and computational complexity than many state-of-the-art alternatives. TinyCD uses a lightweight Siamese U-Net with low-level feature comparison, space-time mixing, and MAMB, and performs comparably or better while remaining smaller and faster. Its low-level, locally oriented approach is constrained when images have large spatial shifts.

  • Problem

    Industrial change detection requires lower-complexity models because many state-of-the-art models are too large for real-time applications.

  • Method

    TinyCD is a Siamese U-Net-like model that compares low-level features, mixes features across space and time, and uses MAMB for attention-mask refinement.

  • Results

    TinyCD performs comparably or better than current state-of-the-art models while remaining the smallest and fastest model in the reported comparisons.

  • Takeaways & Limitations

    Low-level backbone features can support per-pixel change classification and an effective output mask in the evaluated aerial-image datasets.

  • Takeaways & Limitations

    The low-level local approach may perform worse than more global approaches when the two images undergo large spatial shifts.

Abstract

from arXiv · show

In this paper, we present a lightweight and effective change detection model, called TinyCD. This model has been designed to be faster and smaller than current state-of-the-art change detection models due to industrial needs. Despite being from 13 to 140 times smaller than the compared change detection models, and exposing at least a third of the computational complexity, our model outperforms the current state-of-the-art models by at least $1\%$ on both F1 score and IoU on the LEVIR-CD dataset, and more than $8\%$ on the WHU-CD dataset. To reach these results, TinyCD uses a Siamese U-Net architecture exploiting low-level features in a globally temporal and locally spatial way. In addition, it adopts a new strategy to mix features in the space-time domain both to merge the embeddings obtained from the Siamese backbones, and, coupled with an MLP block, it forms a novel space-semantic attention mechanism, the Mix and Attention Mask Block (MAMB). Source code, models and results are available here: https://github.com/AndreaCodegoni/Tiny_model_4_CD

1 Introduction

TinyCD addresses industrial change-detection constraints by combining a compact architecture with low-level feature comparison and efficient space-time feature mixing. The proposed design targets lower complexity while retaining performance comparable to or better than state-of-the-art models.

  • 1 Introduction: Millions-parameter state-of-the-art models constrain industrial deployment, training time, and hyper-parameter optimization resources.The paper identifies model size as a practical constraint for its industrial application.
  • 1 Introduction: TinyCD investigates lower-complexity change detection while targeting performance comparable to state-of-the-art models.The work is motivated by industrial applications requiring real-time performance and smaller models.
  • 1 Introduction: Low-level features are sufficiently expressive for image comparison, enabling the model to keep its parameter count below 300000.The architecture exploits feature-vector channels so a relatively small backbone can be used.
  • 1 Introduction: A novel feature-mixing strategy computes spatio-temporal correlations between the two images with low computational complexity.The strategy also exploits the semantic ordering of channels produced by a Siamese backbone.
  • 1 Introduction: MAMB uses spatially localized features to compute attention masks during up-sampling and refine low-resolution results.The block provides the paper’s fast attention mechanism for the decoder.
  • 1 Introduction: TinyCD achieves performance comparable to or better than state-of-the-art change-detection models while remaining smaller and faster.The paper reports tests on public and proprietary datasets, highlighting aerial-image results on public datasets.

2 Related works

Prior change-detection methods use CNNs, receptive-field expansions, attention mechanisms, and Transformers to model spatial and temporal information. TinyCD is positioned within this progression by using MLP-based pixel-wise processing and a distinct low-complexity feature-comparison strategy.

  • 2 Related works: CNN-based change detection uses early fusion or Siamese feature fusion to process the two time-separated images.The related work describes concatenating the images for U-Net input or processing them with a Siamese U-Net.
  • 2 Related works: Fixed convolutional receptive fields motivate atrous convolutions, spatial attention, channel-wise attention, and self-attention in prior models.These approaches aim to capture broader image context and reweight informative features.
  • 2 Related works: Transformer-based change detection extracts global spatio-temporal information by processing semantic tokens from Siamese backbone features.TinyCD is described as inspired by this semantic-token approach while pursuing a different lightweight design.
  • 2 Related works: MLP blocks provide an alternative for incorporating spatial information, including through patch processing and spatial-shift operations.The paper adopts MLPs to compute spatial attention maps and the final pixel-wise change mask.

3 Proposed model

TinyCD uses a Siamese U-Net that extracts and compares features from two images, then refines decoder outputs with MAMB-derived pixel-level attention masks. Its design combines low-level backbone features, space-time mixing, and a pixel-wise classifier while keeping the backbone and model lightweight.

  • TinyCD uses Siamese encoders, MAMB and bottleneck mixing, an up-sample decoder, and a pixel-level classifier.
  • 3.1 Model overview: The model extracts paired backbone features at each level, using them both for U-Net outputs and for estimating attention masks.
  • 3.4 Up-sampling decoder with skip connections: The decoder upsamples its input to each skip-connection resolution, applies the attention mask, and passes the result through subsequent decoder layers.
  • 3.5 Pixel level classifier: The final PW-MLP assigns each pixel an anomaly probability, which is thresholded to produce the binary change mask.
  • 3.2 Siamese encoder: Using low-level features permits a relatively small backbone and keeps the total parameter count below 300000.
  • 3.3 Mix and Attention Mask Block: MAMB mixes corresponding Siamese features and generates spatial masks that serve as skip connections for decoder refinement.
  • 3.3.1 Mixing block: The mixing block concatenates semantically corresponding channels from both tensors, then applies grouped convolution to mix spatial and temporal information.
  • 3.3.2 Pixel level mask generator: A pixel-wise MLP processes mixed per-pixel channel information to produce a spatio-temporal mask tensor for attention.

4 Experiment Settings and Results

The experiments evaluate TinyCD on two public aerial building-change datasets because the industrial dataset cannot be shared. LEVIR-CD and WHU-CD use 256 × 256 image-pair patches with specified train, validation, and test partitions.

  • 4.1 Datasets: TinyCD is evaluated on LEVIR-CD and WHU-CD, public aerial building datasets selected because the industrial dataset cannot be shared.
  • 4.1 Datasets: LEVIR-CD contains 637 high-resolution image pairs partitioned into 7120 training, 1024 validation, and 2048 test patch pairs.
  • 4.1 Datasets: WHU-CD contains one large image pair split into 256 × 256 patches, yielding 5947 training, 743 validation, and 744 test pairs.

4.2 Loss function and evaluation metrics

TinyCD frames change detection as pixel-wise binary classification, using BCE loss and reporting standard classification and overlap metrics for the change class.

  • TinyCD outputs a per-pixel change probability and thresholds the output mask at 0.5 to obtain the change mask.
  • Binary Cross Entropy compares the binary ground-truth mask with model predictions.The ground-truth mask uses 0 for unchanged pixels and 1 for changed pixels.
  • The authors use BCE partly to attribute performance improvements to the model rather than to an ad hoc loss function.Additional experiments with other widely adopted loss functions are reported in Appendix B.
  • The evaluation reports Precision, Recall, F1 score, IoU, and Overall Accuracy for the change class.The metrics are computed from true positives, true negatives, false positives, and false negatives.

4.3 Implementation details

TinyCD is implemented with a pretrained EfficientNet-b4 backbone, dataset-specific AdamW settings, cosine annealing, synchronized geometric augmentation, and constrained training resources.

  • TinyCD uses the first four blocks of an ImageNet-pretrained EfficientNet-b4 backbone, with remaining weights initialized randomly.The model was implemented in PyTorch and trained on an NVIDIA GeForce RTX 2060 6GB GPU.
  • AdamW hyperparameters were optimized separately for each dataset, yielding different fixed learning rates and weight decays for LEVIR-CD and WHU-CD.The optimization also evaluated the amsgrad variant using Neural Network Intelligence.
  • Cosine annealing adjusts the learning rate during training without warm restarts.
  • Geometric augmentations are applied jointly to both registered images and the ground-truth mask, while non-geometric augmentations are applied independently to the images.The augmentations include flips, free-degree rotations, Gaussian blur, and random brightness/contrast changes.
  • Training uses batch size 8 for 100 epochs because of limited GPU memory and computational power.

4.4 Comparison with SOTA models

TinyCD is compared with baselines, attention-based models, and Transformers on two building change-detection datasets, where the reported results favor TinyCD despite its lower complexity.

  • TinyCD shows superior performance on the LEVIR-CD and WHU-CD building change-detection datasets.Tables 1 and 2 report the benchmark comparisons, with metrics presented as percentages.
  • TinyCD is 146.50 times smaller than the best model found in the literature, while its LEVIR-CD performance increment is more limited.Table 3 compares parameters, computational complexity, and performance.
  • The authors conclude that low-level information can support high-quality building change detection without global attention mechanisms.They attribute this capability to rich information in individual pixels across resolutions.
  • Qualitative comparison with BIT indicates that both models generally perform well, with differences associated more with missing or hallucinated change regions than region quality.Figure 3 highlights regions with significant differences between ground truth and generated masks.

4.5 Ablation study

The ablations show that the sliced backbone paired with the final PW-MLP performs best on both datasets, while skip connections and the proposed mixing strategy improve model quality with limited complexity.

  • 4.5.1 Backbone dimension and final PW-MLP: The sliced backbone with the final PW-MLP achieves the best performance on both LEVIR-CD and WHU-CD.Backbone slicing and PW-MLP must be coupled for the strongest results; slicing alone performs poorly on LEVIR-CD.
  • 4.5.2 Impact of skip connection with MAMB: Skip connections improve all reported metrics on LEVIR-CD and WHU-CD.The study also visualizes intermediate masks produced through the skip connections.
  • 4.5.3 Comparison with other simple mixing strategy: The proposed mixing strategy effectively generalizes subtraction while adding little model size and complexity.Concatenation plus convolution introduces greater parameter overhead but yields little performance difference.
  • 4.5.4 Comparison with CycleMLP: The proposed MLP use outperforms the CycleMLP alternative on LEVIR-CD and WHU-CD.The paper suggests CycleMLP may be more advantageous in hierarchical architectures that generate pyramid features.

5 Limitations of our work

TinyCD learns domain-specific patterns and relies on spatially registered images, limiting adaptation and potentially reducing performance when images undergo large spatial shifts.

  • 5 Limitations of our work: Domain-specific pattern learning reduces TinyCD’s ability to adapt to new scenarios through fine-tuning.The limitation was observed across the reported experiments.
  • 5 Limitations of our work: When images undergo large spatial shifts, TinyCD’s local approach can perform worse than more global approaches such as vision transformers.The reported datasets and industrial case use spatially registered image pairs.

6 Conclusions and future works

The paper concludes that TinyCD combines low-level feature processing, feature mixing, and PW-MLP classification in a compact change-detection model. On aerial-image benchmarks, it performs comparably to or better than state-of-the-art models while remaining smaller and faster.

  • 6 Conclusions and future works: TinyCD is a tiny convolutional Siamese U-Net that uses low-level features for change detection.Its design is guided by industrial requirements.
  • 6 Conclusions and future works: The mixing block and PW-MLP combine spatial-temporal feature comparison with per-pixel classification to generate change masks.Their composition is referred to as MAMB and enriches features used by the U-Net decoder.
  • 6 Conclusions and future works: TinyCD performs comparably to or better than current state-of-the-art models while remaining smaller and faster.The comparison uses public change-detection datasets containing aerial images acquired at two different times.
  • 6 Conclusions and future works: Future work will investigate MAMB and PW-MLP for anomaly detection, surveillance, and semantic segmentation, alongside models combining local and global features.The latter direction is intended to extend the approach to contexts where global features are important.

A Backbones comparison

Backbone experiments show stable performance across architectures, with EfficientNet variants performing best while other backbones provide comparable results and different size-complexity trade-offs.

  • A Backbones comparison: Backbone performance remains stable across the evaluated architectures, indicating robustness to backbone choice.The comparison was conducted on the LEVIR-CD dataset.
  • A Backbones comparison: EfficientNet backbones achieve the best performance, while other backbone types produce comparable results.The appendix frames the alternatives as flexible with respect to required model size and computational complexity.
  • A Backbones comparison: The comparison excludes Transformer-type backbones because the proposed blocks are local whereas Transformers use a global design.Integrating both philosophies is left for future work.

B Hyperparameters’ tuning

The tuning experiments compare learning rates, weight decay, optimizer strategy, and loss functions across 30 runs, showing that BCE and MSE outperform IoU while BCE provides more stable F1 scores. These results motivate selecting BCE for training.

  • The tuning study evaluates learning rate, weight decay, amsgrad usage, and alternative losses using NNI across 30 experiments.The learning-rate range is 10^-3 to 4 · 10^-3, while weight decay ranges from 10^-2 to 8 · 10^-3.
  • BCE and MSE outperform IoU across the explored hyperparameter combinations on the LEVIR-CD validation set.The appendix also reports that BCE + IoU performs better than IoU but worse than BCE and MSE.
  • BCE has lower F1-score variance across hyperparameter choices, providing an additional motivation for selecting it as the loss function.
Loading 2207.13159v2…