PaperlayerView exact revision on arXiv

Source-linked AI summary

Synthetic Data Augmentation for Satellite-Based Analysis of Battle-Damaged Agricultural Fields in Ukraine

Marta Sumyk, Oleksandr Kosovan, Iryna Voitsitska

arXiv:2608.16380v1cs.CVcs.AI

TL;DR

Labeled satellite imagery for war-damaged Ukrainian agricultural fields is scarce, limiting classification development. This paper evaluates class-conditional GAN and DDPM augmentation with a Vision Transformer, finding that balanced DDPM samples improve real-test performance, especially for the underrepresented class.

  • Problem

    Scarce, costly, and difficult-to-collect labeled imagery limits satellite-based classification of battle-damaged agricultural fields in Ukraine.

  • Method

    The study trains class-conditional GAN and DDPM models on real imagery, augments Vision Transformer training, and evaluates classification on an exclusively real test set.

  • Results

    0.84 to 0.88 accuracy, 0.67 to 0.81 balanced accuracy, 0.65 to 0.78 macro F1, and 0.41 to 0.69 not-bombed recall followed balanced DDPM augmentation.

  • Takeaways & Limitations

    Targeted synthetic augmentation can improve class-balanced classification more effectively than simply increasing dataset size while preserving the original class imbalance.

  • Takeaways & Limitations

    The small, geographically restricted real dataset limits conclusions about generalization across regions, seasons, soil types, weather conditions, and sensors.

Abstract

from arXiv · show

Monitoring war-induced damage to agricultural land in Ukraine is important for understanding threats to food security, environmental stability, and post-war recovery. However, the development of computer-vision systems for satellite-based damage analysis is limited by the scarcity of labeled imagery, especially for damaged agricultural fields. This work investigates synthetic data augmentation as a method for improving classification under limited and imbalanced training data. We train class-conditional Generative Adversarial Network (GAN) and Denoising Diffusion Probabilistic Model (DDPM) architectures on real satellite images and use them to generate additional bombed and not-bombed agricultural-field samples. The generated images are used only for training augmentation, while all downstream evaluation is performed on an exclusively real test set. A Vision Transformer classifier is trained under multiple real and synthetic data configurations to measure the practical utility of each generative approach. The best configuration, based on balanced DDPM augmentation, improves accuracy from 84\% to 88\%, balanced accuracy from 67\% to 81\%, macro F1 from 65\% to 78\%, and recall for the underrepresented not-bombed class from 41\% to 69\%. These results demonstrate the potential of synthetic satellite imagery for data-scarce geospatial applications in war-affected regions.

1. INTRODUCTION

The study addresses binary satellite-image classification of shelling-related agricultural damage in Ukraine, where labeled data are scarce and difficult to collect. It evaluates class-conditional GAN and DDPM augmentation by testing downstream Vision Transformer performance exclusively on real imagery.

  • Motivation: Labeled satellite imagery is limited across fields, seasons, soil types, weather conditions, and damage patterns, while collection can be expensive, dangerous, or impossible.Real-world data collection is further constrained by security risks, cloud coverage, restricted high-resolution imagery, and labor-intensive crater annotation.
  • Motivation: Synthetic generation may expand training-set diversity beyond rotations and flips by producing combinations of terrain, vegetation, texture, and crater appearance.The study emphasizes that visual plausibility alone does not establish usefulness for downstream classification.
  • Study design: Class-conditional GAN and DDPM models generate bombed and not-bombed imagery separately, which is added to real training data for Vision Transformer evaluation.Both generative models are trained exclusively on real training images.
  • Study design: The work formulates shelling-related agricultural-field damage recognition as a binary satellite-image classification task.The target classes are bombed and not-bombed agricultural fields.
  • Evaluation: The study compares balanced and proportionally doubled synthetic-data augmentation strategies and evaluates practical generalization on an exclusively real test set.Synthetic data are also analyzed using distributional, feature-space, diversity, memorization, and downstream-utility criteria.

2. RELATED WORK

Related work frames synthetic augmentation as a response to limited labeled data, emphasizing that generative samples must be realistic, diverse, class-consistent, and useful downstream. Prior methods motivate comparing class-conditional GAN and DDPM augmentation for shelling-damaged agricultural fields in Ukraine using feature-space analyses and an exclusively real test set.

  • Synthetic augmentation: Generative augmentation creates new semantic combinations beyond conventional transformations such as rotations, flips, crops, and color perturbations.Conventional methods vary existing images, whereas generative approaches approximate the underlying data distribution.
  • Synthetic augmentation: Synthetic samples should be evaluated for realism, diversity, class consistency, memorization, distributional similarity, and downstream task performance.Unrealistic artifacts can create a domain gap, while repeated samples may add little supervision.
  • Class imbalance: Class-targeted generation can improve representation of rare categories more directly than expanding the majority class while preserving the original distribution.The study compares targeted class balancing with proportional expansion.
  • Generative models: Conditional GANs generate specified bombed or not-bombed field patches, but adversarial training can be unstable on small datasets and produce artifacts or mode collapse.GAN inference requires one generator forward pass, while small-data failure modes include oscillation and limited sample variety.
  • Generative models: Diffusion models offer stable optimization and broad sample coverage, but require multiple denoising iterations and can also generate samples from specified classes.These tradeoffs motivate comparing GANs and diffusion models under matched training data, class definitions, augmentation regimes, and evaluation protocols.
  • Evaluation and gap: The study combines KID, feature-space precision and recall, class-conditional distances, likelihoods, duplicate checks, nearest-neighbor retrieval, and classification on an exclusively real test set.This multi-perspective evaluation distinguishes fidelity, coverage, memorization, and downstream utility.
  • Evaluation and gap: Existing work does not directly address class-conditional generation for current Ukrainian satellite imagery of shelling-damaged agricultural fields.The paper fills this gap by comparing GAN and DDPM augmentation on the same real training set and evaluating a Vision Transformer with an exclusively real test set.

3. DATASET

The study uses a filtered subset of natural-color SkySat imagery from Ukraine’s Bakhmut region to classify agricultural-field patches as bombed or not bombed. The dataset is imbalanced, and synthetic images augment only real training data while the fixed real test set remains synthetic-free.

  • Dataset source and imagery: The dataset derives from natural-color satellite imagery collected over agricultural fields in Ukraine’s Bakhmut region for shelling-damage detection.The imagery was acquired by Planet SkySat satellites at approximately 0.5×0.5 m per pixel, making impact craters detectable at patch level.
  • Dataset source and imagery: The study uses a filtered and restructured subset with the original image domain and class definitions but a different train–test composition.Consequently, its image counts differ from those reported in the original thesis.
  • Class definitions: Patches are classified as bombed when they contain visible crater damage and as not bombed when they contain no visible crater-related damage.Original annotations were crater segmentation masks reviewed by the authors and converted into fixed-size patch labels.
  • Dataset split and imbalance: 600 real satellite-image patches comprise the subset, including 470 training images and 130 test images.The class distribution is imbalanced in both splits, with bombed as the majority class.
  • Synthetic-data usage: Only real training images train the conditional GAN and DDPM, while generated samples augment classifier training and the fixed real test set remains synthetic-free.The real test set is unchanged across all experiments and contains no synthetic images.

4. METHODOLOGY

The methodology trains class-conditional GAN and DDPM generators on real satellite imagery, augments the training data under balanced or doubled regimes, and evaluates a Vision Transformer on a fixed real test set. Synthetic quality is assessed with complementary feature-space, distributional, diversity, and memorization metrics.

  • Synthetic augmentation pipeline: The pipeline trains class-conditional GAN and DDPM models on real bombed and not-bombed fields, combines generated samples with real images, and evaluates a Vision Transformer on a fixed real test set.Each generator learns separate conditional distributions for the two classes.
  • Generative models: The GAN conditions generation and discrimination on class labels, uses transposed convolutions and ReLU activations in the generator, and applies weighted sampling to avoid minority-class underrepresentation.Both GAN objectives use binary cross-entropy with logits.
  • Generative models: DDPM generation uses a class-conditional U-Net that predicts noise during forward corruption and iteratively denoises Gaussian noise during reverse sampling.The reverse process remains conditioned on the selected class.
  • Augmentation regimes: 334 additional not-bombed images are required in the balanced regime because the real training split contains 402 bombed and 68 not-bombed images.The doubled regime instead approximately doubles both classes while preserving the original imbalance.
  • Synthetic-image evaluation: Synthetic quality is evaluated through GMM likelihood, class-conditional Mahalanobis distance, KID, feature-space precision and recall, diversity measures, and memorization checks.Higher GMM likelihood and lower Mahalanobis distance indicate closer agreement with real-data feature distributions; KID is primary over FID for limited samples.
  • Classifier evaluation: The Vision Transformer is trained separately for every data configuration using the same architecture, preprocessing, optimizer, stopping criterion, and evaluation split.This keeps the classifier comparison consistent across real and synthetic training configurations.

5. RESULTS

Results indicate that DDPM augmentation provided stronger synthetic-image quality and downstream classification performance than GAN augmentation, especially when correcting class imbalance. The strongest classifier configuration used balanced DDPM augmentation, although the reported synthetic-image quality estimates are provisional and require direct computation.

  • Synthetic-image quality: Provisional synthetic-image quality estimates must be replaced by measurements computed directly from generated images before submission.The estimates were inferred from observed downstream classification trends and are described as plausible draft values.
  • Synthetic-image quality: DDPM samples showed higher GMM log-likelihood, lower class-conditional Mahalanobis distance, and lower KID than GAN samples, suggesting closer alignment with real-image distributions.Lower values are preferred for Mahalanobis distance and KID, while higher values are preferred for GMM log-likelihood.
  • Synthetic-image quality: DDPM achieved higher feature-space precision and recall, higher diversity, and a lower duplicate rate than GAN samples.The GAN pattern was consistent with mild mode collapse or repetition of training-set textures, whereas DDPM suggested broader sample variation with better distributional fidelity.
  • Classification results: 0.84 accuracy was achieved by the real-only baseline, while balanced accuracy, macro F1, and not-bombed recall were 0.67, 0.65, and 0.41.The baseline also had 0.94 recall for the bombed class, indicating that overall accuracy obscured weaker minority-class performance.
  • Classification results: 0.88 accuracy, 0.81 balanced accuracy, 0.78 macro F1, and 0.69 not-bombed recall were achieved with balanced DDPM augmentation.Relative to the real-only baseline, balanced accuracy increased by 0.14, macro F1 by 0.13, and not-bombed recall by 0.28.
  • Classification results: Balanced configurations outperformed corresponding doubled configurations, and DDPM augmentation outperformed GAN augmentation under both regimes.The balanced setting corrected class imbalance, whereas the doubled setting increased both classes proportionally while preserving the original imbalance.

6. DISCUSSION

The discussion finds that synthetic data help most when they target limitations in the real training set, particularly class imbalance. DDPM augmentation outperforms GAN augmentation, while balanced metrics and not-bombed recall better capture practical gains than overall accuracy.

  • Targeted augmentation of the underrepresented class produces larger gains than simply increasing samples while preserving the original class imbalance.
  • DDPM augmentation performs better than GAN augmentation in both tested regimes, possibly because iterative denoising provides greater diversity and more stable feature-distribution coverage.GAN training on small datasets is more susceptible to mode collapse and repeated textures.
  • Feature-space precision, recall, diversity, and nearest-neighbor analyses are needed to verify the proposed explanation for DDPM’s stronger performance.
  • Balanced accuracy, macro F1, and not-bombed recall provide more meaningful practical evidence of synthetic augmentation’s benefit than the smaller change in overall accuracy.The real-only model already achieves high bombed recall but performs poorly on not-bombed images.

7. LIMITATIONS AND FUTURE WORK

The study is limited by geographically narrow data, coarse binary damage labels, possible generative-model memorization, and restricted validation. Future work should broaden evaluation, refine damage analysis, test additional models and data ratios, and clarify system boundaries.

  • Limitations: The real dataset is small and geographically restricted, limiting conclusions about generalization across soil types, seasons, weather, sensors, and Ukrainian regions.All images come from the same broader region and satellite source.
  • Limitations and future work: The binary task labels any patch containing at least one crater as bombed, regardless of crater count, size, or damage severity.Future work should consider crater localization, instance counting, segmentation, and object detection.
  • Limitations and future work: Generative models may reproduce training images or amplify acquisition-specific biases, so memorization evaluation should include nearest-neighbor and embedding-based analysis.Exact duplicate checks alone are insufficient to exclude memorization.
  • Future work: Future experiments should test ResNet and ConvNeXt classifiers, multiple random seeds, several synthetic-data ratios, and imagery from additional regions and acquisition periods.External evaluation would provide stronger evidence of generalization.
  • Limitations: The system supports damage analysis and prioritization but cannot determine whether a field is safe or whether unexploded damage is present.These uses fall outside the proposed system’s stated capability.

8. CONCLUSION

This work evaluated class-conditional GAN and DDPM synthetic augmentation for classifying battle-damaged agricultural fields in Ukraine using real-only training inputs and a fixed real test set. DDPM-based class balancing produced the strongest improvements in class-balanced Vision Transformer performance.

  • Conclusion: Class-conditional GAN and DDPM models generated synthetic samples from real training images, while evaluation used a fixed real test set.The augmentation targeted classification of battle-damaged agricultural fields in Ukraine.
  • Conclusion: 0.88 accuracy versus 0.84, 0.81 balanced accuracy versus 0.67, and 0.78 macro F1 versus 0.65 were achieved with balanced DDPM augmentation.This was the strongest configuration for the Vision Transformer classifier.
  • Conclusion: 0.69 not-bombed recall versus 0.41 resulted from balanced DDPM augmentation, improving performance for the underrepresented class.The results suggest targeted synthetic augmentation can outperform simply increasing dataset size while preserving the original imbalance.
Loading 2608.16380v1…