PaperlayerView exact revision on arXiv

Source-linked AI summary

LeafGAN: An Effective Data Augmentation Method for Practical Plant Disease Diagnosis

Quan Huu Cap, Hiroyuki Uga, Satoshi Kagiwada, Hitoshi Iyatomi

arXiv:2002.10100v2cs.CV

TL;DR

Plant disease diagnosis systems can overfit and lose performance on images from new environments, while collecting diverse, labeled disease images is costly. LeafGAN generates diseased images from healthy ones by guiding image translation toward leaf regions and preserving varied backgrounds. On unseen cucumber-disease images, LeafGAN augmentation improved average diagnostic performance by 7.4%, compared with 0.7% for CycleGAN augmentation.

  • Problem

    Plant disease diagnosis systems face overfitting and reduced performance in new environments, while practical disease datasets are costly, imbalanced, and limited in background diversity.

  • Method

    LeafGAN combines CycleGAN with the weakly supervised, label-free LFLSeg module to transform leaf regions from healthy images into disease images while preserving backgrounds.

  • Results

    7.4% average diagnostic-performance improvement was achieved on unseen images from other farms with LeafGAN augmentation, compared with 0.7% for CycleGAN augmentation.

  • Takeaways & Limitations

    LeafGAN provides a data augmentation tool that improves generated-image quality and the robustness of plant disease diagnosis against unseen data.

  • Takeaways & Limitations

    LFLSeg can misclassify partial leaves under different shooting distances, and LeafGAN or CycleGAN may sometimes transform color rather than disease symptoms.

Abstract

from arXiv · show

Many applications for the automated diagnosis of plant disease have been developed based on the success of deep learning techniques. However, these applications often suffer from overfitting, and the diagnostic performance is drastically decreased when used on test datasets from new environments. In this paper, we propose LeafGAN, a novel image-to-image translation system with own attention mechanism. LeafGAN generates a wide variety of diseased images via transformation from healthy images, as a data augmentation tool for improving the performance of plant disease diagnosis. Thanks to its own attention mechanism, our model can transform only relevant areas from images with a variety of backgrounds, thus enriching the versatility of the training images. Experiments with five-class cucumber disease classification show that data augmentation with vanilla CycleGAN cannot help to improve the generalization, i.e., disease diagnostic performance increased by only 0.7% from the baseline. In contrast, LeafGAN boosted the diagnostic performance by 7.4%. We also visually confirmed the generated images by our LeafGAN were much better quality and more convincing than those generated by vanilla CycleGAN. The code is available publicly at: https://github.com/IyatomiLab/LeafGAN.

I. INTRODUCTION

Plant disease diagnosis systems often lose accuracy in new environments because practical images contain background variation, limited disease data, class imbalance, and overfitting risks. LeafGAN addresses these challenges by generating disease images from healthy leaves while focusing translation on leaf regions and preserving diverse backgrounds.

  • Disease diagnosis is important for food security and agroecosystem sustainability, but practical deployment remains challenging.
  • 31% accuracy and about 33% success rates were reported when systems trained on laboratory images were tested on on-site or cultivation images.Laboratory datasets commonly use cropped leaves and uniform backgrounds, unlike field conditions.
  • Practical disease datasets require costly biological labeling, are often class-imbalanced, and are vulnerable to overfitting from limited background diversity.Models may respond to backgrounds rather than diagnostic leaf regions when image classes correlate with similar backgrounds.
  • Transforming diverse healthy images into disease cases could expand disease-data diversity while reducing labeling costs.
  • LeafGAN uses a segmentation-guided image-to-image translation system to transform diagnostically relevant leaf regions while preserving varied backgrounds.Its LFLSeg module provides leaf-region guidance without expensive masking data.
  • 7.4% average diagnostic-performance improvement was obtained with LeafGAN augmentation on unseen images from other farms, versus 0.7% with CycleGAN.

II. PROPOSED METHOD - LEAFGAN

LeafGAN extends CycleGAN with label-free leaf segmentation to transform relevant leaf regions while preserving image backgrounds. Its attention mechanism guides disease-image generation toward leaf areas rather than backgrounds.

  • LeafGAN combines CycleGAN with the LFLSeg module to guide transformations toward relevant leaf regions while preserving backgrounds.LFLSeg produces leaf-area masks used to focus the translation process.
  • LeafGAN maps healthy images X to arbitrary target disease images Y and simultaneously learns the reverse mapping.The two mappings use corresponding discriminators to distinguish generated images from real samples in each domain.
  • The discriminators receive masked leaf images rather than complete images, so adversarial training emphasizes leaf regions during generation.For X→Y, masked generated and target images are supplied to DY; the reverse transformation is symmetric.
  • LFLSeg uses heatmaps from full-leaf classification to create binary masks identifying regions relevant to the full-leaf decision.The heatmap comparison is based on contribution to the full-leaf class decision, with warmer regions indicating greater contribution.

A. Label-free leaf segmentation module (LFLSeg)

LFLSeg is a weakly supervised, label-free segmentation module designed to identify dense interior leaf regions without pixel-level annotations. It uses full-leaf, partial-leaf, and non-leaf classification with Grad-CAM-derived masks.

  • A. Label-free leaf segmentation module (LFLSeg): LFLSeg is a weakly supervised label-free module that helps classifiers learn dense and interior leaf regions implicitly.Its simple CNN backbone avoids the pixel-level annotation requirement associated with conventional supervised segmentation.
  • A. Label-free leaf segmentation module (LFLSeg): The module classifies objects as full leaf, partial leaf, or non-leaf to support leaf-region segmentation.Full-leaf images contain one complete leaf, partial-leaf images contain part of one, and non-leaf images contain no leaf part.
  • A. Label-free leaf segmentation module (LFLSeg): Grad-CAM converts the full-leaf classifier’s pixel-contribution heatmap into a binary mask after thresholding at δ.The heatmap represents each pixel’s contribution to the final full-leaf decision.
  • A. Label-free leaf segmentation module (LFLSeg): Adding the partial-leaf class encourages LFLSeg to cover the whole leaf instead of only small discriminative regions.Models trained with partial-leaf data focus on the full shape, whereas models without it focus on scattered regions.

B. Loss functions for LeafGAN

LeafGAN defines CycleGAN-based adversarial and cycle-consistency losses and adds a background similarity loss. The added term keeps generated backgrounds close to those of the source images while disease-related leaf content is transformed.

  • B. Loss functions for LeafGAN: LeafGAN uses adversarial losses for the mappings G: X→Y and F: Y→X, following the CycleGAN framework.Each mapping has a discriminator that distinguishes masked generated images from real images in the corresponding target domain.
  • B. Loss functions for LeafGAN: The same LFLSeg module segments inputs from both domains, and cycle consistency constrains reconstructed images to remain close to their sources.The cycle-consistency loss is defined for the two-way mapping process.
  • B. Loss functions for LeafGAN: LeafGAN adds background similarity loss to minimize the L1 distance between generated and original source backgrounds.Backgrounds are obtained using the inverted leaf mask, 1−S, and element-wise multiplication with the input image.
  • B. Loss functions for LeafGAN: The final objective balances the different loss terms through a coefficient λ.The supplied text identifies λ as controlling the balance among loss components.

A. Cucumber diseases dataset

The study uses cucumber leaf images collected across multiple Japanese locations from 2015–2019. The datasets contain healthy leaves and leaves affected by three diseases, divided into exclusive Datasets A and B.

  • A. Cucumber diseases dataset: Cucumber leaf images were collected from multiple locations in Japan during 2015–2019.Each image contains a single cucumber leaf roughly centered against a varied background.
  • A. Cucumber diseases dataset: The dataset includes healthy leaves and leaves infected with MYSV, brown spot, or powdery mildew.The disease categories are Melon yellow spot virus (MYSV), brown spot, and powdery mildew.
  • A. Cucumber diseases dataset: The images were divided into exclusive Datasets A and B for the study.Images in the two datasets were not shared.

B. Training the LFLSeg module

LFLSeg was trained as a label-free leaf segmentation module, while LeafGAN and CycleGAN translation models were trained to generate diseased images from healthy leaves for classifier augmentation.

  • LFLSeg training: LFLSeg training used full-leaf, partial-leaf, and non-leaf image classes.Partial-leaf examples were created by overlapping crops, while non-leaf examples came from ImageNet.
  • LFLSeg training: 216,000 images formed the LFLSeg dataset, split randomly into 70% training and 30% testing sets.Training used momentum optimization with mini-batches of 128 for up to 30 epochs.
  • Disease translation models: LeafGAN and CycleGAN each trained healthy↔MYSV, healthy↔brown spot, and healthy↔powdery mildew translation models.At test time, only healthy→diseased translation was used to generate augmentation images.
  • Disease translation models: 717 healthy validation images per disease type were translated into new disease images for classifier augmentation.LeafGAN used an LFLSeg segmentation threshold of δ = 0.35, and both translation models trained for 200 epochs.
  • LFLSeg training: LFLSeg heatmaps provided segmentation masks without pixel-level labels.These masks were used as the segmentation outputs of the module.

D. Training the disease classification models

The study compared baseline disease classifiers with versions augmented by CycleGAN or LeafGAN images, using pretrained ResNet-101 models and standard flip augmentation.

  • Evaluation: The evaluation compared classifiers trained with and without generated images to assess disease-diagnosis generality on an unseen dataset.The comparison focused on performance differences between augmentation strategies.
  • Classifier setup: Three classifiers were trained: baseline, baseline+CycleGAN, and baseline+LeafGAN.The latter two added disease images generated by their respective translation models.
  • Classifier setup: All classifiers were fine-tuned from pretrained ResNet-101 models with horizontal and vertical flips during training.The classifiers used SGD momentum with mini-batches of 128 and trained for 30 epochs.
  • Evaluation: Table II reports diagnostic performance for the three classifiers on unseen Dataset B.The table compares the baseline, CycleGAN-augmented, and LeafGAN-augmented models.

IV. RESULTS

LFLSeg achieved high three-class validation accuracy and worked on complex in-field backgrounds, but its segmentation remained weaker for overlapping leaves and than a pixel-labeled comparator.

  • Segmentation performance: 99.8% accuracy was achieved for classifying full leaf, partial leaf, and non-leaf images on Dataset A’s validation set.Figure 4 illustrates heatmaps and corresponding segmented results for full-leaf examples.
  • Segmentation performance: LFLSeg worked on different in-field images with complex backgrounds.The reported examples show that the module could segment leaves outside uniform laboratory settings.
  • Segmentation limitations: LFLSeg failed to correctly segment leaf areas when images contained multiple and overlapping leaves.The study assumes disease-classifier inputs contain a single leaf image, limiting the practical impact in this experiment.
  • Segmentation comparison: 83.9% F1-score was achieved by LFLSeg versus 98.1% by the AOP network on 1,000 full-leaf images.LFLSeg required no masking training data, whereas the comparison involved pixel-level labeled training images.

B. Results from disease translation models

LeafGAN generated more realistic diseased images than CycleGAN by focusing translation on leaf regions and preserving original backgrounds, while improving classification on unseen data.

  • Image quality: LeafGAN focused translation on leaf regions, whereas CycleGAN transformed the whole image, including backgrounds, producing less plausible results.The attention mechanism directed LeafGAN toward disease-relevant regions rather than background areas.
  • Classification results: 97.2%, 97.7%, and 97.9% were the Dataset A validation accuracies for baseline, baseline+CycleGAN, and baseline+LeafGAN, respectively.These values describe the three classifiers before testing on unseen Dataset B.
  • Classification results: 71.3% was the baseline average diagnostic accuracy on unseen Dataset B despite training on 2,000 images per class.Dataset A and Dataset B were completely different, producing a large validation-to-test performance gap.
  • Image quality: LeafGAN preserved the original background, making generated images more realistic than CycleGAN outputs.The segmented leaves shown in Figure 5 came from the LFLSeg module.
  • Classification results: 78.7% was the best average classification accuracy among the three classifiers on unseen Dataset B after augmentation.The passage attributes this best result to the LeafGAN models.
  • Failure cases: Figure 6 shows powdery mildew symptoms across stages alongside failure cases for healthy-to-powdery-mildew translation by both models.The figure places symptom stages in the left column and translation failures in later columns.

V. DISCUSSION

LeafGAN improved disease-classification augmentation by focusing transformations on leaf regions, producing more varied and visually convincing images than CycleGAN. However, segmentation and disease-stage diversity remained practical limitations.

  • LeafGAN generated more persuasive and realistic images than CycleGAN in visual assessment.CycleGAN transformed whole images, making generated backgrounds resemble the target disease dataset.
  • 7.4% improvement in diagnostic performance was achieved on unseen Dataset B using LeafGAN augmentation.The authors attribute this to increased stochastic variety and symptoms appearing in relevant regions.
  • LFLSeg guided LeafGAN toward transforming leaf regions and remained sufficiently effective despite imperfect segmentation.The partial leaf class helped guide attention to the relevant area.
  • Only +0.7% improvement over baseline resulted from augmentation with vanilla CycleGAN.Its lack of attention can place generated symptoms in surrounding background areas.
  • Two limitations remained: LFLSeg may misclassify partial leaves under different shooting distances, and models sometimes changed color instead of disease symptoms.The latter issue was associated with limited representation of later-stage powdery mildew images in the training dataset.
  • LeafGAN is presented as a data augmentation tool for improving robustness and overall automated crop disease diagnosis performance.Its design combines leaf-area segmentation with transformations across diverse backgrounds.
Loading 2002.10100v2…