Fruit Quality and Defect Image Classification with Conditional GAN Data Augmentation

Jordan J. Bird; Chloe M. Barnes; Luis J. Manso; Anikó Ekárt; Diego R. Faria

Source-linked AI summary

Fruit Quality and Defect Image Classification with Conditional GAN Data Augmentation

Jordan J. Bird, Chloe M. Barnes, Luis J. Manso, Anikó Ekárt, Diego R. Faria

arXiv:2104.05647v1Apr 12, 2021cs.CVcs.LGeess.IV

TL;DR

Fruit-quality recognition needs better generalisation because existing results often rely on small, unrepresentative datasets. The paper combines fine-tuned transfer learning with Conditional GAN augmentation and pruning, raising accuracy from 83.77% to 88.75% while retaining 81.16% accuracy at half the original model size.

Problem
Fruit-quality recognition lacks sufficiently representative data for reliable real-world generalisation, because collecting enough labelled fruit images is difficult.
Method
The paper combines fine-tuned VGG16 transfer learning, Conditional GAN generation of labelled fruit images, synthetic-data augmentation, and model pruning.
Results
88.75% recognition accuracy was reached after synthetic augmentation, compared with 83.77% before augmentation, while pruning retained 81.16% accuracy at 50% of the original size.
Takeaways & Limitations
Conditional GANs can generate synthetic healthy and unhealthy fruit images with classifiable characteristics and improve the reported fruit-quality classification pipeline.
Takeaways & Limitations
The comparison uses the best train-test CNN against the Conditional GAN rather than Conditional GANs trained within k-fold cross-validation.

Abstract

from arXiv · show

Contemporary Artificial Intelligence technologies allow for the employment of Computer Vision to discern good crops from bad, providing a step in the pipeline of selecting healthy fruit from undesirable fruit, such as those which are mouldy or gangrenous. State-of-the-art works in the field report high accuracy results on small datasets (<1000 images), which are not representative of the population regarding real-world usage. The goals of this study are to further enable real-world usage by improving generalisation with data augmentation as well as to reduce overfitting and energy usage through model pruning. In this work, we suggest a machine learning pipeline that combines the ideas of fine-tuning, transfer learning, and generative model-based training data augmentation towards improving fruit quality image classification. A linear network topology search is performed to tune a VGG16 lemon quality classification model using a publicly-available dataset of 2690 images. We find that appending a 4096 neuron fully connected layer to the convolutional layers leads to an image classification accuracy of 83.77%. We then train a Conditional Generative Adversarial Network on the training data for 2000 epochs, and it learns to generate relatively realistic images. Grad-CAM analysis of the model trained on real photographs shows that the synthetic images can exhibit classifiable characteristics such as shape, mould, and gangrene. A higher image classification accuracy of 88.75% is then attained by augmenting the training with synthetic images, arguing that Conditional Generative Adversarial Networks have the ability to produce new data to alleviate issues of data scarcity. Finally, model pruning is performed via polynomial decay, where we find that the Conditional GAN-augmented classification network can retain 81.16% classification accuracy when compressed to 50% of its original size.

1 Introduction

The study addresses data scarcity in fruit-quality recognition by combining CNN topology exploration with Conditional GAN augmentation. It reports synthetic images with classifiable defects and improved classification accuracy, while pruning retains substantial accuracy at reduced model size.

The study explores Conditional GAN-based training-data augmentation to address limited datasets for real-world fruit-quality recognition.The contribution targets distinguishing healthy fruit from undesirable fruit such as mouldy or gangrenous fruit.
The authors explore CNN topologies for fruit-quality recognition and implement a Conditional GAN that generates synthetic healthy and unhealthy fruit images.The synthetic-data generation model is made available for future work.
Grad-CAM analysis indicates that synthetic images contain classifiable characteristics including fruit shape, mould, and gangrene.
81.16% classification accuracy was retained when the Conditional GAN-augmented network was compressed to 50% of its original size.

2 Background and Related Work

The background frames fruit-quality recognition as a practical computer-vision task constrained by data scarcity and generalisation demands. It introduces data augmentation, GANs, and Conditional GANs as relevant approaches, with class labels enabling controlled generation of healthy or unhealthy fruit.

2.1 Fruit Quality Recognition: Fruit-quality recognition autonomously scores or classifies fruit from input features such as photographs, supporting machine-based sorting and production efficiency.Camera-and-computer systems are presented as lower-cost alternatives to electronic noses, although electronic noses also perform strongly.
2.2 Data Scarcity and Augmentation: Data scarcity can produce high training accuracy but poor real-world deployment because insufficient population-representative data encourages overfitting.The paper presents augmentation and other methods as alternatives to collecting impractically large datasets.
2.2 Data Scarcity and Augmentation: Data augmentation creates additional training examples by modifying existing data or generating synthetic data, addressing the data requirements of CNN image classification.The paper positions generative models as approaches for alleviating data-scarcity problems across several application areas.
2.3 GAN and Conditional GAN: A GAN uses competing generator and discriminator networks, with the generator creating images and the discriminator distinguishing real from fake inputs through a minimax objective.The displayed GAN objective contains terms for recognizing real images and generated fake images.
2.3 GAN and Conditional GAN: A Conditional GAN extends a GAN with class labels, enabling one generator to produce images from multiple classes such as healthy and unhealthy fruit.The discriminator and generator are conditioned on the class label, allowing controlled generation rather than separate independently trained networks.

3 Method

The method combines lemon-image preprocessing, Conditional GAN augmentation, VGG16-based binary classification, interpretability analysis, and pruning to target practical fruit-quality recognition.

3.1 Data Collection and Preprocessing: 2690 lemon images are converted from COCO annotations into binary healthy or unhealthy labels before training.Images are resized from 1056×1056 to 256×256 pixels to reduce memory demands; mouldy, gangrenous, and dark lemons are unhealthy.
3.2 Data Augmentation via Conditional GAN: A Conditional GAN receives healthy or unhealthy class labels and generates synthetic 256×256 RGB lemon images for training-data augmentation.The generator upsamples an 8×8×3 representation, while the discriminator uses convolutional downsampling and ADAM optimization.
3.2 Data Augmentation via Conditional GAN: Conditional GAN training runs for 2000 epochs with batch size 64 after shorter training produced severe visual glitches and smaller batches stopped learning.Manual inspection motivated extending training from 500 to 2000 epochs; batches below 64 ceased progressing after about 10 epochs.
3.3 Classification, Model Analysis and Pruning: The classifier fine-tunes ImageNet-trained VGG16 by replacing its final layers with an optimized interpretation layer and one sigmoid output neuron.A linear search evaluates interpretation-layer sizes from 8 through 8192 neurons, with early stopping after 10 epochs without further learning.
3.3 Classification, Model Analysis and Pruning: Model analysis uses Grad-CAM for decision explanation, while polynomial-decay pruning tests nine sparsity levels to seek smaller, more practical networks.Pruning is applied to the whole model for 20 epochs, with sparsity ranging from 0.9 to 0.1.

4 Results

The results show that the tuned VGG16 classifier, Conditional GAN-generated images, and pruning each contribute to the proposed fruit-quality classification pipeline. Synthetic augmentation improves classification, while pruning preserves substantial accuracy at half model size.

Non-Augmentation Results: The 4096-neuron VGG16 network achieved 81.99% mean classification accuracy, while its best individual run reached 83.77%.The topology search compared networks with different numbers of interpretation neurons.
Conditional GAN Training: During GAN training, generator loss began at 5.5 and the discriminator losses later became oscillatory; final discriminator losses were 0.013 for real images and 0.005 for fake images.The reported adversarial losses were treated as relative scores rather than direct image-quality measures.
Conditional GAN Training: Conditional GAN training produced realistic class-conditional outputs, although some images resembled potatoes and others showed checkerboarding artifacts.The final-epoch generator was selected because its outputs appeared higher quality than those from the epoch with the lowest observed loss.
Conditional GAN Training: Grad-CAM classified all six synthetic images correctly, focusing on fruit shape for healthy examples and mould or dark patches for unhealthy examples.The analysis used a VGG16 CNN trained only on real photographs, indicating that synthetic images contained classifiable characteristics despite not being used to train the visualizer.
Classification Comparison: 88.75% classification accuracy was achieved by augmenting training with 400 Conditional GAN-generated images, compared with 83.77% for the best individual non-augmented run.All tested augmentation approaches outperformed the vanilla CNN; the weakest increased accuracy to 85.02%.
Pruning: 81.16% classification accuracy was retained when the Conditional GAN-augmented network was compressed to 50% of its original size.The authors frame this as doubling production capacity at a 7.59% accuracy loss.

5 Conclusion and Future Work

The study reports improved lemon-quality recognition through Conditional GAN augmentation and argues that synthetic data can address data scarcity, while pruning reduces model size with limited accuracy loss. Future work concerns GAN evaluation, computational cost, pruning methods, and segmentation for higher-volume real-world use.

5 Conclusion and Future Work: 88.75% recognition accuracy followed augmentation with 400 synthetic data points, up from 83.77% for the convolutional neural network.The conclusion identifies 400 synthetic points as having the largest impact.
5 Conclusion and Future Work: Conditional GAN augmentation is argued to produce new fruit-quality data that alleviates data scarcity in fruit health classification.Grad-CAM analysis indicated that synthetic images contained characteristics such as shape, mould, and gangrene.
5 Conclusion and Future Work: The comparison used the best train-test CNN against one Conditional GAN, motivating k-fold cross-validation with k Conditional GANs for greater scientific accuracy.The authors note that training a Conditional GAN required 17 hours on a leading GPU, limiting near-term viability without better technology.
5 Conclusion and Future Work: Real-world, production-volume testing requires image segmentation to separate fruit from backgrounds before the described preprocessing.A segmentation network applied upstream could enable evaluation on much larger volumes of fruit-production data.