Source-linked AI summary
Fruit recognition from images using deep learning
Horea Mureşan, Mihai Oltean
TL;DR
The paper addresses the need for high-quality fruit-image data and fruit recognition across varied categories. It introduces the Fruits-360 dataset, preprocesses images, and trains neural networks for classification; the experiments report very high accuracy, with RGB-only training performing best in one comparison. The authors identify improving network accuracy and expanding the dataset as future work.
Problem
High-quality datasets are essential for classification, while noisy backgrounds can cause incorrect object classification; fruit categories can also be difficult to differentiate.
Method
The paper constructs the Fruits-360 dataset and trains a deep neural network using processed fruit images for classification.
Results
99.98% or above training accuracy was achieved across tested scenarios, and the model trained only on RGB images obtained the best test-set performance.
Takeaways & Limitations
The dataset and experiments support fruit classification as a starting point for broader object-recognition applications.
Takeaways & Limitations
The authors identify improving neural-network accuracy as future work through further architectural experiments.
Abstract
from arXiv · showhide
In this paper we introduce a new, high-quality, dataset of images containing fruits. We also present the results of some numerical experiment for training a neural network to detect fruits. We discuss the reason why we chose to use fruits in this project by proposing a few applications that could use this kind of neural network.
1 Introduction
The paper introduces Fruits-360, a large fruit-image dataset, and trains a deep neural network for fruit identification. It motivates fruit recognition as a difficult but useful starting point for broader object-classification applications.
- The paper proposes Fruits-360, a dataset of images containing popular fruits.
- Changing an image’s background can cause incorrect object classification when datasets contain noisy backgrounds.
- The authors train a deep neural network to identify fruits as an initial step toward recognizing a wider array of objects.
- Fruit recognition is relevant to autonomous navigation, object modeling, process control, human-robot interaction, store inspection, and fruit harvesting.
- Fruits provide a challenging starting task because visually similar categories, such as oranges and grapefruits, are difficult to differentiate.
2 Related work
Prior work applies handcrafted features, deep neural networks, and synthetic or multimodal data to fruit detection, recognition, counting, and harvesting. These studies address varied settings, including cluttered greenhouses, orchards, and nighttime scenes.
- A greenhouse study recognizes and counts green and red peppers using 28000 images from over 1000 plants.
- A Faster Region-based convolutional network combines RGB and near-infrared images through early or late fusion for fruit detection.
- Fruit-harvesting research also uses Faster Region-based convolutional networks for orchard images containing up to 1500 almonds per image.
- Other studies examine harvesting methods, synthetic agricultural segmentation data, yield prediction, and camera-angle effects on fruit detectability.
- Earlier methods combine fruit color, shape, size, and texture with classifiers such as k-nearest neighbors.
3 Deep learning
The paper reviews deep learning and neural-network architectures used for image recognition, emphasizing convolutional networks and their image-structured processing. It also describes recurrent and deep belief networks as related models.
- Deep learning uses multiple layers of nonlinear processing units to transform inputs into increasingly abstract representations.
- Convolutional neural networks combine convolutional, pooling, ReLU, fully connected, and loss layers while preserving image structure during processing.
- Convolutional layers: Convolutional layers use kernels connected to local receptive fields and stack activation maps as inputs for subsequent layers.
- Convolutional layers: Padding preserves spatial dimensions, while stride controls how far kernels move across the input.
- Pooling reduces spatial dimensions and computation, and can help control overfitting; ReLU increases nonlinearity without reducing network size.
- Recurrent convolutional networks reuse weights to increase effective depth without adding parameters, while deep belief networks can pre-train weights to improve initialization and reduce training time.
4 Fruits-360 data set
Fruits-360 is built from videos of rotating fruits, with backgrounds removed and fruits resized to 100x100 pixels. The resulting dataset contains 90380 single-item images across 131 labels, plus 103 multi-fruit images.
- Images were created by filming fruits rotating on a low-speed motor and extracting frames against a white paper background.The motor rotated at 3 rpm, and each movie lasted 20 seconds.
- A flood-fill algorithm marks edge-connected pixels with sufficiently similar neighboring colors as background, which is then filled white.The color-distance threshold is selected by trial and error for each movie.
- Fruits are scaled to 100x100 pixels because smaller images can make visually similar fruits harder to distinguish.The authors plan to use larger images, but expect longer training times.
- 90380 images span 131 fruit and vegetable labels, with each image containing one item; an additional 103 images contain multiple fruits.The dataset is available on GitHub and Kaggle.
- Table 1 lists image counts for each fruit, treating multiple apple varieties as separate objects and labeling some varieties with digits.
5 TensorFlow library
The paper uses TensorFlow as an open-source data-flow-graph framework for implementing, training, and testing the fruit-recognition network. It describes the key operations and deployment features used to support neural-network computation.
- TensorFlow is an open-source machine-learning framework that represents mathematical operations as graph nodes and multidimensional arrays as tensor-carrying edges.
- The paper uses TensorFlow to implement, train, and test the neural network, and introduces the main methods and data types used.
- TensorFlow supports distributed computation, automatic gradient computation, partial subgraph execution, and device-placement constraints.
- TensorFlow operations: The described convolution operation flattens filters, extracts input patches, and right-multiplies each patch by the filter matrix.
- TensorFlow operations: Convolution and max-pooling operations support either “same” or “valid” padding, determining whether spatial dimensions are preserved or reduced.
- TensorFlow operations: The network uses ReLU activation, computed as max(features, 0), and dropout-like random zeroing that rescales nonzero values to preserve their sum.
6 The structure of the neural network used in experiments
The experiments use a convolutional neural network that processes 100 × 100 RGB images through convolution, ReLU, pooling, fully connected, and softmax layers. Four convolutional stages progressively expand filter counts while pooling reduces spatial dimensions before classification.
- The CNN combines convolutional, ReLU, pooling, fully connected, and loss layers to process image structure before classification.
- The network receives standard RGB images with dimensions 100 × 100 pixels.
- The first convolutional layer applies 16 5 × 5 filters, then 2 × 2 max pooling with stride 2 reduces width and height to 50 pixels.
- The second, third, and fourth convolutional layers apply 32, 64, and 128 5 × 5 filters, respectively, each followed by max pooling.
- Four max-pooling layers reduce representation dimensions by a factor of 16, leaving 7 × 7 × 16 inputs for the first fully connected layer.
- The second fully connected layer maps 1024 inputs to 256 outputs, followed by a softmax layer whose output count equals the number of classes.
7 Numerical experiments
The experiments evaluate the CNN on Fruits-360 using different preprocessing and augmentation scenarios and multiple network configurations. Training accuracy is very high, while test performance varies with configuration and input processing, with rapid early training gains.
- The experiments use 90,380 images: 67,692 for training and 22,688 for testing; 103 multi-fruit images are excluded.
- The study compares RGB, grayscale, HSV, merged HSV-grayscale, and augmented preprocessing scenarios using randomized image generation and augmentation.
- The most extensive scenario applies hue and saturation changes, horizontal and vertical flips, HSV conversion, grayscale conversion, and channel merging.
- 99.98% or above training accuracy was achieved across preprocessing scenarios, while RGB-only training produced the best test-set performance.
- All tested network configurations achieved perfect training accuracy, but configuration 1 performed best on the test set; configurations 6, 7, and 8 were below 98%.
- Accuracy exceeded 90% within the first 5 epochs, after which improvements remained small for the rest of the 25-epoch training period.
8 Conclusions and further work
The paper concludes by presenting its fruit-image database and TensorFlow classification experiments, while identifying accuracy improvement and dataset expansion as future priorities.
- The paper describes a new database of fruit images and numerical experiments using TensorFlow to classify images by content.
- Improving the neural network’s accuracy is identified as a main future objective through further experimentation with network structure.Proposed changes include modifying layers or replacing fully connected layers with convolutional layers.
- Replacing all layers with convolutional ones would increase the number of network parameters.
- A planned application would photograph fruits and label them automatically on a mobile device.
- The dataset is planned to expand with additional fruits, including items not used in most related papers.The authors note that this expansion is time consuming.
Appendix
The appendix documents the project’s source code, configuration, data generators, image preprocessing, model interface, evaluation outputs, and visualization routines.
- The source code and project structure for the numerical experiment are available on GitHub.
- The evaluation code reports train and test accuracy and loss, saves classification reports and confusion matrices, and plots accuracy and loss histories.
- Training data are augmented with random flips and altered hue or saturation to simulate variable lighting conditions.The test generator uses the test set without augmentation.
- The data-generation routine creates training, validation, and test generators, using validation data to monitor accuracy and loss during training.The test set is used to calculate final accuracy and loss after training.
- A custom layer converts RGB images to HSV and grayscale and concatenates the results into a 100 x 100 x 4 input.