PaperlayerView exact revision on arXiv

Source-linked AI summary

Fruit Ripeness Classification: a Survey

Matteo Rizzo, Matteo Marcuzzo, Alessandro Zangari, Andrea Gasparetto, Andrea Albarelli

arXiv:2212.14441v3cs.CVcs.LG

TL;DR

Fruit ripeness grading is important but remains labor-intensive and difficult because fruit types and ripeness cues vary widely. This survey synthesizes feature descriptors and statistical, machine-learning, and deep-learning approaches, highlighting deep learning’s ability to use raw data without engineered features. It concludes that the field spans diverse methods while facing dataset-size, reproducibility, and task-specificity limitations.

  • Problem

    Fruit ripeness assessment is typically subjective and labor-intensive, while fruit-specific variation makes a general automated technique difficult to select.

  • Method

    The survey reviews ripeness-classification methods, feature descriptors, and statistical, machine-learning, deep-learning, and emerging Transformer approaches.

  • Results

    The reviewed methods include engineered-feature approaches and deep-learning models that process raw data without feature engineering.

  • Takeaways & Limitations

    Automated ripeness classification spans multiple fruit representations and model families, with method choice tied to the available feature representation and data.

  • Takeaways & Limitations

    Deep-learning approaches require sufficiently large, well-labeled datasets, while unreleased data and task-specific datasets limit reproducibility and inference potential.

Abstract

from arXiv · show

Fruit is a key crop in worldwide agriculture feeding millions of people. The standard supply chain of fruit products involves quality checks to guarantee freshness, taste, and, most of all, safety. An important factor that determines fruit quality is its stage of ripening. This is usually manually classified by field experts, making it a labor-intensive and error-prone process. Thus, there is an arising need for automation in fruit ripeness classification. Many automatic methods have been proposed that employ a variety of feature descriptors for the food item to be graded. Machine learning and deep learning techniques dominate the top-performing methods. Furthermore, deep learning can operate on raw data and thus relieve the users from having to compute complex engineered features, which are often crop-specific. In this survey, we review the latest methods proposed in the literature to automatize fruit ripeness classification, highlighting the most common feature descriptors they operate on.

1 Introduction

Fruit ripeness is commercially and nutritionally important, but manual assessment is subjective and labor-intensive. This survey reviews automated approaches, emphasizing feature representations and machine/deep learning methods while noting substantial variation across fruit types.

  • Motivation: Manual ripeness grading is subjective, tedious, time-consuming, and labor-intensive, motivating objective automated assessment.Human grading traditionally relies on visual interpretation and practical experience.
  • Challenges: Fruit-specific variation in shape, color, texture, and other attributes makes it difficult to select one general ripeness-classification technique.Both fruit recognition and downstream ripeness classification can be challenging, including within a single variety such as apples.
  • Motivation: Ripeness affects food quality, harvest timing, storage life, consumer acceptance, and economic value across the fruit supply chain.Over- or under-ripe fruit can reduce retail value, taste, shelf life, appearance, and resource efficiency.
  • Methods: Traditional methods use engineered features, whereas deep learning can process raw data and automatically extract semantic features when sufficiently large datasets are available.Statistical and traditional machine-learning methods require feature engineering, while deep learning reduces that burden.
  • Survey scope: The survey provides a broad overview of ripeness classification methods, with particular focus on recent deep-learning techniques and multiple feature descriptors.It extends earlier surveys by covering advanced deep learning alongside image processing and other feature representations.

2 Preliminaries

Fruit ripeness classification maps appearance descriptors to discrete ripeness classes, whose number and visual criteria vary by fruit. The section also connects classification with biological ripening processes and the limitations of human ground-truth assessment.

  • Problem statement: The classification task learns a function that maps fruit descriptors x to predicted ripeness classes ˆc approximating ground-truth classes c.Prediction quality is assessed through an error notion such as mean square error.
  • Problem statement: Ground-truth ripeness is generally assigned visually by human operators using guidelines such as standardized peel-color charts.Human inspection is subjective, tedious, time-consuming, and labor-intensive, whereas instruments can provide more reproducible measurements.
  • Problem variability: The number and meaning of ripeness classes vary substantially across fruit types, with bananas categorized into four visual stages and dates into five stages.Class definitions can correspond to fruit-specific pigment or appearance changes.
  • Biological background: Ripening involves characteristic changes in fruit color, flavor, texture, and aroma after earlier growth through cell division and elongation.The biological background helps explain why these attributes serve as descriptors for ripeness.
  • Biological background: Harvested fruit can be classified as climacteric or non-climacteric according to respiratory and ethylene behavior during ripening.Climacteric fruit shows a respiration and ethylene peak, whereas non-climacteric fruit does not continue ripening after detachment.

3 Fruit Feature Representations

Fruit-ripeness classifiers depend on representative, diverse features, with color serving as a common descriptor linked to biochemical changes. Colorimeters provide precise measurements, while imaging covers larger areas but depends on acquisition conditions.

  • Feature representation: Effective classification requires feature sets that represent each fruit item and are sufficiently diverse across ripeness classes.The survey introduces feature types used to describe fruit for statistical, machine-learning, and deep-learning models.
  • Color: Color is a primary visual quality cue because ripening commonly changes pigmentation through chlorophyll degradation and increased carotenoid or polyphenol concentrations.These changes link measurable appearance to physical and chemical ripening processes.
  • Color: Colorimeters measure fruit color more precisely than visual assessment and can use standardized spaces such as CIELAB for field measurements.Portable instruments support in-situ collection and have been correlated with ripeness using multivariate analysis.
  • Color imaging: Bi-dimensional color imaging rapidly samples a larger fruit area than colorimeters by converting reflected photons into camera sensor signals.The approach uses CCD or CMOS cameras to obtain spatially broader color information.
  • Color imaging: Color-imaging measurements depend on the device and require homogeneous illumination, which can be difficult to achieve in field conditions.RGB intensities vary with illumination and internal camera characteristics.

3.2 Volatiles

Volatile and spectral descriptors provide complementary routes to ripeness assessment, including reflectance-based indices linked to fruit attributes. However, spectroscopic models face spatial-resolution and field-generalization constraints.

  • Volatiles: Fruit aroma and flavor vary with genetically determined volatile compounds and influence consumer acceptance in commercial markets.Aroma-related measurements therefore represent a potentially relevant ripeness descriptor.
  • Spectroscopy: VNIR spectroscopy measures reflected light from 380 nm to 2500 nm, whose behavior depends on fruit physical and chemical properties linked to ripeness.The method is described as nondestructive and fast for assessing multiple quality attributes.
  • Spectral indices: Spectral indices compact information from visible or infrared wavelengths for relating ripeness stages to fruit attributes.Using one wavelength alone is sensitive to sensor, illumination, and particle-size effects.
  • Spectral indices: The IAD index computes absorption differences around 670 nm and 720 nm and correlates with ripeness in peaches, apricots, and nectarines.Its range was reported as similar across different growing seasons.
  • Spectroscopy: Full spectra or selected indices can describe peel-pigment changes and correlate with internal ripening attributes such as firmness.Spectral representations can therefore extend beyond external color assessment.
  • Limitations: Spectroscopic methods have low spatial resolution, and models trained indoors may perform inconsistently in crops, requiring in-field spectra and environmental analysis.Most reported studies focus on indoor post-harvest maturity assessment.

3.4 Fluorescence

Fluorescence methods assess fruit ripeness through chlorophyll-related measurements. Although fluorimetry is established in laboratories, dark adaptation limits field application.

  • Fluorescence methods use chlorophyll degreening as an indicator correlated with fruit ripeness.Chlorophyll content can be measured with a fluorimetric sensor.
  • PAM fluorometers illuminate fruit with actinic light and measure minimum (F0) and maximum (Fm) emitted fluorescence.
  • The maximum quantum yield is calculated as (Fm −F0)/Fm and can be negatively correlated with ripening stage.This relationship was reported for apples.
  • Fluorimetric methods are popular in laboratories but difficult to apply in-field because samples require dark adaptation.

3.5 Spectral Imaging

Spectral imaging captures wavelength-dependent information associated with fruit ripeness. HSI provides continuous spectral cubes, while MSI samples selected wavelengths and offers a more portable, lower-cost field option.

  • Spectral imaging uses multiple electromagnetic-spectrum bands collected with dedicated spectrometers.A smartphone spectrometer included an app for communicating, plotting, and analyzing spectral data.
  • A smartphone spectrometer achieved stability and wavelength resolution comparable to existing top spectrometers.
  • Hyperspectral Imaging: HSI generates a three-dimensional cube across continuous wavelengths, enabling per-pixel spectra containing absorption and textural information correlated with ripeness.HSI is non-destructive and requires little sample preparation.
  • Multispectral Imaging: MSI collects selected wavelengths rather than scanning the whole range, using systems such as LCTF-coupled sensors or rotating filter wheels.
  • Multispectral Imaging: MSI is described as the most promising in-field option because it combines high-resolution selected-wavelength imaging with lower cost and easier portability than HSI.

4 Data Preprocessing

Spectral data are commonly preprocessed before modeling, but preprocessing effects vary by technique and task. Savitzky–Golay smoothing, in particular, may not improve mango ripeness assessment.

  • Studies apply preprocessing techniques to spectra before modeling fruit classification.
  • Savitzky–Golay is the most frequently used digital smoothing filter and fits low-degree polynomials using linear least squares.
  • Smoothing did not improve mango ripeness assessment compared with other preprocessing methods in one comparison.The study warned researchers and developers against using this technique without evidence of benefit.
  • SNV and MSC are frequently used to correct photon scattering, with MSC linearizing spectra against a reference spectrum.The usual reference is the mean spectrum.

5 Approaches To Classification

The survey organizes fruit ripeness classification methods into statistical, machine-learning, and deep-learning groups. Figure 4 presents a workflow from raw input to grade prediction and contrasts raw-data DL with feature-based classifiers.

  • The survey reviews peer-reviewed fruit ripeness classification papers published between 2014 and 2022.This period was selected as a trade-off between survey depth and breadth.
  • It identifies three main method groups: statistical, machine-learning-based, and deep-learning-based approaches.
  • Figure 4 depicts a standard classification workflow from raw input to output grade prediction.
  • Deep-learning classifiers operate on raw data, whereas other classifiers operate on rich feature representations.

5.1 Statistical- and Machine-Learning-based

Statistical and machine-learning methods classify ripeness using engineered descriptors from color, texture, fluorescence, spectra, and other fruit properties. The reviewed studies show that descriptor and model choices are crop- and task-dependent, with some methods achieving strong classification performance.

  • Color and texture: CIELAB color features characterized banana ripening through color changes, brown-spot development, and image texture across expert-labeled stages.The study measured color changes and brown spots over twelve days and extracted homogeneity, contrast, correlation, and entropy features.
  • Color and texture: 94.3% accuracy was reached for tomato ripeness classification using the difference between R and B values on a proprietary 250-image dataset.
  • Color and texture: HSI-, HSV-, and color-histogram-based methods supported ripeness grouping for pineapples and tomatoes using fuzzy logic and artificial neural networks.HSV color moments included mean, standard deviation, and skewness to describe image color distributions.
  • Color and texture: For cape gooseberries, L*a*b* features with an SVM produced the highest f-measure, while PCA combinations improved performance at increased complexity.The comparison covered four machine-learning techniques and RGB, HSV, and L*a*b* color spaces across 925 samples.

5.2 Deep-Learning-based

Deep-learning methods, especially CNNs and multimodal imaging systems, automate feature extraction and ripeness classification across diverse fruit crops. The reviewed studies emphasize transfer learning, segmentation, lightweight deployment, and combinations of visible and spectral information.

  • Deep-learning architectures: CNN-based image processing extracts semantic features automatically and can provide lightweight ripeness classifiers suitable for lower-tier hardware.A banana model using VGG16 transfer learning and an added multilayer perceptron performed comparably to state-of-the-art CNNs.
  • Crop-specific applications: Deep-learning studies addressed crop-specific tasks including dragon-fruit harvest timing, banana maturity, blueberry segmentation, date-fruit harvesting, and oil-palm mobile classification.These systems used architectures including Mask R-CNN, transfer learning, fine-tuning, and lightweight CNNs.
  • Engineered features with neural models: 100%, 100%, and 99.1% accuracy were obtained for unripe, ripe, and overripe mulberries using ANN classification with correlation-based feature selection.The same ANN with consistency-based feature selection achieved 100%, 98.9%, and 98.3% across the three classes.
  • Multimodal imaging: A multimodal banana model combining RGB values and hyperspectral imaging achieved 98.45% overall accuracy with relatively few samples.
  • Multimodal imaging: Multimodal RGB and hyperspectral CNN variants reached up to 0.90 F1 scores for six-way papaya maturity classification.RGB images captured morphological changes, while hyperspectral data represented spectral signatures across 400–900 nm.

6 Prediction Of Optimal Harvest Time

Optimal harvest-time prediction combines nondestructive ripeness assessment with environmental and spectral information. The reviewed approaches use weather inputs, temperature-aware calibration, and hyperspectral models to estimate crop growth or time to commercial harvest.

  • Environmental prediction: Environmental factors such as temperature, light, and humidity are incorporated into crop models for predicting optimal harvest dates.These models were developed for both in-field and greenhouse prediction settings.
  • Environmental prediction: Two-week weather-forecast temperatures were used as model inputs, creating a potentially applicable approach that could pair with nondestructive ripeness measurements.
  • Temperature effects: Temperature affects near-infrared reflectance nonlinearly, motivating calibration models that compensate for surface temperature across 21–31 Celsius.Studies compared global calibration across temperatures with separate calibration models for individual temperature ranges.
  • Spectral prediction: Hyperspectral tomato models predicted growing stage with a best correlation coefficient of 0.89 using key wavelengths in the 400–2100 nm visible and infrared regions.

7 Perspectives

The review identifies opportunities to improve fruit ripeness classification through newer architectures, better optimization, and stronger datasets. It also highlights interpretability and reproducibility as important open issues for high-stakes food-quality applications.

  • 7 Perspectives: Transformer-based models could improve accuracy and robustness in visual fruit ripeness classification, potentially alongside CNNs.The review also identifies CNN optimization, including layer, filter, parameter, and hyperparameter selection, as an ongoing challenge.
  • 7 Perspectives: CNN- and Transformer-based approaches are constrained by the need for sufficiently large, well-labeled datasets.Dataset preparation is described as time-consuming and effort-intensive because it must address underfitting and overfitting.
  • 7 Perspectives: Unreleased data and task-specific datasets limit reproducibility and the inference potential of trained models.The review calls for releasing code and datasets so models can be evaluated across more benchmarks.
  • 7 Perspectives: Because food-quality assessment is high-stakes, future work should examine model interpretability for developers and end users.Suggested analyses include checking data balance, examining output distributions, and applying established interpretability techniques.

8 Conclusions

The survey synthesizes fruit ripeness classification methods, feature representations, and model families, finding deep learning especially promising while emphasizing its opacity. Because ripeness relates to food quality and safety, interpretability remains an important consideration alongside accuracy.

  • 8 Conclusions: The review provides a broad synthesis of fruit ripeness classification, covering biological processes, feature descriptors, preprocessing, and statistical, ML, and DL models.The surveyed descriptors include color, light spectrum, fluorescence, and spectral imaging.
  • 8 Conclusions: Deep learning avoids labor-intensive feature engineering and achieves state-of-the-art accuracy across varied fruit ripeness tasks.The review identifies pretrained and fine-tuned deep learning models as especially promising.
  • 8 Conclusions: Deep learning models are difficult for humans to interpret, creating a trade-off between predictive accuracy and transparency in food-quality applications.The review notes that whether attention heads provide intelligible explanations remains an open question in explainable AI.
Loading 2212.14441v3…