Source-linked AI summary
Apple Flower Detection using Deep Convolutional Networks
Philipe A. Dias, Amy Tabb, Henry Medeiros
TL;DR
Automated bloom-intensity estimation remains limited by laborious manual inspection and flower-detection methods that are sensitive to illumination and occlusion. The paper fine-tunes a pre-trained CNN for flower-sensitive feature extraction and classification, achieving strong performance across four datasets, including dissimilar unseen conditions. Its scope is bounded by the continued need to address limitations of existing methods and future segmentation strategies.
Problem
Bloom-intensity estimation is still commonly manual and existing automated methods are hindered by variable lighting and occlusion, limiting robust flower identification.
Method
The method fine-tunes a saliency-trained CNN, extracts features from superpixel-based region portraits, reduces their dimensionality, and classifies regions with a pre-trained classifier.
Results
The proposed method significantly outperformed state-of-the-art approaches across four datasets, with optimal recall and precision near 80% even on datasets dissimilar from training sequences.
Takeaways & Limitations
The CNN-based model supports accurate flower identification across different flower species and illumination conditions, including datasets previously unseen by the models.
Takeaways & Limitations
The authors identify semantic segmentation with end-to-end pixel-dense prediction as future work for inputs of arbitrary size.
Abstract
from arXiv · showhide
To optimize fruit production, a portion of the flowers and fruitlets of apple trees must be removed early in the growing season. The proportion to be removed is determined by the bloom intensity, i.e., the number of flowers present in the orchard. Several automated computer vision systems have been proposed to estimate bloom intensity, but their overall performance is still far from satisfactory even in relatively controlled environments. With the goal of devising a technique for flower identification which is robust to clutter and to changes in illumination, this paper presents a method in which a pre-trained convolutional neural network is fine-tuned to become specially sensitive to flowers. Experimental results on a challenging dataset demonstrate that our method significantly outperforms three approaches that represent the state of the art in flower detection, with recall and precision rates higher than $90\%$. Moreover, a performance assessment on three additional datasets previously unseen by the network, which consist of different flower species and were acquired under different conditions, reveals that the proposed method highly surpasses baseline approaches in terms of generalization capability.
1. Introduction
Apple bloom intensity guides early-season thinning, but manual estimation is labor-intensive and error-prone, while existing automated methods struggle with illumination changes and occlusion. The paper proposes a fine-tuned CNN-based detector and evaluates it across four datasets.
- Motivation: Bloom intensity determines how many flowers and fruitlets should be removed early to guide thinning and fruit production.Proper thinning affects fruit size, coloration, taste, firmness, and market value.
- Existing limitations: Manual bloom estimation inspects a random sample of orchard trees and extrapolates individual-tree estimates to the remainder.The process is time-consuming, labor-intensive, uncertain, and prone to errors.
- Existing limitations: Color-thresholding methods are hindered by variable lighting and occlusion from leaves, stems, and other flowers.These limitations reduce applicability outside controlled conditions.
- Proposed approach: The proposed method fine-tunes an existing CNN trained for saliency detection to become particularly sensitive to flowers.Its contributions include a CNN-based algorithm, evaluation on realistic uncontrolled data, and testing on previously unseen datasets.
- Evaluation: The method is evaluated against three baseline comparison methods across four datasets acquired under different conditions.The paper also analyzes generalization on additional datasets previously unseen by the evaluated models.
2. Related Work
Earlier flower-detection systems rely mainly on hand-engineered color and size rules that require controlled acquisition or parameter adjustment. Related fruit-quantification research uses richer machine-learning techniques, motivating the paper’s CNN-based approach for robust flower identification.
- Flower detection: Early flower-detection systems use color thresholding and may require specific daylight times and a black cloth background screen.One reported system achieved an 18% predicted-yield error but was applicable only in that controlled scenario.
- Flower detection: HSL color and size thresholding requires parameter adjustment when illumination, flowering density, or environmental conditions change.The method does not require a background screen but remains sensitive to environmental variation.
- Flower detection: A peach bloom system using multispectral aerial orchard images reported an average detection rate of 84.3% for 20 test images.Its applicability is limited by reliance on color or spectral information.
- Fruit quantification: Fruit-quantification methods employ segmentation, sparse autoencoders, logistic regression, conditional random fields, SVMs, and metadata features.These approaches incorporate richer representations than the color-only methods common in flower detection.
- Deep learning: Deep CNNs learn hierarchical features through stacked convolutional and downsampling layers, but their millions of parameters require large labeled datasets.Data augmentation and fine-tuning are discussed as ways to reduce domain-specific labeling demands.
- Deep learning: CNN-based detection methods such as R-CNN pre-train on large datasets and fine-tune on smaller task-specific datasets.Related agricultural applications include Faster R-CNN fruit detection and CNN-based plant identification.
- Research gap: The paper identifies existing flower methods as condition-specific and proposes fine-tuning a saliency-trained CNN for robustness to clutter, illumination changes, and different flower species.The approach is motivated by the limitations of hand-engineered color and size features.
3. Proposed Approach
The proposed CNN+SVM method detects apple flowers through superpixel-based region proposals, fine-tuned CNN feature extraction, dimensionality reduction, and SVM classification.
- 3. Proposed Approach: The CNN+SVM method performs region proposal generation, fine-tuned CNN feature extraction, and final flower-presence classification.These operations are applied to image regions produced from superpixels.
- 3. Proposed Approach: SLIC groups nearby pixels into perceptually meaningful superpixels that provide candidate regions for flower detection.The approach uses an unsupervised, local-context-based proposal strategy because flowers are often salient against their surroundings.
- 3. Proposed Approach: Each superpixel is enclosed in the smallest square portrait, mean-padded around the region, and resized to 227×227 for CNN processing.The padding uses the average RGB color of the training set.
- 3. Proposed Approach: CNN features are extracted from mean-centered portraits, reduced by PCA, and classified with a pre-trained binary SVM.The selected PCA dimensionality is k = 69, retaining approximately 94% of the original variance while preserving nearly identical performance.
- 3. Proposed Approach: Fine-tuning adapts an existing saliency-detection CNN to flower identification using labeled training portraits, followed by SVM training on reduced CNN features.The network was further tuned using labeled portraits from the training set.
- 3.1. Network fine-tuning and SVM training: AppleA contains 147 naturally illuminated images captured from multiple angles and distances, split into 100 training and 47 validation images.The split produced 91,488 training portraits and 42,430 validation portraits.
- 3.1. Network fine-tuning and SVM training: Because only 4% of labeled samples were positive, positive examples were quadrupled through mirroring across the vertical, horizontal, and both axes.This augmentation addressed the imbalance between flower and non-flower samples.
- 3.2. Comparison Approaches: The comparison methods primarily use HSV color thresholding, with HSV+Bh extending superpixel histograms through an SVM classifier.These baselines represent thresholding-based approaches used for flower or fruit detection.
4. Experiments and Results
Experiments validate the CNN+SVM design, showing strong performance with compact features, improved classification over color-based baselines, and generalization across unseen datasets.
- Experimental design: The optimal CNN+SVM model uses FC6 features reduced to 69 dimensions and classifies regions with an SVM.The experiments evaluated these design choices alongside baseline comparisons and cross-dataset generalization.
- Dimensionality analysis: Positive and negative samples are almost linearly separable even in 2D projections of the original feature space.The learned feature space forms distinguishable clusters for flowers, grass/floor, branches/leaves, and sky.
- Feature analysis: The three feature-classification approaches perform similarly, while SVM variants slightly outperform direct softmax classification.Method C, using earlier-layer FC6 features, was selected because it provides slight increases in optimal F1 score and AUC-PR.
- Portrait analysis: Mean-padded portraits provide the best performance by preventing surrounding flowers from causing false positives in cropped superpixel regions.Mean-padding removes the misleading flower context present when a square crop contains a well-defined neighboring flower.
- Baseline comparison: The CNN+SVM method achieves an optimal F1 score higher than 92% and AUC-PR above 97% on the evaluated dataset.Its hierarchical features incorporate color together with morphological and spatial characteristics, unlike color-only baselines.
- Generalization: On three additional unseen datasets, CNN+SVM produces AUC-PR above 85% for all datasets and outperforms HSV+SVM by a large margin.Baseline performance declines when flower colors or flower-leaf color relationships differ from the training conditions.
5. Conclusion
The proposed CNN-based flower detector combines color and morphological information to improve identification across varying illumination, occlusion, and flower species. The authors also identify segmentation, tracking, localization, and broader transfer learning as future extensions.
- 5. Conclusion: The CNN-based model combines hierarchical color and morphological features, outperforming color-only methods across the evaluated cases.The authors attribute the improvement to features extracted by the CNN.
- 5. Conclusion: Across four datasets, the model achieved recall and precision rates near 80% even on datasets substantially dissimilar from the training sequences.The datasets included different flower species and illumination conditions.
- 5. Conclusion: Future work will investigate end-to-end semantic image segmentation using pixel-dense prediction maps for inputs of arbitrary size.The proposed strategies avoid external computation of region proposals.
- 5. Conclusion: The authors plan to extend the system to flower tracking and localization using probabilistic motion-based approaches.Particle filtering is given as an example for predicting flower locations between frames.
- 5. Conclusion: Additional transfer learning approaches, including affine-transform data augmentation and external datasets, are planned for fruitlet and other-flower detection.These extensions aim to broaden the model’s applicability beyond the current task.