Peduncle Detection of Sweet Pepper for Autonomous Crop Harvesting - Combined Colour and 3D Information

I. INTRODUCTION

• Accurate peduncle estimation can reduce crop and branch damage, preserve storage life and market value, and improve crop detachment success.Field occlusion, changing light, and similar colours make detection difficult.Source: §I · p. 1
• Prior approaches either detected peduncles after harvest or achieved impractically low accuracy; RGB colour alone poorly separates in-field peduncles from green leaves and crops.The paper responds by combining colour and geometry features captured from multiple RGB-D views.Source: §I · p. 1
• The method targets peduncle detection on a 3D reconstructed sweet pepper model, assuming scene models with the peduncle and pepper are already registered.The paper builds on earlier detection and multi-view pose-estimation work.Source: §I · pp. 1–2
• The paper contributes Point Feature Histograms alongside colour information and releases manually annotated 3D pepper and peduncle models with ground truth.The detection system is described as operating across red, green, and mixed sweet peppers.Source: §I · p. 2

II. RELATED WORK/BACKGROUND

• RGB colour classifiers for fruit peduncles generally used inspection chambers with ideal lighting, clean backgrounds, no occlusions, and static imagery, limiting harvesting use.Geometry signatures based on radius, curvature, contour distance, and angle changes were also used to detect peduncles.Source: §II · p. 2
• Sweet-pepper multispectral plant-part classification achieved 40% accuracy, which was deemed insufficient for real-robot use.A related greenhouse harvesting project cut peduncles without detecting them, using each pepper’s centroid.Source: §II · p. 2
• Laser-based studies classified plant organs from geometry, whereas this work uses an affordable consumer RGB-D camera with measurement noise in a field site.One cited laser scan produced a point cloud with 45 um accuracy.Source: §II · p. 2
• Strawberry peduncle detection used a predefined region of interest and colour to locate the fruit–peduncle boundary, then measured peduncle inclination.The passage notes that the boundary is harder to exploit for green sweet peppers than for red strawberries.Source: §II · p. 2

III. METHODOLOGY

• Sweet pepper peduncles can be highly curved or flattened against the fruit, unlike the straight, vertical peduncles of some other crops.Source: §III · p. 2
• The method combines colour cues that separate red peppers from green peduncles with geometry features that distinguish green peppers from their peduncles.A supervised classifier uses the resulting feature representation for detection.Source: §III · p. 2

A. Feature representation

• The feature representation uses colour and geometry shape, with the PFH illustration depicting point-pair angular features and a query point’s influence region.Source: A. Feature representation · p. 2, Figure 3 · p. 3

1) Colour feature (HSV):

• HSV colour features are less affected by brightness variation, but colour alone poorly separates green peppers, peduncles, and other green plant parts.The HSV space represents hue, saturation, and value; RGB dimensions are highly correlated.Source: 1) Colour feature (HSV) · p. 3, 2) Geometry feature (Point feature… · p. 3

2) Geometry feature (Point feature histograms [14]):

• PFHs combine distances and surface normals between a query point and its neighbours; search radii that are too large blur adjacent surfaces, while radii that are too small amplify noise.The paper empirically sets the influence-region and normal-search radii to 0.01 m to capture pepper-body and peduncle curvature.Source: 2) Geometry feature (Point feature… · p. 3, Figure 4 · p. 3

B. Support Vector Machines (SVMs) classifier tuning

• Each 3D point is represented by 36 float features: 3 HSV values and 33 PFH values, concatenated for SVM classification.The resulting feature vector has n × 36 dimensions for n points.Source: B. Support Vector Machines (SVMs)… · p. 3

IV. SYSTEM SETUP AND DATA COLLECTION

• The detection setup uses Harvey’s robotic arm and RGB-D sensor; integrating detection with the custom end-effector for autonomous harvesting is future work.Source: Figure 5 · p. 4
• The pipeline reconstructs a dense point cloud, removes outliers and downsamples it, then extracts PFH and HSV features for RBF-kernel SVM detection.Surface normals are calculated before PFH extraction.Source: §IV · p. 4
• The ROS integration uses C++ and PCL for reconstruction, preprocessing, and feature extraction, while peduncle detection is implemented in Python 2.7.Source: §IV · p. 4

A. Training/testing data collection

• Field trials were conducted over 10 days on a North Queensland farm in a protected cropping facility with relatively even, diffused lighting.Source: A. Training/testing data collection · p. 4
• The RGB-D camera scanned each pepper from multiple viewpoints along a predefined triangular trajectory around a detected red pepper’s centroid.The scan took 15 seconds and reconstructed the target pepper and adjacent green crops.Source: A. Training/testing data collection · p. 4
• 72 sweet peppers were manually annotated; the RGB-D data contain noise attributed partly to the low-cost sensor and surface shades or reflections.Each 3D point is labelled as sweet pepper (0) or peduncle (1).Source: A. Training/testing data collection · p. 4
• The annotated models were split into training and testing sets using 50–50 and 79–21 ratios for system evaluation.Table I is captioned as the 50–50 field-trip model-count table.Source: A. Training/testing data collection · p. 4, Table I · p. 4

V. EXPERIMENTAL EVALUATION AND RESULTS

• AUC of the precision-recall curve is used to evaluate 3D point-segmentation performance.Source: §V · p. 4
• The panoramic facility figure contrasts left and right regions with significantly different brightness.Source: Figure 8 · p. 5
• The 79–21 field-trip table identifies annotated-model counts, while Table III identifies pepper types for each field trip and uses blue for mixed peppers.Source: Table II · p. 5, Table III · p. 5

A. Peduncle detection results

• 0.52 AUC is reported for green peppers; red peppers are easily distinguished, while mixed peppers show an intermediate result from only two test samples.Source: A. Peduncle detection results · p. 5
• 0.73 AUC is consistent for Field Trip 1, while Field Trip 2 ranges from 0.54 to 0.67 AUC, mainly attributed to more challenging green and mixed peppers.Figure 9 presents precision-recall curves and AUC for 50–50 and 79–21 training/testing splits.Source: Figure 9 · p. 5, A. Peduncle detection results · p. 5

1) Qualitative evaluation of detection:

• Red and mixed peppers showed impressive detections, while green peppers had false positives mainly where colour and PFH features poorly separated peppers and peduncles.The paper presents one viewpoint of the 3D model; the accompanying video shows additional views.Source: 1) Qualitative evaluation of detection · p. 5, Figure 11 · p. 6

B. Processing time and parameter tuning

• Parallel processing across quad-core processes achieved a fourfold speed-up over single-process prediction.Predicting 6,094 input points took about 40 seconds before parallelization.Source: B. Processing time and parameter tuning · p. 6, Figure 12 · p. 6

1) Model (Kernel function) selection and parameter tuning:

• 0.971 AUC was the best result for an RBF kernel with gamma=0.01 and C=100 in the red-pepper parameter sweep.The model-selection dataset contained only red peppers and differed from the dataset in Table II.Source: 1) Model (Kernel function) selection… · p. 6
• RBF kernels outperformed linear kernels, and adding PFH to colour improved detection around recall 0.1, although colour remained dominant for red peppers.The feature comparison used HSV alone or HSV+PFH with different kernels.Source: 1) Model (Kernel function) selection… · pp. 6–7, Figure 13 · p. 6

• Harvester integration and assessment of its performance remain outside the study's scope and are ongoing work.The paper characterizes the point-level segmentation results as initial and encouraging.Source: §VI · p. 7
• Stems and leaves with colour and shape similar to peduncles are a false-positive boundary for the trained model.The authors suggest exploring context features based on the geometric relationship between a pepper and its peduncle to reduce these errors.Source: §VI · p. 7

• The field-site detection system uses RGB-D colour and shape features, while PFHs add discriminative power and demonstrate the feasibility of depth information for agricultural applications.Source: §VII · p. 7
• An AUC of 0.71 was achieved for point-level segmentation, with qualitative results described as promising.Source: §VII · p. 7
• Future work includes GPU parallelization, comparing classifiers, collecting more training data across weather conditions, reducing manual annotation effort, and testing other pepper varieties or peduncle types.Source: §VII · p. 7