PaperlayerView exact revision on arXiv

Source-linked AI summary

MinneApple: A Benchmark Dataset for Apple Detection and Segmentation

Nicolai Häni, Pravakar Roy, Volkan Isler

arXiv:1909.06441v2cs.CV

TL;DR

Fruit detection, segmentation, and counting in orchards lack a unified dataset for direct comparison, despite their importance in agricultural automation. MinneApple provides diverse, high-resolution orchard imagery with instance annotations and patch-counting data, together with benchmark analyses across detection, segmentation, counting, and yield estimation. The dataset contains over 41,000 annotated apple instances in 1000 images and is intended to support comparisons on challenging orchard scenes.

  • Problem

    Existing fruit datasets and benchmarks are limited in size or variety, making direct comparison difficult for orchard detection, segmentation, and counting.

  • Method

    The paper constructs MinneApple from high-resolution orchard imagery with polygonal instance annotations, plus a second dataset for patch-based counting of clustered fruits.

  • Results

    MinneApple contains over 41,000 labeled apple instances in 1000 images and includes baseline analyses for detection, segmentation, counting, and yield estimation.

  • Takeaways & Limitations

    The dataset supports evaluation and comparison of methods for small-object detection, segmentation, and counting in diverse, cluttered orchard environments.

Abstract

from arXiv · show

In this work, we present a new dataset to advance the state-of-the-art in fruit detection, segmentation, and counting in orchard environments. While there has been significant recent interest in solving these problems, the lack of a unified dataset has made it difficult to compare results. We hope to enable direct comparisons by providing a large variety of high-resolution images acquired in orchards, together with human annotations of the fruit on trees. The fruits are labeled using polygonal masks for each object instance to aid in precise object detection, localization, and segmentation. Additionally, we provide data for patch-based counting of clustered fruits. Our dataset contains over 41, 000 annotated object instances in 1000 images. We present a detailed overview of the dataset together with baseline performance analysis for bounding box detection, segmentation, and fruit counting as well as representative results for yield estimation. We make this dataset publicly available and host a CodaLab challenge to encourage comparison of results on a common dataset. To download the data and learn more about MinneApple please see the project website: http://rsn.cs.umn.edu/index.php/MinneApple. Up to date information is available online.

I. INTRODUCTION

Fruit detection, counting, and localization support agricultural automation but remain difficult in orchards because of changing illumination, fruit appearance, and occlusion. MinneApple addresses limited direct comparison by introducing a benchmark dataset with diverse orchard images and annotations.

  • Detection, counting, and localization support resource management, harvest decisions, and automated fruit picking.
  • Outdoor orchard vision is challenged by varying illumination, fruit appearance, and occlusion by foliage, branches, or other fruits.
  • MinneApple introduces a dataset and benchmark evaluation suite for apple detection, segmentation, and counting in orchard settings.
  • 1000 images contain over 41,000 labeled apple instances, with small objects and 1 to 120 objects per image.
  • The dataset spans multiple fruit varieties over two years to increase diversity for orchard-based evaluation.

II. RELATED WORK

Prior fruit-vision research uses varied sensors, datasets, and algorithms for detection and yield-related tasks. Existing studies include small or specialized datasets and a range of detection approaches, motivating broader benchmark resources.

  • Computer-vision research has used cameras, RGB-NIR combinations, object-detection networks, Faster RCNN, and fully convolutional networks for fruit analysis.
  • Earlier fruit-detection methods often relied on static color thresholds, sometimes supplemented with thermal or NIR sensors.
  • Prior datasets included 122 indoor images, roughly 1000 apple image crops, and 308×202-pixel images with circular fruit annotations.

B. Fruit counting

Fruit-counting datasets have generally been small and limited in variety because annotation is costly. MinneApple addresses these issues with full-resolution orchard data and splits designed to test generalization across tree rows and years.

  • Annotation costs have led researchers to use small fruit detection and counting datasets with limited variation.
  • MinneApple’s listed dataset entry is outdoor, public, and includes 64,597 training images, 5,764 test images, and six scenes.
  • Image crops can increase dataset size artificially without increasing variation, leaving methods prone to overfitting.
  • Splitting whole datasets into training and testing sets limits analysis of algorithm generalization.
  • MinneApple uses full-resolution images and separates training and testing data by tree rows and years.

III. IMAGE COLLECTION

The dataset was collected as orchard video at the University of Minnesota’s Horticultural Research Center across multiple years and tree-row conditions. Sampling and split choices were designed to capture varied fruit, lighting, and orchard settings.

  • Data were collected at the University of Minnesota’s Horticultural Research Center between June 2015 and September 2016.
  • Video was recorded horizontally along tree rows at approximately 1 m/s, with slow movement used to mitigate motion blur.
  • Seventeen datasets were collected over two years, including fruits of different colors and ripening stages.
  • Data were captured from sunny and shady sides of tree rows and across multiple days to vary illumination conditions.
  • Training images were sampled from six tree rows in 2015, while evaluation videos came from four orchard sections collected in 2016.

B. Counting Datasets

The counting datasets provide annotated image patches for red and green apples, including positive patches with manual counts and negative patches without apples, plus diverse test sequences.

  • Training Sets: 13,000 image patches were manually annotated with ground-truth counts for green and red apples, with 4,500 random apple-free patches added as negatives.Both training datasets were acquired in 2015 from the sunny side of tree rows.
  • Dataset Format: Detection and segmentation use instance masks, whereas the counting dataset uses image patches paired with corresponding ground-truth counts.
  • Test Sets: 2,874 test patches span red, green, mixed-color, and low-resolution fruit sequences to evaluate counting generalization.The low-resolution sequence was acquired from farther away.

IV. IMAGE ANNOTATION

MinneApple combines polygon-based instance annotation with dataset designs that expose dense scenes, small fruits, and varied object sizes across orchard imagery.

  • Instance Annotation: Fruit and tree-trunk instances were labeled with polygons, while ground and background fruits were excluded and only fully or partially visible fruits were annotated.Each scene object was categorized as fruit or tree trunk.
  • Annotation Procedure: The annotation process was labor-intensive because images contain many small, heavily occluded fruit instances, requiring up to 30 minutes per image.The reported effort was roughly 18 work-hours per 1,000 instances.
  • Counting Annotation: Patch-based counting annotations assign each patch a single integer count, with two workers and a third validator resolving disagreements.The small patch resolution and large annotation volume made the task error-prone.
  • Dataset Statistics: MinneApple contains 1.5 categories and 41.2 instances per image on average, with counts ranging from 1 to 120 objects.COCO averages 7.7 instances per image, while ImageNet and PASCAL VOC each average fewer than three.
  • Object Size: The dataset is dominated by small objects averaging 40×40 pixels in 1280×720 images, occupying only 0.17% of the original image area.The passage states that MinneApple contains almost exclusively small instances.

VI. ALGORITHMIC ANALYSIS

The study runs state-of-the-art algorithms across detection, segmentation, and counting to establish a shared baseline for future comparisons.

  • Benchmark Scope: State-of-the-art algorithms are evaluated separately on object detection, segmentation, and counting tasks to establish a common baseline.

A. Detection and Segmentation Baselines

The benchmark evaluates established detection and segmentation models under common protocols, finding that whole-image detection is stronger than tiled processing and pretrained UNet leads segmentation overall.

  • Experimental Setup: 600 images were used for training, 30 for validation, and 331 for testing each algorithm.
  • Detection Evaluation: Detection uses AP@0.5:0.05:0.95 as the main metric, supplemented by AP@0.5, AP@0.75, and size-specific scores.The evaluation separates small, medium, and large objects.
  • Detection Baselines: The detection baselines include Faster RCNN, tiled Faster RCNN, and Mask RCNN, using ResNet-50 backbones and pretrained weights.Tiled Faster RCNN splits images into overlapping 500 × 500 pixel chunks and aggregates detections with non-maximum suppression.
  • Detection Results: Whole-image detectors outperform tiled processing, Faster RCNN slightly exceeds Mask RCNN, and all detectors struggle with smaller objects.The authors hypothesize that final non-maximum suppression causes the tiled approach to underperform.
  • Segmentation Evaluation: Semantic segmentation is evaluated with IoU, class IoU, pixel accuracy, and class accuracy to avoid favoring bounding-box methods.
  • Segmentation Results: UNet with pretrained weights outperforms the other segmentation methods overall, although user-supervised GMM achieves higher class IoU.The authors report that pretrained weights improve performance slightly and suggest increased training data explains the improved deep-learning results.

B. Patch-based Fruit Counting Baselines

The paper provides a patch-based fruit-counting dataset and compares unsupervised GMM and CNN baselines across test sets. The CNN outperforms GMM overall, but its performance varies considerably by dataset.

  • The benchmark evaluates patch-based counting of clustered fruits using GMM and CNN approaches.GMM fits Gaussian mixtures to previously segmented images, while CNN classifies fruits into six classes using a ResNet50 backbone.
  • The ResNet50 CNN outperforms the GMM model on all test sets.
  • CNN counting performance varies considerably across datasets with different apple colors, viewpoints, and acquisition distances.The best CNN performance occurs on test dataset 3, containing green apples; dataset 4 contains red apples acquired from farther away.
  • GMM performs best on test dataset 1 because that dataset is closest to its training data.

C. Yield estimation

Yield estimation combines fruit detection and counting with tracking and tree-row merging to avoid double counting and account for orchard structure. The reported combination achieves 95.5–97.8% accuracy against harvested ground truth.

  • 95.5–97.8% accuracy is achieved for yield estimation using tracking, tree-row merging, GMM detection, and CNN counting.The results are measured against harvested ground truth.
  • Yield estimation requires additional processing beyond detection and counting, including tracking fruits across image sequences to avoid double counting.The method also addresses the planar structure of modern apple orchards, where fruits can be visible from both sides of a tree row.
  • The reported yield-estimation results combine tracking components with unsupervised GMM detection and CNN counting.These results were previously published and are included for completion.

VII. CONCLUSION

The paper introduces datasets for apple detection, segmentation, and clustered-fruit counting, emphasizing diverse annotations for small objects in cluttered orchard environments. Baseline results indicate challenging scenarios and identify directions for improving future performance.

  • The work introduces one dataset for detecting and segmenting apples and a second dataset for counting clustered fruits.
  • The datasets emphasize diverse apple varieties, many object instances, and small objects in cluttered environments.The authors aim to support progress in detection, segmentation, and counting.
  • Baseline statistics and results indicate that the images contain challenging scenarios for current state-of-the-art object detectors.
  • Future work could improve detection accuracy by focusing more broadly on small instances and improve segmentation by weighting foreground–background class imbalance.The paper identifies small instances as having area < 322 pixels.
Loading 1909.06441v2…