PaperlayerView exact revision on arXiv

Source-linked AI summary

RF-DETR Object Detection vs YOLOv12 : A Study of Transformer-based and CNN-based Architectures for Single-Class and Multi-Class Greenfruit Detection in Complex Orchard Environments Under Label Ambiguity

Ranjan Sapkota, Rahul Harsha Cheppally, Ajay Sharda, Manoj Karkee

arXiv:2504.13099v1cs.CV

TL;DR

Greenfruit detection in complex orchards is difficult under occlusion, camouflage, and ambiguous labels, and RF-DETR had not been directly compared with YOLOv12 in this setting. This study evaluates both models on orchard greenfruit detection; RF-DETR led mAP@50 in both tasks, while YOLOv12 variants led mAP@50:95.

  • Problem

    Direct comparative evidence for RF-DETR versus YOLOv12 in agricultural scenes with occlusion, camouflage, and ambiguous labels was lacking.

    Source: §1.3 · p. 4

  • Method

    The study evaluates RF-DETR and YOLOv12 for greenfruit detection in a commercial orchard, where labels are complicated by visibility ambiguity.

    Source: §2.1 · p. 4, §2.2 · p. 6, §2.3.1 · p. 6

  • Results

    RF-DETR led mAP@50 in single-class (0.9464) and multi-class (0.8298) detection, while YOLOv12N and YOLOv12L led mAP@50:95 at 0.7620 and 0.6622, respectively.

    Source: §5 · p. 14

  • Takeaways & Limitations

    Model choice depends on the evaluation target: RF-DETR led mAP@50, while YOLOv12 variants led mAP@50:95 in the reported tasks.

    Source: §5 · p. 14

Abstract

from arXiv · show

This study conducts a detailed comparison of RF-DETR object detection base model and YOLOv12 object detection model configurations for detecting greenfruits in a complex orchard environment marked by label ambiguity, occlusions, and background blending. A custom dataset was developed featuring both single-class (greenfruit) and multi-class (occluded and non-occluded greenfruits) annotations to assess model performance under dynamic real-world conditions. RF-DETR object detection model, utilizing a DINOv2 backbone and deformable attention, excelled in global context modeling, effectively identifying partially occluded or ambiguous greenfruits. In contrast, YOLOv12 leveraged CNN-based attention for enhanced local feature extraction, optimizing it for computational efficiency and edge deployment. RF-DETR achieved the highest mean Average Precision (mAP50) of 0.9464 in single-class detection, proving its superior ability to localize greenfruits in cluttered scenes. Although YOLOv12N recorded the highest mAP@50:95 of 0.7620, RF-DETR consistently outperformed in complex spatial scenarios. For multi-class detection, RF-DETR led with an mAP@50 of 0.8298, showing its capability to differentiate between occluded and non-occluded fruits, while YOLOv12L scored highest in mAP@50:95 with 0.6622, indicating better classification in detailed occlusion contexts. Training dynamics analysis highlighted RF-DETR's swift convergence, particularly in single-class settings where it plateaued within 10 epochs, demonstrating the efficiency of transformer-based architectures in adapting to dynamic visual data. These findings validate RF-DETR's effectiveness for precision agricultural applications, with YOLOv12 suited for fast-response scenarios. >Index Terms: RF-DETR object detection, YOLOv12, YOLOv13, YOLOv14, YOLOv15, YOLOE, YOLO World, YOLO, You Only Look Once, Roboflow, Detection Transformers, CNNs

1. Introduction

CNNs and transformer-based detectors dominate current object detection, but their relative performance in complex, label-ambiguous agricultural settings remains uncertain. This study compares RF-DETR and YOLOv12 on greenfruit detection to address that gap.

1. Introduction

  • CNNs and transformer-based models are widely adopted, while vision-language models are emerging among six object-detection approaches.The six approaches also include hybrid, sparse-coding, and traditional feature-based methods.Source: Figure 1 · p. 1, §1 · p. 2

1.1. CNN-based Object Detection Approaches

  • CNNs exploit hierarchical local features and suit real-time and edge deployment, but have a stated limitation in comprehensively modeling global context.YOLO is among the CNN-based detection families discussed.Source: §1.1 · p. 3

1.2. Transformers-based Object Detection Approaches

  • DETR uses encoder-decoder set prediction without anchor boxes or NMS, while deformable attention focuses on selected sampling points to reduce computation.The section also surveys real-time, hybrid, open-vocabulary, and other transformer-based variants.Source: §1.2 · p. 3

1. Introduction

  • The study targets the lack of direct RF-DETR–YOLOv12 comparisons in complex agricultural scenes, where occlusion, camouflage, and ambiguous labels complicate detection.The authors compare the models for commercial-orchard greenfruit detection.Source: §1.3 · p. 4

1.3. Objectives

  • The study evaluates RF-DETR and YOLOv12 using a custom greenfruit dataset, identical training protocols, and single-class and multi-class tasks.The evaluation includes precision, recall, F1-score, mAP@50, and mAP@50:95, as well as inference speed and processing efficiency.Source: §1.3 · p. 4

2. Methods

The study collects orchard imagery, labels greenfruits under two schemes, and trains RF-DETR and YOLOv12 under shared conditions. Its methods are designed to evaluate detection in visually complex orchard scenes.

2. Methods

  • Field images were processed and both models were evaluated for single-class and multi-class greenfruit detection using the same dataset, hyperparameters, and epochs.The workflow begins with field-image collection and robotic capture, then preprocessing and model evaluation.Source: §2 · p. 4

2.1. Study Site and Data Acquisition

  • The dataset images came from a commercial ’Scifresh’ apple orchard in Prosser, Washington, where immature fruitlets blend into green foliage.A robotic platform with an Intel RGB-D camera on a UR5e arm captured RGB imagery in May 2024; 857 images were collected.Source: §2.1 · p. 4

2.2. Data Preprocessing and Preparation

  • Manual annotation created a single “greenfruit” class and a second scheme separating occluded from non-occluded fruit, with at least 90% visibility defining non-occluded.The single-class set contained 4,125 object labels from 857 images.Source: §2.2 · p. 6
  • Label ambiguity arose when fruit overlap, shadows, foliage resemblance, or image-edge truncation made visibility and class assignment subjective.The authors note that guidelines could not make annotation fully objective in these scenes.Source: §2.2 · p. 6

2.3.1. Training RF-DETR Object Detection Model

  • RF-DETR combines a pretrained DINOv2 backbone with deformable cross-attention and omits anchor boxes and NMS to attend to relevant features under occlusion and clutter.Its single-scale feature extraction strategy is described as reducing computational overhead.Source: §2.3.1 · p. 6

2.3.2. Training YOLOv12 Object Detection Model

  • YOLOv12 uses R-ELAN and 7×7 separable convolutions, with area attention processing feature-map regions through FlashAttention-optimized operations.The passage describes four model configurations and hardware-aware optimizations for edge deployment.Source: §2.3.2 · pp. 6–7

2.4. Training Methodology

  • RF-DETR Base was trained for 50 single-class and 100 multi-class epochs; YOLOv12N, L, and X each trained for 100 epochs on both datasets.The models used 640×640 inputs; evaluation was intended to compare accuracy, convergence, and efficiency under standardized conditions.Source: §2.4 · p. 8, §2.5 · p. 8

Detection Evaluation Metrics

The evaluation metrics describe detection correctness, completeness, and balance using predicted boxes and ground-truth annotations.

Detection Evaluation Metrics

  • A true positive is a predicted box with IoU at least the defined threshold, commonly 0.50; a false positive is a mislocalized box or a detection of a nonexistent object.A false negative is a real fruit without a sufficiently overlapping predicted box.Source: Detection Evaluation Metrics · p. 8
  • Precision measures the accuracy of detected greenfruits, while recall measures detection completeness.The section also lists F1-score, mAP@50, mAP@50:95, and mIoU as evaluation metrics.Source: Detection Evaluation Metrics · p. 8
  • F1-score combines precision and recall into a single measure of model efficacy.It is one of the standardized metrics used to assess the models.Source: Detection Evaluation Metrics · p. 8

Intersection over Union (IoU) and Mean IoU (mIoU)

IoU describes bounding-box overlap, and mIoU averages overlap across detections; mAP metrics assess average precision at specified IoU thresholds.

Intersection over Union (IoU) and Mean IoU (mIoU)

Application to Our Dataset

The single-class task treats all greenfruits uniformly, while the multi-class task also evaluates occlusion status.

Application to Our Dataset

  • Occlusion-status errors count as false positives, while undetected fruit—particularly occluded fruit—count as false negatives.Single-class annotations treat all greenfruits uniformly; multi-class annotations distinguish occluded from non-occluded fruit.Source: Application to Our Dataset · p. 9

3. Results

RF-DETR generally handled difficult orchard examples better, while YOLOv12 configurations led on selected precision, recall, F1, and stricter-overlap metrics. RF-DETR also converged earlier in both detection settings.

3. Results

  • Across qualitative cases, RF-DETR detected camouflaged and heavily occluded fruit and reduced false positives and occlusion-label errors relative to YOLOv12.In a dense edge cluster, RF-DETR detected five actual fruits, though it did not classify their occlusion status with high certainty.Source: §3 · p. 9, Figure 6 · p. 11

3.1. Evaluation of Precision, Recall and F1-Score

  • YOLOv12N achieved the highest single-class recall (0.8901) and F1 score (0.8784), while YOLOv12L achieved the highest precision (0.8892).Table 1 compares precision, recall, and F1 across detection tasks and model configurations.Source: §3.1 · p. 9, Table 1 · p. 10

3.2. Analysis of Mean Average Precision (mAP)

  • Single-class: RF-DETR led mAP@50 (0.9464), while YOLOv12N led mAP@50:95 (0.7620); multi-class: RF-DETR led mAP@50 (0.8298), YOLOv12L mAP@50:95 (0.6622).Figure 7 presents the mAP comparisons for both tasks.Source: §3.2 · pp. 10–11, Figure 7 · p. 12

3.3. Training Dynamics and Model Convergence Analysis

  • RF-DETR plateaued before 10 epochs for single-class detection and stabilized around 20 epochs for multi-class detection, while YOLOv12X continued seeking equilibrium.Figure 8 plots mAP@50 against training epochs.Source: §3.3 · pp. 11–12, Figure 8 · p. 13

4. Discussion

The discussion situates greenfruit detection within efforts to improve accuracy and efficiency in variable orchard environments, and reports strong RF-DETR performance. It also identifies open-vocabulary, multimodal, and low-label approaches as future directions.

4. Discussion

  • Prior orchard studies compared YOLOv11 and YOLOv8 for occluded and non-occluded fruits [31] [49], and used YOLOv8 with 3D point-cloud shape fitting for fruit-size estimation.These studies reflect ongoing efforts to refine detection accuracy and efficiency in variable orchard environments.Source: §4 · p. 13
  • 0.9464 mAP@50 was reported for RF-DETR, which surpassed YOLOv12 and performed particularly well under partial visibility and camouflage [51].The discussion also highlights RF-DETR's rapid convergence during training.Source: §4 · p. 13
  • Open-vocabulary detection and vision-language models can identify broader fruit types and characteristics without retraining [53].The discussion links this adaptability to variation in fruit appearance, lighting, and occlusion in agricultural settings.Source: §4 · p. 13
  • Multimodal learning is proposed as a way to address camouflage and label ambiguity, while semi-supervised and few-shot learning could reduce reliance on extensive labeled datasets.Source: §4 · p. 14
  • Lightweight transformer variants and efficient vision-language models are identified as directions for real-time field analytics on mobile or edge systems.Source: §4 · p. 14

5. Conclusion

The study evaluates RF-DETR and YOLOv12 on single-class and multi-class greenfruit detection in commercial orchards, including training dynamics. RF-DETR leads mAP@50 in both tasks and converges quickly, while YOLO variants lead mAP@50:95 in each task.

5. Conclusion

  • The evaluation used real-world orchard images, occlusion-based labels, and standardized protocols across single- and multi-class tasks, assessing precision, recall, F1-score, and mAP.Source: §5 · p. 14
  • 0.9464 mAP@50 was RF-DETR’s single-class high, while YOLOv12N led mAP@50:95 at 0.7620.Source: §5 · p. 14
  • 0.8298 mAP@50 was RF-DETR’s multi-class high, while YOLOv12L led mAP@50:95 with 0.6622.Source: §5 · p. 14
  • RF-DETR plateaued in under 10 epochs in single-class detection, and the study reports quicker convergence than YOLOv12 overall.Source: §5 · p. 14
Loading 2504.13099v1…