Source-linked AI summary
Pedicel-Targeted Strawberry Harvesting via Imitation Learning on a Low-Cost Teleoperated Robotic Arm
Nitin Rai, Won Suk Lee, Hongyoung Leon, Heping Zhu
TL;DR
Pedicel-targeted strawberry harvesting requires precise manipulation in occluded scenes while avoiding fruit bruising. The paper trains an end-to-end imitation-learning policy from teleoperation demonstrations; it learned approach and grasp trajectories and ran autonomously on an edge device, though reliable full detachment was not achieved.
Problem
Pedicel detachment requires a multi-phase trajectory, sub-millimeter stem alignment, controlled closure to avoid bruising, and pull-twist motion in partially occluded scenes.
Source: §I · p. 1, §II · p. 2Method
The authors train an ACT policy on synchronized wrist-camera observations and joint-space actions from human teleoperation demonstrations of the harvesting sequence, then deploy it autonomously on a Jetson AGX Orin.
Source: §I · p. 1Results
≈15 Hz inference on a Jetson AGX Orin accompanied successful learning of the approach and grasp trajectory from 130+ human demonstrations.
Source: B. Deployment Characteristics · p. 2, §IV · p. 3Takeaways & Limitations
The pipeline validates the core technical approach and identifies concrete directions for improvement.
Source: §IV · p. 3Full pedicel detachment was not reliably achieved in this initial system.
Source: §IV · p. 3
Abstract
from arXiv · showhide
Precise pedicel-targeted grasping in strawberry harvesting remains a critical bottleneck due to the pedicel's small diameter (1.3-3 mm), color similarity with surrounding foliage, and precise force control required for clean abscission zone detachment. As a market-grade crop, strawberry aesthetic value makes bruise-free pedicel detachment essential over direct fruit grasping. Classical approaches struggle to unify vision models, motion planning, and contact manipulation into a robust framework, particularly in occluded canopy environments. Therefore, this work presents a visuomotor imitation learning pipeline for pedicel-targeted strawberry harvesting that unifies perception, motion planning, and contact manipulation from collecting human teleoperation demonstrations, training an Action Chunking with Transformers (ACT) policy with a ResNet-50 visual backbone, and deploying the learned policy autonomously on a Jetson AGX Orin edge platform at 15 Hz. The trained policy achieved consistent loss convergence from 2.84 to 0.091 over 100K gradient steps on 130 demonstrations. We have also established a reproducible open-source pipeline with publicly released dataset and model weights for the agricultural robotics community. Full dataset is available at, https://huggingface.co/datasets/nitindominicrai/strawberry_pedicel_grasp_teleoperation and trained policy at, https://huggingface.co/nitindominicrai/act_strawberry_pedicel. To replicate this work, all code, configurations, and deployment instructions are publicly available at, https://github.com/nitin-dominic/autonomous_strawberry_pedicel_harvesting.
I. INTRODUCTION
Strawberry harvesting is labor-intensive, while direct fruit grasping risks bruising and pedicel targeting requires alignment to a slender, occluded stem. The paper proposes a teleoperation-trained visuomotor policy and presents an open-access dataset and edge deployment as contributions.
I. INTRODUCTION
- Florida’s strawberry production requires timely harvesting, with an average of 110 workers per 100 acres needed during the February–March peak season.The introduction reports 16,200 planted acres and more than 332M lbs of fruit in 2024.Source: §I · p. 1
- Direct fruit contact can bruise strawberries, while 1.3–3 mm pedicels are difficult to localize amid foliage and calyx under canopy occlusion.The introduction also notes that coupling classical detection with motion planning can compound error in unstructured field conditions.Source: §I · p. 1
- The proposed end-to-end imitation-learning approach bypasses explicit intermediate perception and planning stages, learning from synchronized wrist-camera observations and joint-space actions.The policy is trained on human demonstrations of the full reach-grasp-twist-pull sequence [6].Source: §I · p. 1
- The contribution includes an open-access dataset of strawberry pedicel-grasping episodes recording complete reach-grasp-twist-pull sequences at 30 FPS.The introduction describes the dataset contribution as a teleoperation protocol and open-access dataset.Source: §I · p. 1
- The paper presents ACT policy training with a ResNet-50 backbone and deployment on a Jetson AGX Orin, including assessment of inference constraints and failure modes.The introduction identifies convergent learning behavior as part of the training contribution.Source: §I · p. 2
II. METHODOLOGY
The method frames pedicel detachment as a multi-phase manipulation task under partial occlusion. It uses teleoperated demonstrations to train a unified end-to-end visuomotor policy.
II. METHODOLOGY
- Successful detachment requires an approach under partial occlusion, sub-millimeter gripper alignment, controlled closure, and a pull-twist motion that avoids slipping.The manipulation combines twisting, pulling, and detaching in an unstructured scene.Source: §II · p. 2
- The framework uses teleoperation-based behavioral cloning to capture expert demonstrations spanning the manipulation sequence from approach through detachment.The demonstrations form a unified visuomotor dataset for policy training.Source: §II · p. 2
- The resulting end-to-end imitation-learning policy learns from the unified dataset to perform the pedicel-targeted manipulation sequence.The method presents this as a response to the task’s combined perception and contact-manipulation demands.Source: §II · p. 2
A. Data Collection via Human Demonstration
The demonstrations capture the approach and contact phases of pedicel-targeted grasping, but not full abscission-zone detachment because a task-specific end-effector was unavailable.
A. Data Collection via Human Demonstration
- About 130 teleoperation episodes, comprising around 250k frames, were collected for strawberry pedicel-targeted grasping.Each episode lasted 19–42 seconds.Source: A. Data Collection via Human… · p. 2
- Operators maneuvered through dense foliage to align the end-effector with the pedicel and used clutch-based grasping to simulate detachment contact behavior.Episodes contained a complete reach-approach-grasp sequence.Source: A. Data Collection via Human… · p. 2
- The dataset focuses on approach and contact rather than full abscission-zone detachment, which was not performed without a task-specific end-effector.These phases were identified as the primary perceptual and manipulation challenges.Source: A. Data Collection via Human… · p. 2
B. Policy Training and Deployment
The policy uses ACT with a ResNet-50 visual backbone and runs on a low-cost SO-ARM101 arm. Deployment uses FP32 inference on a Jetson AGX Orin.
B. Policy Training and Deployment
- The ACT policy uses ResNet-50 instead of ResNet-18 to improve feature extraction for the small-target pedicel-detection subtask.Hyperparameters were tuned for the 130-demonstration data regime.Source: B. Policy Training and Deployment · p. 2
- The trained ACT policy ran on a NVIDIA Jetson AGX Orin with Jetpack 5.1 and CUDA 11.4, averaging 25 Hz in FP32 precision mode.Source: B. Policy Training and Deployment · p. 2
A. Training Convergence
The ACT policy showed convergent training behavior, with joint prediction error plateauing after training on the collected demonstrations.
A. Training Convergence
- 0.062 was the plateaued l1 joint prediction error after 160k gradient steps.The KL divergence term stabilized near zero around 17k steps, indicating convergence of the CVAE latent distribution.Source: A. Training Convergence · p. 2
B. Deployment Characteristics
The policy ran autonomously on a Jetson AGX Orin and reproduced its learned approach trajectory; the section also identifies constraints on deployment and detachment.
B. Deployment Characteristics
- ≈15-17 Hz was the Jetson AGX Orin inference rate using FP32 and a ResNet-50 visual backbone.The arm consistently reproduced the learned approach trajectory autonomously.Source: B. Deployment Characteristics · p. 2
- Fig. 3 shows autonomous camera keyframes through a reach-align-contact sequence learned from teleoperation demonstrations.The sequence executes without human intervention.Source: Figure 3 · p. 2
- Short pedicels connecting directly to the main stem constrain end-effector maneuverability and require more compact gripper designs.Wrist-camera placement is critical because suboptimal mounting can occlude the gripper-pedicel contact region and degrade policy performance.Source: B. Deployment Characteristics · p. 3
- Synthetic environments fail to replicate pedicel dynamics and real-canopy light variation, limiting large-scale data augmentation.A cutting-based end-effector also requires sub-millimeter precision to clip individual pedicels without contacting the calyx or fruit.Source: B. Deployment Characteristics · p. 3
IV. CONCLUSIONS
The pipeline learned approach and grasp trajectories from human demonstrations and ran autonomously, but reliable full pedicel detachment remains unresolved. Planned work targets broader demonstrations, phase-aware training, and faster inference.
IV. CONCLUSIONS
- Full pedicel detachment was not reliably achieved, although the pipeline validated the core technical approach.The ACT policy learned approach and grasp trajectories from 130+ demonstrations and deployed autonomously on a Jetson Orin AGX at ≈15 Hz.Source: §IV · p. 3
- The authors plan to expand the dataset to at least 300 episodes covering varied berry positions and approach angles, with reliability expected to improve.The authors cite behavioral-cloning policies’ scaling with demonstration diversity.Source: §IV · p. 3
- Planned changes include phase-aware weighting of contact and detachment frames and JetPack 6.2 for 30 Hz inference and real-time reactive control.The weighted loss is intended to address precision asymmetry between approach and grasp phases; the firmware upgrade is intended to remove the current speed constraint.Source: §IV · p. 3