PaperlayerView exact revision on arXiv

Source-linked AI summary

Sim-to-Real Reinforcement Learning for Path-Safe Target Reaching in Robotic Harvesting of Hydroponic Lettuce

Al Bashir, Partho Ghose, Yibin Wang, Robert G. Hardin, Joshua Peeples, Genhua Niu, Azlan Zahid

arXiv:2610.22337v1cs.RO

TL;DR

Crop-proximal lettuce harvesting requires more than accurate endpoint reaching: approach direction, orientation, and clearance also matter, while physical policy optimization can require risky exploratory actions. PSA-PPO learns target-conditioned, path-safe local reaching entirely in simulation and transfers to a UR10e without real-world policy fine-tuning. It achieves the highest physical reaching reliability reported in the conclusion, and the integrated greenhouse system reaches 94.4% of vision-guided targets.

  • Problem

    Harvesting-compatible reaching must preserve approach geometry and trajectory safety, while physical trial-and-error policy optimization risks damage to crops, infrastructure, or the robot.

    Source: §1 · pp. 1–2

  • Method

    PSA-PPO combines target-conditioned local reaching, spatial and trajectory-history safety, task-specific success criteria, and simulation-only training for physical deployment.

    Source: §1 · p. 3

  • Results

    PSA-PPO achieved 94.7% reaching reliability on the physical robot, and the integrated greenhouse system achieved 94.4% vision-guided target reaching.

    Source: §1 · p. 3, §4 · p. 16

  • Takeaways & Limitations

    Path-safety and side-approach constraints materially affected strict task performance even when endpoint reaching remained high, so terminal accuracy alone is incomplete.

    Source: §3.4 · pp. 12–13

    The physical study used one manipulator, one greenhouse environment, a frozen deployment correction, and one trained policy; path-safe does not guarantee formal uncertainty awareness for the complete robot or greenhouse scene.

    Source: §3.4 · p. 14

Abstract

from arXiv · show

Robotic harvesting requires precise manipulation to ensure accurate positioning of the end-effector, especially in a constraint workspace. This study introduces the Path-Safe Approach Proximal Policy Optimization (PSA-PPO), a target-conditioned local reaching framework designed for robotic harvesting of nutrient film technique (NFT)-hydroponic lettuce in greenhouses. The proposed PSA-PPO preserves the standard clipped PPO objective while introducing a harvesting-oriented policy formulation that integrates target-conditioned observations, corridor-aware spatial constraints, trajectory-history-based safety encoding, and task-specific success criteria for constrained robotic manipulation. PSAPPO was trained entirely in Isaac Lab and successfully transferred to a physical UR10e robotic arm without any real-world policy fine-tuning. In a controlled physical benchmark against 150 scheduled targets, PSA-PPO achieved the highest reaching rate of 94.7%, outperforming Cartesian+TOTG (86.0%), RRTConnect (88.0%), and RRT* (80.0%), while reducing median RMS jerk by 39.8 to 73.0% relative to the classical planners during the final approach segment. A formulation ablation evaluated over 200 simulation targets demonstrates that path-safety and side-corridor shaping are the principal drivers of harvesting-compatible performance: removing either term substantially degrades strict success even when terminal endpoint accuracy is preserved or improved, confirming that optimization toward the endpoint alone is an insufficient objective for constrained crop-proximal manipulation. Finally, PSA-PPO was integrated with an RGB-D vision module and an end-effector and evaluated as a complete greenhouse harvesting system across 144 lettuce harvesting attempts, achieving 94.4% vision-guided target reaching and 86.1% complete harvesting success.

1. Introduction

Lettuce harvesting requires repeatable, crop-compatible local reaching because targets sit among constrained workspace, plant material, and cultivation infrastructure. PSA-PPO addresses this with a simulation-trained, target-conditioned path-safe approach policy, evaluated on a physical robot and in a vision-guided harvesting system.

1. Introduction

  • The study identifies a gap in converting vision-derived lettuce cutting points into repeatable motion that respects approach geometry and clearance constraints.For lettuce, success involves more than final TCP error: approach direction, target-height geometry, tool orientation, and wrist–TCP clearance also matter.Source: §1 · pp. 1, 3
  • PSA-PPO reformulates target-conditioned local reaching with geometry-conditioned safe starts, clearance-aware observations, coupled task and safety rewards, and trajectory-history-dependent success criteria.The policy is trained entirely in Isaac Lab and transferred without real-world policy fine-tuning.Source: §1 · p. 3
  • Across 144 greenhouse harvesting attempts, the integrated system achieved 94.4% target-reaching success.The pipeline combines PSA-PPO with RGB-D vision and an end effector.Source: §1 · p. 3
  • The study benchmarks the simulation-trained policy on a physical UR10e against Cartesian+TOTG, RRTConnect, and RRT* over 150 common targets.The comparison covers reliability, precision, convergence, motion dynamics, joint tracking, and trajectory geometry.Source: §1 · p. 3

2. Methodology

The methodology combines RGB-D cutting-point streaming, simulation-trained local side-approach reaching, a cut-and-hold tool, and ROS deployment. The policy is transferred to the UR10e as a frozen actor, with independent execution safeguards and a separate harvesting sequence.

2.1. 3D Target Streaming

  • RGB-D cutting points are back-projected into 3D and temporally averaged to reduce frame-to-frame variation before publication as robot targets.The perception process associates detections with tracks and removes tracks not observed in the current frame to prevent stale targets.Source: §2.1 · pp. 3–4

2.2. Simulation-Trained Policy for Path-Safe Side-Approach Target Reaching

  • PSA-PPO learns only the final local approach after deterministic, target-conditioned pre-positioning, using corridor, orientation, clearance, and trajectory-history information to define harvesting-compatible success.The 42-dimensional observation includes wrist and TCP clearances; strict capture requires path-safety conditions, and successful termination requires consecutive strict-capture steps.Source: §2.2 · pp. 4–5
  • The policy uses PPO with a clipped update and a reward combining target progress, corridor tracking, tool orientation, path safety, motion regularization, and terminal capture.Training uses workspace-sampled targets and simulation evaluation; crop cutting and plant interaction are not part of policy optimization.Source: §2.2 · pp. 4–5

2.3. End-Effector Integration

  • A single gripper-closing action drives the shear mechanism and retains the detached lettuce for transfer.The redesigned end effector adapts the cut-and-hold tool for actuation by an OnRobot 2FG7 gripper.Source: §2.3 · pp. 5–6

2.4. ROS-Based Deployment and Harvest

  • In ROS deployment, vision targets are transformed and corrected, then used for safe-start pre-positioning before the frozen PSA-PPO actor controls local reaching.Independent timing and acceleration checks can halt motion; after reaching, the gripper cuts, holds, transfers, and releases the lettuce.Source: §2.4 · p. 6

3. Experiment

The section evaluates PSA-PPO’s simulation-to-physical transfer through reaching, motion, ablation, and greenhouse trials. It also discusses deployment-stack effects, downstream harvesting bottlenecks, the scope of the findings, and priorities for broader evaluation and system improvement.

3.1. Experimental Setup

  • The policy was trained and evaluated in Isaac Lab, then transferred to an RB-VOGUI platform with a UR10e without physical-robot fine-tuning.The controlled benchmark used scheduled targets to isolate controller behavior, while greenhouse trials used online RGB-D targets in the complete harvesting pipeline.Source: §3.1 · p. 7

3.2. Evaluation Protocol

  • The evaluation covered simulation, controlled physical motion, and greenhouse harvesting, with the physical comparison using a common 20-mm reaching tolerance and local-motion analysis over the final 0.22 m.Greenhouse outcomes were tracked sequentially through reaching, cutting, retention, and release; overall success required completion of all stages.Source: §3.2 · pp. 7–8, Table 1 · p. 8

3.3.1. Simulation Validation and Formulation Ablation

  • On 200 simulation targets, PSA-PPO achieved 92.0% strict success; removing path-safety terms reduced strict success to 63.5%, while removing side-corridor terms reduced it to 73.0%.Without path-safety terms, final reaching fell from 99.0% to 89.0%, and P95 maximum below-plane violation rose from 2.11 to 26.8 mm. Without side-corridor terms, all targets were reached and median final TCP error improved from 32.3 to 25.3 mm, yet strict success fell. One independently trained checkpoint was tested per formulation, so results do not quantify retraining variability.Source: §3.3.1 · pp. 8–10, Table 2 · p. 9

3.3.2. PSA-PPO vs. Classical Motion Planners on the Physical Robot

  • 94.7% of scheduled targets were reached by PSA-PPO, versus 88.0% for RRTConnect, 86.0% for Cartesian+TOTG, and 80.0% for RRT*.In the final 0.22-m approach, median RMS jerk was 11.38 m/s^3 for PSA-PPO, with reductions of 73.0%, 72.1%, and 39.8% relative to Cartesian+TOTG, RRTConnect, and RRT*, respectively. Classical planners retained better terminal precision and more direct average trajectories, so the benchmark showed a trade-off rather than uniform dominance.Source: Table 3 · p. 10, §3.3.2 · pp. 10, 12, Table 4 · p. 12

3.4. Discussion

  • The results indicate that path-safety and side-corridor shaping support harvesting-compatible success beyond endpoint accuracy, while the tested motion-regularization removal did not establish a direct jerk benefit.The physical results suggest a distinct operating trade-off: PSA-PPO favored reliability and lower dynamic excitation, while classical planners favored precision and path efficiency; lower jerk alone does not establish collision safety.Source: §3.4 · pp. 12–14

3.3.3. Target-Reaching Performance in Harvesting

  • Across 144 greenhouse attempts, PSA-PPO reached 136 vision-derived cutting points (94.4%) and completed cutting for 124 plants (86.1% of all attempts).Conditional success was 91.2% for cutting, 89.5% for retention, and 97.3% for basket release; the main downstream losses occurred during crop interaction, especially cutting and retention.Source: §3.3.3 · p. 12, Table 5 · p. 13

3.4. Discussion

  • The observed jerk advantage reflects both stack-level filtering and sustained smoother approach motion, but the learned policy’s contribution cannot be isolated from the hardware safeguards.The temporal action filter (𝛼= 0.18) and acceleration safeguard (0.0045 rad/𝑠2) attenuate motion; the motion-regularization ablation does not support directly attributing smoothness to reward terms.Source: §3.4 · p. 14
  • Incomplete harvests followed successful reaching, shifting the main system bottleneck toward crop and tool interaction rather than target acquisition or basket release.The similar reaching rates with predefined and RGB-D-derived targets suggest limited degradation after perception integration, but do not establish statistical equivalence; improving reaching alone is unlikely to substantially improve complete-harvest success.Source: §3.4 · p. 14
  • “Path-safe” denotes side-corridor entry and specified collision-clearance criteria, not formal uncertainty-awareness guarantees for the complete robot or greenhouse scene.The physical study covered one manipulator, one greenhouse, one trained policy per formulation, and a frozen deployment correction; it does not quantify variability across independent training runs.Source: §3.4 · pp. 14, 16
  • Planner baselines were tested only in controlled reaching, while the perception and cutting modules were inherited from earlier work rather than independently re-evaluated.Broader claims require repeated training seeds, additional robot and crop configurations, and evaluation under changing canopy and infrastructure geometry.Source: §3.4 · p. 16
  • Future work proposes hybrid planning and control that combines global scene-aware planning, learned crop-proximal control, and contact-aware crop interaction.This could pair planners’ geometric strengths with PSA-PPO’s observed reliability and lower local dynamic excitation.Source: §3.4 · p. 16

4. Conclusion

The study presents PSA-PPO as a simulation-trained, path-safe lettuce-reaching policy transferred to a UR10e without real-world policy fine-tuning. Its physical benchmark and ablation results support a distinct crop-proximal operating regime, not universal controller superiority.

4. Conclusion

  • 94.7% reaching reliability on the physical robot was the highest among the evaluated controllers, with lower local dynamic excitation; classical planners retained advantages in terminal precision.The study frames this as a distinct, practically useful operating regime for crop-proximal manipulation rather than universal controller superiority.Source: §4 · p. 16
  • Path-safety and side-approach constraints materially affected strict task performance even when endpoint reaching remained high.The ablation indicates that terminal accuracy alone is an incomplete measure of harvesting-compatible motion.Source: §4 · p. 16
  • PSA-PPO was trained entirely in simulation and transferred to a physical UR10e without real-world policy fine-tuning.The policy was formulated for target-conditioned, path-safe, side-corridor lettuce reaching.Source: §4 · p. 16

CRediT authorship contribution statement

Al Bashir is credited across the study’s core research and reporting activities. Other listed contributors are credited with review, while Azlan Zahid is credited with supervision and funding acquisition.

CRediT authorship contribution statement

A. PSA-PPO Training and Task Configuration

PSA-PPO uses the standard clipped PPO update, with harvesting-specific initialization, observations, rewards, and strict-success criteria. The appendix specifies the task configuration and explains how the full policy and component-removal variants were trained and evaluated.

A. PSA-PPO Training and Task Configuration

  • PSA-PPO retains the standard clipped PPO update while defining the policy through harvesting-specific initialization, observations, rewards, and strict-success criteria.The appendix directs readers to tables listing thresholds, workspace and safe-start rules, reward terms, and PPO/network settings.Source: A. PSA-PPO Training and Task… · p. 18

1.24 −0.04𝑧𝑛+ 𝛿𝑒

  • The nominal safe-start posture was target-determined, with a small independent joint perturbation added during training.Each joint received a uniform perturbation in [−0.004, 0.004] rad after the posture was constructed.Source: §1.24 · p. 18, A. PSA-PPO Training and Task… · p. 18
  • Strict capture required final-pose thresholds, positive wrist–TCP clearance, and bounded below-plane violations; successful termination required consecutive qualifying steps.An accumulated violation above the hard threshold ended the episode early.Source: §1.24 · p. 18, Table A.1 · p. 18
  • The ablations changed only their listed reward or termination logic, retaining observation size, action processing, target sampling, safe-start construction, capture thresholds, and strict evaluation.The implementation-level reward terms and workspace/safe-start configuration are documented in the appendix tables.Source: Table A.3 · p. 19, Table A.2 · p. 19, Table A.4 · p. 20
  • All formulations used seed 42 and the same actor–critic architecture and PPO optimizer settings, and were evaluated at the model-6000 checkpoint.Each checkpoint was assessed against the same 200 targets using identical strict validation criteria; the full runner allowed 12,000 iterations and ablation runners 7,000.Source: §1.24 · p. 20, Table A.5 · p. 20

B. Simulation Validation and Formulation-Ablation Details

The appendix documents paired evaluation of PSA-PPO and component-removal formulations on common targets, with target-level inference and several complementary outcome diagnostics. It also cautions that the diagnostics do not establish retraining robustness or isolate every reward component’s effect.

B. Simulation Validation and Formulation-Ablation Details

  • The ablation evaluation used identical target coordinates and strict criteria, with paired binary outcomes analyzed using bootstrap intervals, McNemar tests, and Holm correction.Targets, rather than trajectory samples, were the inferential unit; the intervals and tests characterize evaluation-target variation, not retraining variability.Source: B. Simulation Validation and… · p. 20, Table B.1 · p. 20
  • Figure B.1 partitions paired strict outcomes by which formulation succeeded, while Table B.2 reports the corresponding outcome differences.Figure B.2 maps target locations and failure types; Table B.3 reports robustness of the selected PSA-PPO checkpoint across predefined workspace bins.Source: B. Simulation Validation and… · p. 20, Figure B.1 · p. 21, Table B.2 · p. 21
  • Representative target-matched trajectories are qualitative diagnostics, not estimates of population effects.The examples were selected to expose each component-removal contrast, and the star marks the target location.Source: B. Simulation Validation and… · p. 21, Figure B.3 · p. 22
  • Post-hoc fixed-band corridor measures are secondary diagnostics, not a direct decomposition of the complete corridor reward.The no-motion-regularization policy did not produce a monotonic increase in the action and joint-velocity diagnostics, so lower TCP jerk is not attributed causally to the regularization terms.Source: B. Simulation Validation and… · p. 21, Figure B.4 · p. 23, Figure B.5 · p. 23
  • Orientation shaping and final orientation criteria were retained across all three component-removal formulations.Figure B.6 shows final approach-axis alignment and absolute tool-axis vertical-component distributions.Source: B. Simulation Validation and… · p. 21, Figure B.6 · p. 23

C. Physical Controller Benchmark and Deployment Details

The physical benchmark used shared target definitions and a 20-mm paper-level reaching criterion, while each controller retained its deployment-specific execution setup. The MoveIt baselines used a staged, collision-checked harness.

C.1. Physical Execution and Planner

  • 20 mm was the paper-level reaching tolerance; the online 2-mm monitor threshold managed execution but did not define reported task success.The benchmark compared PSA-PPO, Cartesian+TOTG, RRTConnect, and RRT* against the same physical target definitions.Source: §C.1 · p. 24
  • The comparison covered complete deployed controllers, not identical low-level command generation, because only PSA-PPO used its frozen Cartesian correction.Planner targets went directly to MoveIt without that policy-specific correction.Source: §C.1 · p. 24
  • MoveIt baselines used measured joint states when available, rejected planned-start mismatches above 0.005 rad, and collision-checked trajectories before execution.Cartesian motion used collision-checked interpolation followed by time-optimal trajectory generation; OMPL trajectories went to the UR scaled joint-trajectory controller.Source: §C.1 · p. 24

C.2. Trajectory Reconstruction and Analysis

Controller trajectory comparisons were recomputed offline from archived data using one common processing pipeline. Binary reaching outcomes and continuous trajectory metrics used different target populations.

C.2. Trajectory Reconstruction and Analysis

  • All controllers’ TCP transforms and joint/controller states were archived at 100 Hz, then processed offline through one common pipeline.Online per-segment estimates were excluded after invalid path-length estimates appeared for some RRTConnect segments.Source: §C.2 · p. 24
  • Trajectories were timestamp-ordered, resampled, smoothed over 0.15 s, and evaluated with central finite differences; the high-end jerk statistic was the 99.5th percentile.Path length used 30-Hz smoothed positions and 2-mm Ramer–Douglas–Peucker simplification; the final 0.22-m target-relative window was fixed before comparison.Source: §C.2 · p. 24
  • Binary reaching rates used all scheduled targets, while continuous paired comparisons used only targets successful for all four controllers.The physical target was the experimental unit; paired differences used Wilcoxon signed-rank tests and 10,000 paired bootstrap resamples, with Holm correction.Source: §C.2 · p. 24

C.3. Detailed Motion and Geometry Statistics

The appendix reports physical motion comparisons both for the learned local-reaching region and for the complete reconstructed approach. It separates dynamic excitation from path geometry and endpoint accuracy.

C.3. Detailed Motion and Geometry Statistics

  • The same qualitative dynamic ordering seen in the final 0.22-m local region also appeared over the complete reconstructed approach.Table C.4 reports selected complete-approach motion metrics on the common-success set of 97 targets.Source: §C.3 · p. 24
  • Table C.5 reports target-wise effect sizes for the final 0.22-m region, defined as PSA-PPO minus planner; negative differences indicate a lower PSA-PPO metric.Source: §C.3 · p. 24
  • Lower dynamic excitation does not imply a shorter or straighter path: population-level trajectory geometry is reported in Table C.7, while numerical endpoint-error medians are retained in Table C.6 for direct lookup.Source: §C.3 · p. 24

C.4. Failure-Identity Sensitivity and Timing

Because some additional planner failure identities were not independently preserved, the appendix uses a deterministic sensitivity population rather than treating selected identities as observed ground truth. Benchmark timing is retained only as a harness diagnostic.

C.4. Failure-Identity Sensitivity and Timing

  • Aggregate physical reaching totals were witnessed, but some additional planner failure identities were not independently preserved and were selected from logged-success trials by telemetry anomaly rank.The selected identities form a sensitivity population consistent with witnessed counts, not independently observed failure identities.Source: §C.4 · pp. 24, 26
  • Hard diagnostic events included tool-orientation travel of at least 360°, single-joint travel of at least 2π, or a wrist/elbow margin within 5° of a multiple of π.Remaining trials were ranked within each planner using percentile tails of trajectory and motion diagnostics.Source: §C.4 · pp. 24, 26
  • The primary four-controller continuous-analysis population contained 97 targets; 25 RRT* targets without trajectory archives remained binary failures but were missing for continuous metrics.Table C.3 summarizes the populations used for binary and continuous comparisons.Source: §C.4 · p. 25, Table C.3 · p. 25
  • Tables C.4–C.6 cover complete-approach motion metrics, paired local-region effects, and terminal positioning accuracy, respectively.The reduction percentages use controller medians, with paired target-level bootstrap intervals and Wilcoxon tests corrected within the prespecified comparison family.Source: Table C.4 · p. 26, Table C.5 · p. 26, Table C.6 · p. 26
  • Elapsed benchmark time includes monitoring and waiting behavior, so timing variables are harness diagnostics rather than controller-speed measures.The PSA-PPO harness monitored a 2-mm threshold while paper-level success used 20 mm; only 32 of 1,080 commands reached the monitor threshold, and all 142 successful targets included at least one inferred timeout.Source: §C.4 · p. 26

C.5. Kinematic and Atypical-Trajectory

This section audits planner trajectory anomalies using kinematic and selected hard-event diagnostics, without treating them as causal evidence of failure. Its matched trajectory examples characterize tail behavior, not representative population trajectories.

C.5. Kinematic and Atypical-Trajectory

  • Wrist and elbow margins measure angular distance to the nearest multiple of π; the 5° threshold is diagnostic and does not establish singularity-caused failure.Source: §C.5 · p. 26, Figure C.1 · p. 27
  • Selected sets contained no listed hard events for Cartesian+TOTG; RRTConnect had 2 orientation, 1 single-joint, 3 wrist-near-singularity, and 1 elbow-near-singularity cases, versus 4, 3, 2, and 1 for RRT*.The counts describe the sensitivity-selected subsets, not the full N=97 population.Source: §C.5 · p. 26
  • Trajectory geometry covers the reconstructed physical approach; medians describe typical paths, while P95 and maximum deviations describe distribution tails.Source: Table C.7 · p. 27
  • For the common-success set (N=97), lower detour ratio, backtracking, turning, azimuth and joint travel, and line deviation indicate more compact motion; higher path efficiency indicates greater directness.Source: §C.5 · p. 27
  • The 25 unarchived RRT* attempts remain binary failures and are unavailable for continuous metrics; the corrected four-controller common-success set contains 97 targets.Source: §C.5 · p. 27
  • Figure C.2 pairs PSA-PPO with each planner on matched targets, but the selected examples show tail behavior rather than representative trajectories or collision events.Source: §C.5 · p. 27, Figure C.2 · p. 28

Related papers

Loading 2610.22337v1…