PaperlayerView exact revision on arXiv

Source-linked AI summary

Agri-Sim: Agricultural Simulation Platform for Embodied Intelligence Evaluation in Greenhouse Robotics

Shuhan Shi, Zhenfeng Xue, Minghao Mei, Chao Zheng, Nan Li, Zhonghua Miao

arXiv:2608.29100v1cs.RO

TL;DR

Agricultural-robot development lacks simulation environments that jointly support realistic scenes, virtual sensing, navigation, planning, and manipulation. Agri-Sim combines Unity, ROS2, MoveIt 2, a configurable tomato greenhouse, and a mobile dual-arm robot for closed-loop evaluation, achieving repeatable functional workflows across navigation and harvesting tasks.

  • Problem

    Agricultural-robot development requires simulation that jointly supports realistic scene construction, virtual sensing, navigation, motion planning, and manipulation-task execution.

  • Method

    Agri-Sim integrates a configurable tomato greenhouse, mobile dual-arm harvesting robot, multimodal virtual sensors, and bidirectional Unity–ROS2 communication, with Unity executing simulation and ROS2/MoveIt 2 providing robotics functions.

  • Results

    In 50 trials per task, navigation succeeded in 45 cases (90%), trajectory planning in 48 (96%), and complete dual-arm harvesting in 43 (86%).

  • Takeaways & Limitations

    Agri-Sim supports closed-loop integration and repeatable functional evaluation of virtual sensing, navigation, collision-aware planning, and coordinated dual-arm execution.

  • Takeaways & Limitations

    The experiments assess functional integration rather than algorithm superiority, and Unity tomato attachment does not reproduce gripping force, fruit deformation, stem detachment, or damage.

Abstract

from arXiv · show

Agricultural-robot development requires simulation environments that can jointly support realistic scene construction, virtual sensing, autonomous navigation, motion planning, and manipulation-task execution. This paper presents Agri-Sim, a Unity and ROS2-based simulation platform for the closed-loop development and functional evaluation of agricultural robots. The platform contains a configurable tomato-greenhouse environment, a mobile dual-arm harvesting robot, virtual RGB-D, LiDAR, IMU, and joint sensors, and a bidirectional communication interface between Unity and ROS2. Unity is responsible for scene rendering, rigid-body dynamics, collision detection, virtual sensing, and task-state execution, whereas ROS2 and MoveIt 2 provide localization, navigation, collision-aware motion planning, inverse kinematics, and trajectory generation. Autonomous greenhouse navigation and dual-arm tomato harvesting were used to evaluate the complete simulation workflow. The experiments covered virtual sensor publication, ROS2-based navigation, collision-aware motion planning, mobile-base control, tomato acquisition, inter-arm handover, and box placement. The results demonstrate that Agri-Sim supports closed-loop integration and repeatable functional evaluation of navigation and manipulation workflows in a controlled virtual greenhouse, providing a practical foundation for subsequent algorithm development and Sim-to-Real studies.

1. Introduction

Agri-Sim addresses the need for agricultural simulation that is repeatable yet faithful to complex greenhouse scenes, sensors, and integrated robot operations. It combines a configurable Unity greenhouse and mobile dual-arm robot with bidirectional ROS2 communication to evaluate navigation and harvesting workflows.

  • Greenhouse harvesting requires coordinated perception, localization, navigation, and manipulation under confined, occluded, deformable, and variable conditions.
  • Physical agricultural-robot experiments are costly, time-consuming, and difficult to repeat because environmental and crop conditions vary over time.
  • Existing simulators often lack the combined agricultural-scene, sensor, and system fidelity needed for integrated mobile manipulation.
  • Agri-Sim provides adjustable environmental, scene, sensor, and simulation-time parameters for controlled and repeatable experiments.
  • The platform integrates a tomato greenhouse, mobile dual-arm harvesting robot, multimodal virtual sensors, and bidirectional Unity–ROS2 communication.
  • Autonomous navigation and dual-arm harvesting validate a closed-loop workflow covering acquisition, inter-arm handover, and box placement.

2. Related Work

Prior agricultural simulation research advances individual capabilities and specialized platforms, but system-level integration across sensing, navigation, planning, and manipulation remains incomplete. Agri-Sim addresses this gap with a unified, repeatable Unity–ROS2 workflow while remaining outside the scope of high-fidelity biological modeling and reinforcement-learning transfer experiments.

  • Existing platforms span rigid-body dynamics, virtual sensing, robot learning, visual rendering, plant deformation, digital twins, and agricultural task simulation.
  • Unity offers detailed programmable environments but does not itself provide the complete localization, navigation, motion-planning, and control stack.
  • Connecting Unity with ROS2 for greenhouse mobile manipulation requires consistent frames, timestamps, sensor messages, robot states, commands, and trajectories.
  • Many agricultural systems target a particular robot type, sensing modality, or isolated task rather than connecting the full mobile-manipulation pipeline.
  • Agri-Sim combines a Unity tomato greenhouse, mobile dual-arm robot, configurable RGB-D, LiDAR, IMU, and joint-state sensors, and bidirectional ROS2 communication.
  • Agri-Sim is not intended to replace specialized platforms for high-accuracy plant deformation, biological growth modeling, or massively parallel reinforcement-learning training.

3. Proposed Platform

Agri-Sim integrates a configurable greenhouse, mobile dual-arm robot, multimodal virtual sensing, and ROS2–Unity closed-loop control for agricultural mobile-manipulation evaluation. Its modular design supports repeatable navigation and manipulation experiments while retaining explicit scope limits on biological and sensor fidelity.

  • Platform architecture: Agri-Sim combines a configurable tomato-greenhouse environment, mobile dual-arm robot, multimodal virtual sensors, and bidirectional ROS2–Unity communication.Unity handles rendering, physics, sensing, collision detection, and motion execution; ROS2 and MoveIt 2 handle transformation, navigation, planning, and task control.
  • Platform architecture: The platform connects sensing, planning, control, and execution in a closed loop, returning ROS2-generated commands to Unity and republishing updated observations.The loop advances from environment and robot state to observations, planning, command execution, and the next state.
  • Platform architecture: Four functional layers organize the environment, robot and sensors, communication, and ROS2/MoveIt 2 algorithms.This separation allows ROS2-side algorithms, greenhouse layouts, and sensor configurations to be modified without reconstructing the entire system.
  • Virtual greenhouse: Modular greenhouse assets support confined navigation passages, variable tomato occlusion and target accessibility, and inserted obstacles for safety and collision-aware planning tests.Structure, plants, fruits, cultivation facilities, and obstacles can be rearranged without rebuilding the complete scene.
  • Virtual greenhouse: Environmental controls vary date, time, weather, season, temperature, humidity, and rendering quality while preserving greenhouse geometry.Experimental groups can fix layout, robot pose, targets, obstacles, sensors, physics, and simulation time to support repeatability.
  • Scope boundary: The platform currently models controllable geometric and visual conditions, excluding high-fidelity crop growth, flexible plant dynamics, fruit detachment, and detailed IMU error effects.Plants and fruits use collision geometries, while bias drift, scale-factor error, and temperature dependence are left for future higher-fidelity models.
  • Robot model and control: The robot uses an omnidirectional chassis, lifting mechanism, two articulated manipulators, end effectors, and a rear enclosure for greenhouse mobile manipulation.Its Unity articulation hierarchy matches the ROS2 and MoveIt 2 robot description, supporting consistent state feedback, inverse kinematics, and trajectory execution.
  • Virtual sensing: Virtual RGB-D, LiDAR, IMU, joint-state, and collision sensors provide configurable observations, poses, frames, acquisition rates, and ROS2 topics.RGB-D outputs synchronized color and depth, while LiDAR generates ray-based three-dimensional point clouds for obstacle perception, mapping, localization, and navigation.

4. Experimental Evaluation

The evaluation tests Agri-Sim as a closed-loop simulation platform linking Unity and ROS2 across communication, sensing, navigation, and manipulation components. Experiments use a configurable greenhouse and mobile dual-arm robot with multimodal virtual sensors, while the LiDAR only approximates the physical sensor's scan behavior.

  • Experimental scope: The experiments evaluated ROS2–Unity communication, autonomous navigation, and dual-arm tomato harvesting as a functional platform workflow.The evaluation targeted closed-loop interaction between simulated sensors and external ROS2 algorithms rather than benchmarking a specific navigation or manipulation algorithm.
  • Experimental environment: The virtual greenhouse measured approximately 70 × 55 × 10 m and contained 160 configurable crop rows, with six rows enabled during experiments.Greenhouse aisles were approximately 2 m wide for mobile navigation and manipulation.
  • Robot configuration: The simulated robot combined a four-wheel-steering omnidirectional base, vertical lift, and two 7-DoF manipulators.The two arms provided 14 primary arm degrees of freedom, excluding grippers and other joints.
  • Virtual sensors: The robot used RGB-D, LiDAR, IMU, and joint-state sensors to generate perception and state information for ROS2 navigation and manipulation.The RGB-D camera used a 1280 × 720 depth image at 10 Hz, while the LiDAR followed principal parameters of the Livox MID-360.
  • Virtual sensors: The virtual LiDAR used MID-360-compatible parameters but did not reproduce the physical device's proprietary non-repetitive scanning pattern.This limits direct fidelity between simulated and physical LiDAR observations.
  • Communication: The experiments configured bidirectional information publication and reception at 10 Hz over the Unity–ROS2 interface.Unity transmitted sensor and state data to ROS2, while ROS2 sent navigation and manipulator trajectory commands in reverse.

4.3. Evaluation Criteria

Evaluation criteria measure functional message exchange, collision-free waypoint navigation, and completion of the full harvesting sequence. The study demonstrates closed-loop operation and aggregate task outcomes, but does not quantify communication latency or detailed navigation behavior.

  • Evaluation criteria: The platform was evaluated on continuous message exchange, collision-free waypoint reaching, and completion of the dual-arm harvesting sequence.These criteria cover communication, mobile navigation, and end-to-end manipulation execution.
  • Evaluation boundary: The evaluation measured functional connectivity rather than quantitative network performance because communication latency and round-trip delay were not recorded.The results should not be interpreted as a network-latency benchmark.
  • Navigation criteria: Navigation success required the mobile-base center to enter a 250-mm-radius goal region without collision within 120 s.Trials were scored as binary success or failure, without distance-error bins or other positioning-error statistics.
  • Harvesting criteria: The harvesting criterion required executable trajectories, tomato acquisition and attachment, inter-arm transfer, and placement into the harvesting box.A trial succeeded only when every stage was completed.
  • Communication verification: Communication evaluation verified bidirectional transmission of virtual sensor data, robot states, mobile-base commands, and manipulator trajectories.Unity published multimodal observations and states, while ROS2 sent control commands back to the simulated robot.
  • Closed-loop operation: Both message publication and reception operated at 10 Hz, and the bidirectional connection remained operational throughout navigation and harvesting.This enabled the perception–planning–control loop without manual transfer of intermediate data.

4.6. Tomato-Harvesting Evaluation

The tomato-harvesting evaluation tests a complete dual-arm workflow using ground-truth target poses, collision-aware planning, handover, and box placement. Across 50 trials, planning succeeded in 48 and the full harvesting sequence succeeded in 43, with failures occurring during planning or task execution.

  • Evaluation setup: The experiment evaluated complete harvesting rather than initial grasp alone, using Unity-provided ground-truth tomato poses.It therefore tested coordinate transformation, motion planning, and task execution rather than autonomous detection or pose estimation.
  • Harvesting workflow: The workflow comprised first-arm grasping and attachment, inter-arm handover, transfer by the second arm, and placement into the harvesting box.Representative frames show target selection, pre-grasp positioning, acquisition, handover, transfer, placement, and completion.
  • Motion planning: 48 of 50 trials generated executable trajectories for all acquisition, handover, and placement planning requests.The resulting motion-planning success rate was 96%, with a Wilson 95% CI of 86.5%–98.9%.
  • Failure analysis: Five planning-successful trials failed during execution: two during grasp attachment, two during inter-arm handover, and one during box placement.The two remaining unsuccessful trials failed during motion planning because of a planning-time limit or invalid trajectory generation.
  • Integrated capability: The platform supported collision-aware single-arm planning and coordinated dual-arm execution through attachment, handover, and final placement.This result covers the integrated manipulation stages rather than an isolated grasping operation.

4.7. Simulation-Time and Runtime Performance

Agri-Sim used a 50 Hz Unity physics update with 10 Hz Unity–ROS2 communication, while experiments confirmed functional execution and message exchange rather than detailed runtime performance. The evaluation also distinguishes trajectory-planning success from complete dual-arm task success.

  • Simulation-time configuration: 50 Hz physics updates ran against 10 Hz Unity–ROS2 communication, yielding approximately five physics steps per communication interval.The fixed physics timestep was 0.02 s, and multiple physics updates could occur between consecutive messages.
  • Runtime validation: The six-row greenhouse experiments maintained functional task execution and continuous ROS2–Unity message exchange with multimodal sensing enabled.RGB-D, LiDAR, IMU, joint-state feedback, and TF publication were active during navigation and harvesting.
  • Runtime validation: No exact mean FPS, minimum FPS, or frame-time distribution was reported because no dedicated frame-rate logging experiment was performed.The available results do not establish detailed rendering-performance behavior.
  • Evaluation scope: The experiments evaluated closed-loop functionality rather than superiority of a navigation or manipulation algorithm.The reported workflow verifies integration and execution, not comparative algorithmic performance.
  • Evaluation scope: A 96% planning success rate exceeded the 86% complete-task success rate because execution failures can occur after trajectory generation.Attachment, handover, release, and box placement may fail even when all required trajectories are generated.

5. Discussion

The discussion characterizes Agri-Sim as a functionally integrated greenhouse mobile-manipulation platform while defining important boundaries on task interpretation, sensing fidelity, communication measurement, and scalability. Current results support workflow verification, not physical realism or broad runtime benchmarking.

  • Integration findings: Agri-Sim consistently connects communication interfaces, coordinate frames, planning scenes, and command execution for greenhouse mobile manipulation.The platform provides modular closed-loop integration between a Unity greenhouse and external ROS2 navigation and manipulation modules.
  • Evaluation boundaries: The 45/50 navigation outcome supports functional integration but is not a precise localization benchmark.A fixed 250 mm goal radius and 120 s time limit define binary success, while positioning-error magnitudes and historical trajectories were not retained.
  • Simulation fidelity: The crop models reproduce visual appearance and collision geometry but omit deformability, fruit damage, force-dependent detachment, and detailed grasp-contact mechanics.Harvesting therefore assesses logical and kinematic execution rather than full physical interaction fidelity.
  • Simulation fidelity: Virtual sensors approximate their physical counterparts, including simplified LiDAR scanning and RGB-D behavior without motion blur, reflective artifacts, or missing-depth effects.The ray-casting LiDAR does not reproduce the MID-360’s complete non-repetitive scan pattern, and the D435i profile omits several real-sensor effects.
  • Runtime and scalability: Communication was functionally verified at 10 Hz against 50 Hz physics updates, but latency, jitter, bandwidth, and packet loss were not measured.Accordingly, Agri-Sim is not evaluated here as a network-performance benchmark.
  • Runtime and scalability: Only six active crop rows were evaluated; the 160-row scene lacked measurements of frame rate, real-time factor, memory, or communication load.Detailed trial-level statistics and Sim-to-Real performance also remain unestablished because physical-robot experiments were outside scope.

6. Conclusion

The paper concludes that Agri-Sim integrates configurable greenhouse simulation, multimodal sensing, mobile dual-arm execution, and ROS2-based control. Its experiments demonstrate closed-loop functionality across navigation, planning, and harvesting, while future work targets realism, measurement, scalability, and Sim-to-Real evaluation.

  • Conclusion: Agri-Sim integrates a configurable greenhouse, mobile dual-arm robot, multimodal virtual sensors, and bidirectional ROS2–MoveIt 2 execution.The system supports ROS2 navigation commands and MoveIt 2-generated manipulator trajectories.
  • Conclusion: Across 50 trials per task, navigation succeeded in 45 cases, trajectory planning in 48, and complete dual-arm harvesting in 43.The corresponding success rates were 90%, 96%, and 86%, respectively.
  • Conclusion: These results demonstrate closed-loop integration of virtual sensing, navigation, collision-aware planning, and coordinated dual-arm execution.The conclusion frames the findings as functionality verification rather than algorithmic superiority.
  • Future work: Future work will add deformable-crop and force-aware contact models, calibrate virtual sensors, retain trial-level data, and evaluate communication, runtime, and scalability.Domain randomization, parallel learning interfaces, and physical-robot experiments are planned for quantitative Sim-to-Real evaluation.

Data and Code Availability

The source code, Unity scenes, ROS2 interface definitions, and detailed experimental files are not publicly available at present. Research access may be requested from the corresponding authors.

  • Availability: Source code, Unity scenes, ROS2 interface definitions, and detailed experimental files are currently unavailable publicly.Reasonable research-access requests may be directed to the corresponding authors.

CRediT Authorship Contribution Statement

The CRediT statement assigns methodological, investigative, software, visualization, writing, conceptual, supervisory, administrative, and funding roles across the authors.

  • Shuhan Shi contributed methodology, software, investigation, visualization, and writing.
  • Zhenfeng Xue contributed conceptualization, methodology, and supervision.
  • Minghao Mei contributed investigation, while Chao Zheng contributed visualization and methodology.
  • Nan Li contributed visualization, methodology, and software.
  • Zhonghua Miao contributed conceptualization, project administration, and funding acquisition.
Loading 2608.29100v1…