Source-linked AI summary
Visual-SLAM for the detection of hidden tomatoes in greenhouses by Hierarchical Localization and GLOMAPfor robotized harvesting
Fernando Cañadas-Aránega, José C. Moreno, José L. Blanco-Claraco, Francisco Rodríguez
TL;DR
Greenhouse crop monitoring needs accurate 3D fruit localization, while conventional 2D vision and high-performance sensors have important limitations. The paper develops a low-cost monocular Visual-SLAM pipeline that integrates HLoc with GLOMAP for tomato-cluster reconstruction from ROS2-acquired RGB images. The reconstructed model correctly characterizes occluded tomatoes and achieves small geometric errors against manual measurements.
Problem
Greenhouse fruit monitoring requires three-dimensional localization, but traditional 2D vision can be insufficient and many fruits remain occluded to existing technologies.
Method
The paper combines ROS2 RGB acquisition from a monocular camera with HLoc hierarchical matching and GLOMAP Structure-from-Motion for 3D tomato-cluster reconstruction.
Results
The reconstructed models match manual measurements with approximately ±0.2 mm diameter error, ±0.4 mm centroid-position error, and ±0.75° orientation error.
Takeaways & Limitations
The methodology supports accurate 3D modeling of occluded tomato clusters and reliable estimation of picking coordinates using low-cost monocular vision.
Takeaways & Limitations
Existing technologies still face difficulty detecting and counting fruits that are occluded and inaccessible.
Abstract
from arXiv · showhide
Advanced crop monitoring inside greenhouses is becoming one of the primary objectives of research centers. High-performance sensors, such as LiDAR or stereo cameras, have traditionally been employed for this purpose, though these often have a high cost. This work proposes a Visual-SLAM system using a monocular camera, which is significantly more cost-effective and specifically tailored for agricultural applications, such as mapping tomato crops in a greenhouse. Tests were carried out on a real tomato bunch, located in the Agroconnect experimental greenhouse. A ROS 2 Humble node was developed to run on the robot in order to capture images of these crops, which were then stored for offline processing. To generate a 3D mapped model for the crop in the greenhouse, the GLOMAP mapper, based on Structure-From-Motion, was integrated with the Hierarchical Localization toolbox. This initial mapping is a foundation for future, more advanced algorithms to analyze growth patterns, and optimize agricultural management. The system leverages a hierarchical localization paradigm based on a coarse-to-fine strategy: it first performs global retrieval to generate location hypotheses, then combines local features within the identified candidate regions. The results show a correct identification of the tomato cluster, correctly characterising the tomato that is occluded and inaccessible by classical vision technologies. The reconstructed 3D model was further validated against manual ground-truth measurements of fruit size, centroid position, and orientation, confirming the geometric accuracy of the proposed low-cost monocular pipeline.
1. Introduction
Greenhouse robotics must operate in less controlled environments while locating fruit in three dimensions, including occluded tomatoes. This study investigates a low-cost monocular Visual-SLAM pipeline combining HLoc and GLOMAP for greenhouse crop monitoring.
- 1. Introduction: Greenhouse environments require task-specific robotic systems that can navigate and support complex operations such as automated harvesting.The study frames sensor selection and autonomous operation as central challenges in greenhouse robotics.
- 1. Introduction: 2D cameras provide insufficient information for reliable fruit identification and three-dimensional localization.Accurate environmental mapping is presented as essential for determining fruit positions before detection and classification.
- 1. Introduction: Occluded fruits remain difficult to count and detect with existing vision technologies, motivating methods that estimate occlusions.The cited RGB-D, LiDAR, and detection approaches do not fully resolve hidden tomatoes.
- 1. Introduction: The proposed approach combines hierarchical localization with GLOMAP Structure-from-Motion to reconstruct tomato crops from sequences of low-cost 2D images.HLoc supplies feature matching and GLOMAP performs larger-scale 3D reconstruction using depth-oriented features.
- 1. Introduction: The work records RGB imagery from a real greenhouse tomato cluster using ROS2 Humble and an Intel RealSense D435i camera for offline processing.Only the camera’s RGB information is used in the described recording setup.
2. Materials and methods
The experimental system records greenhouse tomato imagery with a monocular RGB camera and processes it through hierarchical localization and global Structure-from-Motion. The methods cover the greenhouse setting, ROS2 acquisition, HLoc candidate matching, and GLOMAP reconstruction.
- 2. Materials and methods: The experiments use an Agroconnect greenhouse with organized aisles, central and secondary paths, and tomato plants cultivated in north–south rows.The facility includes 2-meter central access and secondary paths for mobile robot navigation.
- 2. Materials and methods: Only RGB images from the Intel RealSense D435i are used, despite the camera integrating RGB-D and inertial sensing.The camera provides an approximately 86° × 57° field of view and capture rates up to 90 Hz depending on resolution.
- 2. Materials and methods: ROS2 provides the modular infrastructure for integrating localization, geometric reconstruction, trajectory planning, and actuation components.The system runs on Linux Ubuntu 22.04 with ROS2 Humble and supports communication among robotic components.
- 2. Materials and methods: HLoc first retrieves candidate images from a georeferenced database, then applies local point correspondence to estimate the robot’s 6D pose.The coarse-to-fine process reduces the search space before local feature matching.
- 2. Materials and methods: GLOMAP globally estimates camera trajectories from image correspondences using rotation averaging, translation averaging, and multi-view triangulation.This global approach estimates camera poses together rather than adding images incrementally.
3. Result and discussion
The study reconstructs a tomato cluster from monocular RGB imagery using HLoc and GLOMAP, then refines and validates the resulting 3D model against manual measurements.
- Data acquisition: 1600 frames were extracted from ROS 2 rosbag recordings and paired through exhaustive image combinations for reconstruction.The frames were captured from accessible robot positions while focusing on tomato bunches.
- HLoc and GLOMAP pipeline: GLOMAP consumed a COLMAP image dataset and pairs file, while HLoc selected coincident points and estimated image translation between frames.The resulting database.bin file and images were then used for 3D reconstruction.
- Model reconstruction: The reconstruction produced a camera trajectory and point cloud containing the tomato cluster alongside leaves, greenhouse structures, and other crop components.RANSAC filtering and CloudCompare manual cleaning were applied to improve the acquired data.
- Validation: The reconstructed model was compared with manually measured tomato diameter, centroid position, and orientation using the representative cluster.The reference measurements used a digital caliper and a junction between two upper tomatoes on the main branch.
- Validation: ±0.2 mm diameter, ±0.4 mm centroid-position, and ±0.75° orientation errors were obtained against manual ground-truth measurements for all tomatoes.The comparison used measurements from the real tomato cluster and the reconstructed point cloud.
4. Conclusion
The study concludes that integrating HLoc and GLOMAP with monocular RGB imaging and ROS 2 can reconstruct tomato-cluster morphology under occlusion and variable lighting. Preprocessing, point-cloud refinement, and ground-truth validation supported accurate picking-coordinate estimation with reduced hardware requirements.
- HLoc and GLOMAP successfully reconstructed tomato-cluster morphology under occlusions and variable lighting within a Structure-from-Motion framework.
- RANSAC filtering followed by CloudCompare denoising improved point-cloud quality while preserving relevant structures and removing visual artifacts.
- Ground-truth validation confirmed geometric fidelity, with errors of only a few millimeters in diameter and centroid position and a few degrees in orientation.
- The methodology demonstrates accurate 3D modeling with low-cost monocular vision, reducing hardware requirements and supporting scalable robotic perception in commercial greenhouses.