PaperlayerView exact revision on arXiv

Source-linked AI summary

Visual-SLAM for the detection of hidden tomatoes in greenhouses by Hierarchical Localization and GLOMAPfor robotized harvesting

Fernando Cañadas-Aránega, José C. Moreno, José L. Blanco-Claraco, Francisco Rodríguez

arXiv:2609.11766v1cs.RO

TL;DR

Greenhouse crop monitoring needs accurate 3D fruit localization, while conventional 2D vision and high-performance sensors have important limitations. The paper develops a low-cost monocular Visual-SLAM pipeline that integrates HLoc with GLOMAP for tomato-cluster reconstruction from ROS2-acquired RGB images. The reconstructed model correctly characterizes occluded tomatoes and achieves small geometric errors against manual measurements.

  • Problem

    Greenhouse fruit monitoring requires three-dimensional localization, but traditional 2D vision can be insufficient and many fruits remain occluded to existing technologies.

    Source: §1 · pp. 2–3

  • Method

    The paper combines ROS2 RGB acquisition from a monocular camera with HLoc hierarchical matching and GLOMAP Structure-from-Motion for 3D tomato-cluster reconstruction.

    Source: §1 · p. 3, §3.2 · p. 10

  • Results

    The reconstructed models match manual measurements with approximately ±0.2 mm diameter error, ±0.4 mm centroid-position error, and ±0.75° orientation error.

    Source: §3.2 · p. 12

  • Takeaways & Limitations

    The methodology supports accurate 3D modeling of occluded tomato clusters and reliable estimation of picking coordinates using low-cost monocular vision.

    Source: §3.2 · p. 12, §4 · pp. 12–13

    Existing technologies still face difficulty detecting and counting fruits that are occluded and inaccessible.

    Source: §1 · p. 3

Abstract

from arXiv · show

Advanced crop monitoring inside greenhouses is becoming one of the primary objectives of research centers. High-performance sensors, such as LiDAR or stereo cameras, have traditionally been employed for this purpose, though these often have a high cost. This work proposes a Visual-SLAM system using a monocular camera, which is significantly more cost-effective and specifically tailored for agricultural applications, such as mapping tomato crops in a greenhouse. Tests were carried out on a real tomato bunch, located in the Agroconnect experimental greenhouse. A ROS 2 Humble node was developed to run on the robot in order to capture images of these crops, which were then stored for offline processing. To generate a 3D mapped model for the crop in the greenhouse, the GLOMAP mapper, based on Structure-From-Motion, was integrated with the Hierarchical Localization toolbox. This initial mapping is a foundation for future, more advanced algorithms to analyze growth patterns, and optimize agricultural management. The system leverages a hierarchical localization paradigm based on a coarse-to-fine strategy: it first performs global retrieval to generate location hypotheses, then combines local features within the identified candidate regions. The results show a correct identification of the tomato cluster, correctly characterising the tomato that is occluded and inaccessible by classical vision technologies. The reconstructed 3D model was further validated against manual ground-truth measurements of fruit size, centroid position, and orientation, confirming the geometric accuracy of the proposed low-cost monocular pipeline.

1. Introduction

Greenhouse robotics must operate in less controlled environments while locating fruit in three dimensions, including occluded tomatoes. This study investigates a low-cost monocular Visual-SLAM pipeline combining HLoc and GLOMAP for greenhouse crop monitoring.

1. Introduction

  • Greenhouse environments require task-specific robotic systems that can navigate and support complex operations such as automated harvesting.The study frames sensor selection and autonomous operation as central challenges in greenhouse robotics.Source: §1 · p. 2
  • 2D cameras provide insufficient information for reliable fruit identification and three-dimensional localization.Accurate environmental mapping is presented as essential for determining fruit positions before detection and classification.Source: §1 · p. 2
  • Occluded fruits remain difficult to count and detect with existing vision technologies, motivating methods that estimate occlusions.The cited RGB-D, LiDAR, and detection approaches do not fully resolve hidden tomatoes.Source: §1 · p. 3
  • The proposed approach combines hierarchical localization with GLOMAP Structure-from-Motion to reconstruct tomato crops from sequences of low-cost 2D images.HLoc supplies feature matching and GLOMAP performs larger-scale 3D reconstruction using depth-oriented features.Source: §1 · pp. 3–4
  • The work records RGB imagery from a real greenhouse tomato cluster using ROS2 Humble and an Intel RealSense D435i camera for offline processing.Only the camera’s RGB information is used in the described recording setup.Source: §1 · p. 3

2. Materials and methods

The experimental system records greenhouse tomato imagery with a monocular RGB camera and processes it through hierarchical localization and global Structure-from-Motion. The methods cover the greenhouse setting, ROS2 acquisition, HLoc candidate matching, and GLOMAP reconstruction.

2. Materials and methods

  • The experiments use an Agroconnect greenhouse with organized aisles, central and secondary paths, and tomato plants cultivated in north–south rows.The facility includes 2-meter central access and secondary paths for mobile robot navigation.Source: §2.1.1 · pp. 4–5
  • Only RGB images from the Intel RealSense D435i are used, despite the camera integrating RGB-D and inertial sensing.The camera provides an approximately 86° × 57° field of view and capture rates up to 90 Hz depending on resolution.Source: §2.1.2 · p. 6
  • ROS2 provides the modular infrastructure for integrating localization, geometric reconstruction, trajectory planning, and actuation components.The system runs on Linux Ubuntu 22.04 with ROS2 Humble and supports communication among robotic components.Source: §2.1.3 · pp. 6–7
  • HLoc first retrieves candidate images from a georeferenced database, then applies local point correspondence to estimate the robot’s 6D pose.The coarse-to-fine process reduces the search space before local feature matching.Source: §2.2.1 · p. 7
  • GLOMAP globally estimates camera trajectories from image correspondences using rotation averaging, translation averaging, and multi-view triangulation.This global approach estimates camera poses together rather than adding images incrementally.Source: §2.2.2 · pp. 7–8

3. Result and discussion

The study reconstructs a tomato cluster from monocular RGB imagery using HLoc and GLOMAP, then refines and validates the resulting 3D model against manual measurements.

Data acquisition

  • 1600 frames were extracted from ROS 2 rosbag recordings and paired through exhaustive image combinations for reconstruction.The frames were captured from accessible robot positions while focusing on tomato bunches.Source: §3.1 · p. 8

HLoc and GLOMAP pipeline

  • GLOMAP consumed a COLMAP image dataset and pairs file, while HLoc selected coincident points and estimated image translation between frames.The resulting database.bin file and images were then used for 3D reconstruction.Source: §3.2 · pp. 8–10

Model reconstruction

  • The reconstruction produced a camera trajectory and point cloud containing the tomato cluster alongside leaves, greenhouse structures, and other crop components.RANSAC filtering and CloudCompare manual cleaning were applied to improve the acquired data.Source: §3.2 · pp. 10–11, Figure 7 · p. 10

Validation

  • The reconstructed model was compared with manually measured tomato diameter, centroid position, and orientation using the representative cluster.The reference measurements used a digital caliper and a junction between two upper tomatoes on the main branch.Source: §3.2 · pp. 11–12, Table 1 · p. 12
  • ±0.2 mm diameter, ±0.4 mm centroid-position, and ±0.75° orientation errors were obtained against manual ground-truth measurements for all tomatoes.The comparison used measurements from the real tomato cluster and the reconstructed point cloud.Source: §3.2 · p. 12

4. Conclusion

The study concludes that integrating HLoc and GLOMAP with monocular RGB imaging and ROS 2 can reconstruct tomato-cluster morphology under occlusion and variable lighting. Preprocessing, point-cloud refinement, and ground-truth validation supported accurate picking-coordinate estimation with reduced hardware requirements.

  • HLoc and GLOMAP successfully reconstructed tomato-cluster morphology under occlusions and variable lighting within a Structure-from-Motion framework.Source: §4 · p. 12
  • RANSAC filtering followed by CloudCompare denoising improved point-cloud quality while preserving relevant structures and removing visual artifacts.Source: §4 · p. 13
  • Ground-truth validation confirmed geometric fidelity, with errors of only a few millimeters in diameter and centroid position and a few degrees in orientation.Source: §4 · p. 13
  • The methodology demonstrates accurate 3D modeling with low-cost monocular vision, reducing hardware requirements and supporting scalable robotic perception in commercial greenhouses.Source: §4 · p. 13

Related papers

Loading 2609.11766v1…