Source-linked AI summary
Evaluating Mesh Reconstruction Methods for Crop Phenotyping
Karanvir Singh, Theo Morales, Binh-Son Hua, Mukesh Saini
TL;DR
Remote crop phenotyping needs accessible 3D digitization because specialists may not be available on-site and meshes must preserve crop detail efficiently. The paper evaluates seven mesh reconstruction pipelines using qualitative and quantitative measures, finding GGGS, PGSR, and 2DGS preferable, with GGGS about 27% ahead of 2DGS on a five-dimensional radar chart.
Problem
Remote crop phenotyping is difficult because specialists may not be available on-site, while useful meshes must retain detailed crop appearance and morphology.
Method
The study benchmarks seven mesh reconstruction pipelines on crop data using four quantitative metrics and a phenotyping-focused user study.
Results
GGGS, PGSR, and 2DGS are preferable to the other pipelines, and GGGS beats 2DGS by about 27% on a radar chart spanning five dimensions.
Takeaways & Limitations
GGGS is the most preferred pipeline overall, while 2DGS offers greater storage efficiency for temporal phenotyping.
Takeaways & Limitations
All evaluated pipelines have notable visual defects, so the study indicates scope remains for a perfect 3D reconstruction pipeline.
Abstract
from arXiv · showhide
Phenotyping an agricultural crop is crucial for studying its entire life cycle, as it provides vital insights to improve yield and, ultimately, food production. Doing the same for crops grown on remote sites is a challenge for the specialists who cannot be available on-site. 3D reconstruction techniques offer a promising solution to this problem by enabling crop digitization, allowing specialists to access the resulting 3D crop models from anywhere at any time. In this work, we evaluate recent 3D reconstruction pipelines for crop phenotyping. We focus on 7 mesh reconstruction pipelines and measure the fidelity and consistency of their outputs qualitatively and quantitatively. Our results suggest that the meshes produced by the GGGS, PGSR, and 2DGS are preferable to the other pipelines, owing to their quantitative metrics and visually pleasing outputs. The GGGS pipeline is better than the second-best pipeline (2DGS) by about 27\% on the radar chart with 5 dimensions, namely, User ratings, Chamfer distance, LPIPS, PSNR, and SSIM.
1. Introduction
Crop phenotyping supports crop-health assessment and applications including breeding and management, but remote inspection requires accurate, efficient 3D mesh digitization. This study addresses the lack of experimental agricultural comparisons by benchmarking seven mesh pipelines, a new dataset, and an evaluation protocol.
- 1. Introduction: Crop phenotyping identifies observable properties across a crop’s life cycle to help deduce crop health and support breeding, management, and processing.
- 1. Introduction: Remote visual inspection is difficult because experts may not be physically available on-site, motivating accurate crop digitization for remote work.
- 1. Introduction: Phenotyping meshes must preserve photometric and morphological details, including color, leaf count, and the complete shoot system, while remaining efficient to load, render, and store.
- 1. Introduction: Seven mesh reconstruction pipelines are experimentally compared for crop phenotyping, addressing a gap in agricultural benchmarking studies.The evaluated pipelines are Alicevision, SuGaR, 3DGS-to-PC, NeRF2Mesh, 2DGS, PGSR, and GGGS.
- 1. Introduction: The study contributes the Cauliflower-13 dataset, an up-to-date traditional and state-of-the-art evaluation, and a documented protocol for future research.The dataset records a cauliflower plant for 13 consecutive days using RGB images from 120 consistently maintained camera angles per day.
2. Background
3D reconstruction pipelines span traditional geometry and image-processing methods, deep-learning methods, and Gaussian Splatting, which combines elements of both paradigms. Their outputs include point clouds, meshes, radiance fields, and splats, with this study focusing on meshes.
- 2. Background: Traditional pipelines use camera calibration, pose estimation, feature matching, and geometric calculations, whereas deep-learning pipelines learn mappings from images to 3D scenes directly.
- 2. Background: Traditional pipelines include active sensor-based reconstruction and passive camera-based reconstruction using 3D computer vision techniques.
- 2. Background: NeRF models learn radiance fields that capture view-dependent lighting effects, but NeRF-based pipelines suffer from long training times and slow rendering speeds.
- 2. Background: 3D Gaussian Splatting uses Gaussian primitives initialized by traditional pipelines and optimized with deep-learning approaches, forming a middle paradigm between traditional and deep-learning methods.
3. 3D Mesh Reconstruction Pipelines
The study compares seven mesh-producing 3D reconstruction pipelines, spanning traditional, NeRF-based, and Gaussian-splatting approaches. The methods use different geometry, appearance, regularization, and mesh-extraction strategies.
- Seven 3D reconstruction pipelines are compared because the work focuses on mesh representations and their application-oriented capabilities.
- 3.1. Alicevision Meshroom: Alicevision reconstructs meshes through feature extraction, matching, structure-from-motion, depth estimation, point-cloud fusion, and Delaunay triangulation.
- 3.2. 3DGS-to-PC: 3DGS-to-PC samples dense point clouds from Gaussian splats and converts them into meshes without requiring retraining.
- 3.4. 2D Gaussian Splatting: 2DGS uses planar Gaussians, depth-distortion and normal-consistency regularization, then extracts meshes by fusing rendered depth maps with TSDF.
- 3.5. NeRF2Mesh: NeRF2Mesh learns geometry and view-dependent appearance before refining a coarse Marching Cubes mesh through joint differential rendering.
- 3.6. PGSR: PGSR flattens each Gaussian along its minimum-scale axis and optimizes flattening, photometric, and geometric losses to improve depth accuracy and global geometry preservation.
- 3.7. GGGS: GGGS models Gaussians as stochastic solids with smooth transmission, combines photometric, consistency, and multi-view losses, and extracts meshes using TSDF fusion.
4. Evaluation
The evaluation examines pipeline outputs using both quantitative and qualitative analyses. It covers the dataset, metrics, quantitative evaluation, user study, and result analysis.
- The pipelines’ outputs were evaluated quantitatively and qualitatively across dataset, metrics, quantitative evaluation, user study, and result analysis.
4.1. Dataset
Cauliflower-13 is a publicly released, multi-day, multi-view image dataset designed to benchmark 3D reconstruction methods for crop phenotyping.
- Cauliflower-13 contains a potted cauliflower photographed from multiple views on each of 13 days, with 120 images captured per day.
- The acquisition setup used a turntable, tripod, smartphone camera, and white background cloth with repeated captures at different height levels.
- The dataset is publicly released after earlier preliminary use with only Alicevision for crop-asset creation and virtual-reality visualization.
4.2. Metrics
The evaluation uses four metrics that assess geometric similarity and rendered-image quality against COLMAP or pose-matched original-image ground truth.
- Four metrics are used: Chamfer distance, PSNR, SSIM, and LPIPS.
- Chamfer Distance: Chamfer distance measures similarity between two point clouds and is computed against a COLMAP dense reconstruction.
- PSNR: PSNR measures reconstructed-image fidelity from pixelwise mean squared error and is averaged over all poses for each mesh.
- SSIM: SSIM evaluates structural similarity through inter-pixel dependencies, averaging block-level indices across images and poses.
- LPIPS: LPIPS measures perceptual image similarity using features extracted by a pretrained AlexNet backbone.
4.3. Quantitative Evaluation
Seven pipelines were evaluated on 13 image sets using Chamfer distance, PSNR, SSIM, and LPIPS. GGGS had the highest PSNR, while Alicevision had the lowest Chamfer distance and 2DGS was competitive on Chamfer distance.
- The four quantitative metrics were computed across 13 samples using common reconstruction and rendering procedures.Chamfer distance used mesh vertices against dense-COLMAP ground truth, while the other metrics used meshes rendered from the original image poses.
- Alicevision produced the lowest Chamfer distance because ICP alignment followed dense reconstruction.2DGS had lower Chamfer distance than all pipelines except NeRF2Mesh for some samples.
- GGGS achieved the highest PSNR values, followed by PGSR, 2DGS, and SuGaR.PSNR values were low overall because rendered and real-scene lighting conditions differed.
4.4. User Study
The user study asked 32 participants to rank seven mesh pipelines across visual crop-phenotyping criteria. GGGS received the highest overall sentiment, while artifact judgments favored GGGS, NeRF2Mesh, and SuGaR.
- Thirty-two participants interactively ranked seven meshes on leaf color, vein visibility, edge artifacts, soil details, and stem and branch realism.The study used browser-based mesh viewing with interactive controls.
- The study also compared structural and perceptual metrics, with GGGS highest on SSIM and lowest on LPIPS.Lower LPIPS indicates better perceptual similarity, while higher SSIM indicates stronger structural similarity.
- Ranking responses were converted to integer sentiment values from 1 to 7, with higher values indicating more positive sentiment.
- Most participants did not find leaf-edge artifacts in GGGS, NeRF2Mesh, and SuGaR, whereas they did for the other pipelines.
- GGGS ranked highest in overall user-study sentiment, followed by PGSR, 2DGS, and Alicevision.The final rank averaged the ranking aspects across all seven methods.
4.5. Result Analysis
Across five preference dimensions, GGGS ranked first, followed by 2DGS and PGSR, while visual inspection revealed distinct defects and trade-offs across pipelines.
- Observations: GGGS ranked first overall, followed by 2DGS, PGSR, Alicevision, NeRF2Mesh, SuGaR, and 3DGS-to-PC.The ranking combined user-study results with Chamfer distance, LPIPS, SSIM, and PSNR rankings using radar-chart area.
- Observations: 2DGS and PGSR differed by metric: 2DGS was better on Chamfer distance, LPIPS, and SSIM, whereas PGSR led on PSNR and user ratings.
- Observations: 3DGS-to-PC produced poor meshes despite strong point-cloud reconstruction, because Chamfer distance evaluates point clouds rather than complete mesh quality.Its Chamfer distance was better than GGGS, PGSR, and NeRF2Mesh, making it useful when only a quickly reconstructed point cloud is required.
- Visual Results: 2DGS generated lighter meshes than GGGS, enabling all 13 temporal meshes to load simultaneously and supporting inspection of crop growth over time.GGGS otherwise received strong ratings but produced larger meshes and granular surface artifacts.
- Visual Defects: No pipeline produced a perfect plant mesh: GGGS showed granules, 2DGS and PGSR contour patterns, Alicevision edge problems, NeRF2Mesh holes, SuGaR seams, and 3DGS-to-PC poor structure.
5. Conclusion
The study evaluates seven mesh reconstruction pipelines on cauliflower phenotyping data using quantitative metrics and a user study. Gaussian Splatting methods performed best overall, with GGGS leading but 2DGS offering greater storage efficiency.
- Seven pipelines were evaluated on the Cauliflower-13 dataset using Chamfer distance, PSNR, LPIPS, SSIM, and a phenotyping-focused user study.
- Gaussian Splatting methods outperformed NeRF-based and traditional photogrammetry-based methods, with GGGS preferred overall.
- GGGS beat 2DGS, the second-best pipeline, by around 27% on the five-dimension radar chart.
- 2DGS used less storage space than GGGS, an important consideration for temporal phenotyping.
- All evaluated pipelines had notable defects, leaving scope for a perfect 3D reconstruction pipeline.