PaperlayerView exact revision on arXiv

Source-linked AI summary

Automated Maize Ear Phenotyping Using 3D Reconstructions

Ritwesh A. Kumar, Som Tripathi, Peja Matthews, Srikar Reddy, Talukder Zaki Jubery, Patrick Schnable, Adarsh Krishnamurthy, Baskar Ganapathysubramanian

arXiv:2609.01921v1cs.CV

TL;DR

Maize kernel traits are genetically important for yield, but manual measurement does not scale to breeding-program volumes. This paper develops and validates an automated, training-free pipeline that extracts multiple kernel and ear traits from calibrated 3D point clouds, achieving strong held-out performance and producing genotype-linked multi-trait data.

  • Problem

    Maize kernel traits carry genetically determined variance and are associated with yield-related loci, but manual measurement cannot keep pace with breeding-program scale.

  • Method

    The pipeline reconstructs calibrated 3D ear point clouds from video, aligns and unwraps them, then uses zero-fine-tuned Cellpose-SAM segmentation with seam handling to extract kernel instances and traits.

  • Results

    Kernel count reached R^2 = 0.921 and MAPE = 10.33%, while KRN was within ±2 rows for 95.2% of held-out ears; eleven traits were recovered across validation tiers.

  • Takeaways & Limitations

    The pipeline provides breeding-scale, genotype-linked phenotyping from 360° video without specialist instrumentation or supervised crop-specific models.

  • Takeaways & Limitations

    Physical occlusion hides lateral contact boundaries and proximal kernel surfaces, so convex hull volume and bounding-box metrics are relative shape descriptors and spatial proxies rather than complete radial measurements.

Abstract

from arXiv · show

Maize kernel traits such as row number, kernels per row, and kernel size vary largely for genetic reasons and are consistently associated with regions of the genome that influence yield. Manual measurement of these traits, however, cannot keep pace with the volume of maize generated in a breeding program. To address this, we developed and validated a fully automated pipeline for extracting these traits from 3D point clouds of corn ears, built on a recently developed video-to-point-cloud platform. Raw video frames are processed through COLMAP and NeRF, the ear is isolated via density-based separation, and the point cloud is distance-calibrated to physical units. The calibrated ear point cloud was Z-axis aligned via PCA and cylindrically unwrapped to a 2D image. We enhanced contrast and performed zero-fine-tuning instance segmentation using Cellpose-SAM. A triple-juxtaposed unwrap strategy was used to prevent double-counting at the seam. The pipeline achieved kernel count R^2 = 0.921 (MAPE = 10.33%) and kernel row number within +-2 rows for 95.2% of ears (MAE = 0.75 rows) on a 168-ear held-out set from the 268-ear labeled dataset. The resulting multi-trait dataset has known genotype identity for each ear, positioning it for phenotype-to-genotype association analyses.

Plain Language Summary

The paper uses abbreviations for maize ear phenotyping methods, traits, measurements, and evaluation metrics.

  • The abbreviations include COLMAP, NeRF, Cellpose-SAM, kernel count (KC), kernels per row (KPR), and kernel row number (KRN).They also include MAE, MAPE, and other imaging, geometry, and trait-extraction terms.
  • MAE denotes mean absolute error, while MAPE denotes mean absolute percentage error.
  • NeRF denotes neural radiance field and COLMAP denotes a structure-from-motion and multi-view stereo pipeline.

1 Introduction

Maize ear and kernel traits are important breeding phenotypes, but existing imaging methods do not jointly provide broad kernel-level measurements at breeding scale. This work introduces a training-free pipeline that extracts eleven traits from consumer-accessible 3D imaging and validates them across multiple datasets.

  • 1 Introduction: Maize yield-related traits such as KRN, KPR, kernel size, and kernel count show genetically determined variation and associations with yield-related genomic loci.KRN is relatively heritable, whereas KPR is more sensitive to environmental stress, resources, and management.
  • 1 Introduction: 2D single-image methods leave roughly half the kernels unobserved, while rotational and 360° systems improve surface coverage but differ in trait scope and hardware requirements.Existing approaches include symmetry-based doubling, visible-face proxies, rotating scanners, and consumer-grade cylindrical projections.
  • 1 Introduction: No existing workflow jointly measures kernel count, volumetric shape proxies, and packing traits from one consumer-accessible imaging process without supervised training.Prior 3D methods recovered kernel geometry or ear-level traits, but not this combined trait set.
  • 1 Introduction: The pipeline aligns calibrated point clouds with PCA, unwraps them cylindrically, enhances contrast with CLAHE, and performs zero-fine-tuned Cellpose-SAM kernel segmentation.A triple-juxtaposed unwrap resolves the cylindrical seam so each kernel is counted once.
  • 1 Introduction: Eleven ear- and kernel-level traits include count, KRN, KPR, surface area, volume proxy, packing geometry, hue, aspect ratio, and axial profiles.Hue and aspect ratio are additionally summarized in 10% height bands along the ear.
  • 1 Introduction: Validation uses 27 synthetic ears, six exhaustively annotated ears, and 168 held-out manually annotated ears to separate extraction error from physical measurement error.
  • 1 Introduction: The pipeline was applied to 1,091 ears with known genotype identity, producing a multi-trait dataset positioned for future phenotype-to-genotype association analyses.
  • 1 Introduction: The authors state that eleven traits spanning count, shape, color, and packing geometry are recoverable from 360° video alone without specialist instrumentation or supervised crop-specific models.The stated workflow is intended to support breeding-scale deployment.

2 Materials and Methods

The pipeline converts reconstructed maize-ear point clouds into 2D kernel instances and whole-ear traits, using calibrated 3D geometry, cylindrical unwrapping, contrast enhancement, and zero-shot segmentation. Kernel rows are estimated from centroid-angle patterns with an even-parity postprocessing rule selected on tuning data.

  • Pipeline overview: The workflow adapts video-to-point-cloud reconstruction and processes isolated ears through alignment, unwrapping, contrast enhancement, segmentation, and trait extraction.PCA aligns the ear’s principal axis with Z before cylindrical projection; 2D segments retain mappings to source 3D points.
  • Study populations: 268 manually annotated ears were split into a 100-ear tuning set and a 168-ear held-out set, within a larger population of 1,091 known-genotype ears.The unlabeled ears reflect the labor-intensive nature of manual phenotyping.
  • Kernel segmentation: Triple-juxtaposed unwrapping retains segments centered in the middle copy, preventing seam-straddling kernels from being double-counted.Three horizontally concatenated copies are segmented, and only segments with centers of mass in x ∈ [W, 2W) are retained.
  • Kernel segmentation: Zero-shot CPSAM segments individual kernels without task-specific fine-tuning on corn imagery.Its parameters were tuned sequentially on the 100-ear tuning set against manual kernel counts.
  • Trait extraction: Kernel count is obtained from retained segments, while KRN and KPR are estimated with adaptive FFT and a row chain graph, respectively.The resulting segments also provide per-kernel 3D point sets for downstream volumetric and geometric traits.
  • Row count estimation: Even-valued postprocessing improved KRN MAE on tuning data despite shifting Signed ME from −0.08 to +0.26 rows.The rule rounds odd predictions upward because cultivated maize KRN is predominantly even.

2.3 Kernel-Level Traits

Kernel-level traits are computed from reconstructed 3D point sets generated by re-projecting segmented 2D instances. The pipeline estimates surface area, centroid geometry, a visible-geometry volume proxy, and circular-mean hue.

  • Surface area: Per-kernel surface area is estimated from each kernel’s 3D point set using the Ball-Pivoting Algorithm.Surface normals use KNN search and tangent-plane propagation for consistent orientation.
  • Centroid geometry: Area-weighted 3D centroids are computed from the BPA mesh and used for downstream kernel-level computations.The centroids are overlaid on reconstructed ear point clouds in Figure 8.
  • Volume proxy: The convex hull volume proxy measures relative size from visible outer-cap geometry rather than true physical kernel volume.Occluded kernel interiors and proximal cob-facing surfaces cause the hull to close across unobserved regions.
  • Color: Kernel hue is summarized with a circular mean after converting RGB values to HSV hue angles.Circular averaging handles wraparound at 0°/360°.

3D Bounding Box Aspect Ratio

The pipeline computes kernel bounding-box aspect ratios from visible 3D segments and uses them as relative shape descriptors. These ratios are summarized across ear height and across base, middle, and tip regions to characterize axial shape variation.

  • The three axis-aligned bounding-box edge lengths are computed from each kernel’s 3D point set, sorted descending, and used to define its aspect ratio.The edge lengths are ℓ1 = xmax − xmin, ℓ2 = ymax − ymin, and ℓ3 = zmax − zmin, with ℓ(1) ≥ ℓ(2) ≥ ℓ(3).
  • Visible-surface bounding boxes capture only exposed kernel caps, so the resulting aspect ratio is a consistent relative descriptor for non-destructive, line-of-sight phenotyping.
  • Kernel centroids are divided into ten equal 10% height bands along the PCA-aligned Z-axis to compute mean aspect ratio profiles from base to tip.The mean profile is the arithmetic mean of ARk within each band, with per-band variability summarized by standard deviation over per-ear means.
  • Gabriel-graph neighbor relationships provide a complementary spatial view of local kernel packing, while mean 3D neighbor distance measures physical spacing between proximal kernels.
  • Aspect ratio is also computed separately for the base, middle, and tip thirds to capture spatial differences in kernel shape development.

3 Results

The pipeline was evaluated across synthetic, held-out, and manually annotated ears, with strong breeding-scale kernel-count and row-number performance but a known synthetic row-detection failure mode. Additional analyses characterized trait distributions, computational throughput, and comparison boundaries.

  • Validation framework: Three validation tiers tested exact synthetic ground truth, breeding-scale held-out accuracy, and kernel-resolution tracking on manually annotated ears.The framework used 27 synthetic ears, 168 held-out ears, and a six-ear manually annotated subset.
  • Synthetic ground-truth validation: 7.26 rows pooled KRN MAE occurred on the 27-ear synthetic dataset, substantially exceeding row-number error on the real held-out set.The synthetic lattice’s regularity was investigated as a source of this discrepancy.
  • Synthetic ground-truth validation: KC achieved R2 = 0.769 on the 27-ear synthetic dataset, while geometric and color traits closely agreed with their generating parameters.Row-structure traits KRN and KPR showed substantially larger synthetic-dataset errors than on the real held-out set.
  • Synthetic ground-truth validation: The FFT row detector can lock onto a harmonic: a synthetic ear with GT KRN = 12 produced Pred = 24.Perfectly regular synthetic lattices create prominent secondary harmonics that can exceed the true fundamental frequency.
  • Held-out set validation: 95.2% of held-out ears had KRN predictions within ±2 rows, with 121 of 168 ears predicted exactly.The tuning set achieved 63.0% exact and 96.0% within ±2 rows; its composition was enriched for highly curved ears.
  • Throughput and deployment: The held-out pipeline processed 1,091 reconstructed ears, achieving KC R2 = 0.921 (MAPE = 10.33%) and requiring approximately 10 seconds per ear downstream of reconstruction.Trait extraction took roughly three hours across the full 1,091-ear dataset on a single NVIDIA A100 GPU.

4 Discussion and Conclusions

The pipeline’s validation isolates seam, alignment, and packing errors while supporting multi-trait extraction from calibrated 3D ear point clouds. Applied to 1,091 ears, it produced genotype-linked phenotypes, with important limits from occlusion and imaging scope.

  • Validation and limitations: Triple-juxtaposed unwrapping eliminated seam-boundary double-counting in cylindrical projections of maize ears.This addresses a failure mode inherent to naive cylindrical projection.
  • Validation and limitations: Most row-count errors fell within two rows, while larger deviations occurred for ears with non-aligned rows that violated the FFT method’s angular-regularity assumption.The manually annotated subset showed exact KRN for four of six ears; two differed by two rows.
  • Validation and limitations: KRN underestimation merged true rows in the row-chain graph, inflating predicted kernels-per-row for the merged chain.This coupling explains the KPR disagreements concentrated in Samples 4 and 6.
  • Validation and limitations: Physical occlusion makes convex-hull volume and bounding-box measurements relative shape descriptors rather than absolute kernel-volume measurements.Absolute kernel volume requires destructive shelling or non-destructive X-ray micro-CT, and absolute fidelity remains untested against physical measurement.
  • Validation and limitations: 10.33% MAPE for kernel count, ±2 rows for KRN in 95.2% of ears, and 0.57-neighbor mean Earth-Mover Distance characterized validation across three tiers.The tiers were synthetic, manually annotated, and held-out datasets.
  • Breeding-scale implications: Applied to 1,091 ears with known genotype identity, the pipeline generated a multi-trait dataset positioned for phenotype-to-genotype association analyses.The study did not attempt association analysis.

ORCID

The section lists ORCID identifiers for the study’s named contributors.

  • Contributors: Ritwesh A. Kumar is listed with an ORCID identifier.The identifier is provided in the contributor list.
  • Contributors: Talukder Zaki Jubery is listed with an ORCID identifier.The identifier is provided in the contributor list.
  • Contributors: Patrick Schnable, Adarsh Krishnamurthy, and Baskar Ganapathysubramanian are each listed with ORCID identifiers.All three identifiers appear in the same contributor list.

Manual Annotation Protocol

Manual annotations used a consistent base-to-tip and circumferential numbering convention, with annotation depth differing between the large labeled dataset and the six-ear validation subset.

  • Numbering convention: Rows were numbered from the ear base toward the tip, with row 1 nearest the base.The convention was used across both manually annotated datasets.
  • Numbering convention: Rows were counted sequentially around the full ear circumference from an arbitrary row 1, which did not affect KRN or KPR.The starting point was treated as rotationally arbitrary.
  • Numbering convention: Kernel positions within each row were numbered in the same base-to-tip direction.The protocol was illustrated for Sample 1 and applied to all six manually annotated ears.
  • Dataset annotation scope: The 268-ear dataset recorded only total kernel count and KRN from one expert annotator.Individual kernel positions within rows were not resolved in this dataset.
  • Dataset annotation scope: Three independent annotators recorded full per-row kernel identity and position for the six-ear subset.This enabled exhaustive KPR and packing validation.

Additional Distribution Plots

Additional plots complete the six-ear manual validation by showing KPR and Gabriel graph neighbor-count accuracy distributions for Samples 2–6, with corresponding distances reported in Table 9.

  • KPR distributions: Figure A.2 presents KPR accuracy distributions for Samples 2–6.It completes the evaluation beyond the detailed treatment of Sample 1.
  • Neighbor-count distributions: Figure A.3 presents Gabriel graph neighbor-count accuracy distributions for Samples 2–6.The plots cover the remaining manually annotated ears in the six-ear subset.
  • Distance summary: Table 9 collects the corresponding Earth-Mover Distances for the KPR and neighbor-count distributions.The distributions and distances together complete the packing-performance evaluation.

Corn Ear 3D Reconstruction Using Stationary Camera

This section describes a stationary-camera workflow for reconstructing corn ears in 3D, adapted from Young et al. (2026).

  • The reconstruction workflow uses a stationary camera to produce a 3D model of the corn ear.
  • The procedure is adapted from Young et al. (2026).
  • The section outlines the steps used for corn-ear 3D reconstruction.

360° Video Acquisition

Corn ears were recorded during 360° rotation, then reconstructed into dense point clouds, separated from the turntable, and calibrated to millimeters using a known holder diameter.

  • Video acquisition: Each ear was recorded continuously through a full 360° rotation on a motorized turntable with a stationary camera.The ears were mounted against a dark background before video acquisition.
  • Video acquisition: The known diameter of the cylindrical sample holder provided the metric reference for distance calibration.The holder was visible in every frame and used to establish physical scale.
  • 3D reconstruction: COLMAP estimated camera parameters and poses from the extracted video frames, producing a sparse 3D reconstruction for NeRF.The camera poses described the camera’s position and orientation relative to the scene.
  • 3D reconstruction: NeRF used the video frames and camera poses to generate a dense 3D scene representation and extract a point cloud.The extracted scene included both the corn ear and turntable.
  • Ear isolation: Density-based separation retained the ear’s compact, high-density cluster while excluding sparser turntable and surrounding-structure points.
  • Distance calibration: The point cloud was converted to millimeters by multiplying all coordinates by the ratio of known to reconstructed holder diameter.
Loading 2609.01921v1…