PaperlayerView exact revision on arXiv

Source-linked AI summary

Automated Maize Ear Phenotyping Using 3D Reconstructions

Ritwesh A. Kumar, Som Tripathi, Peja Matthews, Srikar Reddy, Talukder Zaki Jubery, Patrick Schnable, Adarsh Krishnamurthy, Baskar Ganapathysubramanian

arXiv:2609.01921v1cs.CV

TL;DR

Breeding programs need measurements of genetically important maize kernel traits, but manual measurement cannot keep pace with their volume. This paper presents an automated pipeline for extracting traits from 3D ear point clouds; on held-out ears, row number was within ±2 rows for 95.2% of ears.

  • Problem

    Genetically variable kernel traits are associated with yield-related loci, yet existing 3D approaches require specialist instruments or recover ear-level rather than kernel-level traits.

    Source: §1 · pp. 2, 4

  • Method

    The paper develops an automated pipeline that recovers eleven count, shape, color, and packing traits from 360° video without supervised crop-specific models.

    Source: §1 · p. 4

  • Results

    95.2% of held-out ears (160/168) had KRN predicted within ±2 rows.

    Source: §3.3 · p. 23

  • Takeaways & Limitations

    The reported traits are recoverable from 360° video alone at throughput compatible with breeding-scale deployment.

    Source: §1 · p. 4

    Surface occlusion hides kernel contact boundaries and embedded proximal surfaces, so volume proxies and bounding boxes are relative descriptors rather than absolute measurements.

    Source: §4 · p. 34

Abstract

from arXiv · show

Maize kernel traits such as row number, kernels per row, and kernel size vary largely for genetic reasons and are consistently associated with regions of the genome that influence yield. Manual measurement of these traits, however, cannot keep pace with the volume of maize generated in a breeding program. To address this, we developed and validated a fully automated pipeline for extracting these traits from 3D point clouds of corn ears, built on a recently developed video-to-point-cloud platform. Raw video frames are processed through COLMAP and NeRF, the ear is isolated via density-based separation, and the point cloud is distance-calibrated to physical units. The calibrated ear point cloud was Z-axis aligned via PCA and cylindrically unwrapped to a 2D image. We enhanced contrast and performed zero-fine-tuning instance segmentation using Cellpose-SAM. A triple-juxtaposed unwrap strategy was used to prevent double-counting at the seam. The pipeline achieved kernel count R^2 = 0.921 (MAPE = 10.33%) and kernel row number within +-2 rows for 95.2% of ears (MAE = 0.75 rows) on a 168-ear held-out set from the 268-ear labeled dataset. The resulting multi-trait dataset has known genotype identity for each ear, positioning it for phenotype-to-genotype association analyses.

Plain Language Summary

This section defines abbreviations for image-processing methods, kernel traits, and evaluation metrics.

Abbreviations

  • COLMAP, NeRF, PCA, CLAHE, and CPSAM denote point-cloud reconstruction, alignment, contrast enhancement, and segmentation methods.COLMAP is Structure-from-Motion and Multi-View Stereo pipeline; NeRF is Neural Radiance Field; PCA is Principal Component Analysis; CLAHE is Contrast Limited Adaptive Histogram Equalization; CPSAM is Cellpose-SAM.Source: Abbreviations · p. 2
  • KC, KPR, and KRN abbreviate Kernel Count, Kernels Per Row, and Kernel Row Number.Other defined traits include SA (Kernel Surface Area), Vol (Kernel Convex Hull Volume Proxy), Pack (Kernel Packing Traits), and Hue Circular Mean (Kernel Hue).Source: Abbreviations · p. 2
  • MAE, MAPE, RMSE, and Signed ME name error metrics, while GT denotes Ground Truth and SD denotes Standard Deviation.The glossary also defines EMD as Earth-Mover Distance and KNN as K Nearest Neighbor.Source: Abbreviations · p. 2

1 Introduction

This study develops a fully automated pipeline that extracts ear- and kernel-level traits from 3D maize-ear point clouds. It combines training-free segmentation, multi-tier validation, and a genotype-identified dataset intended for future phenotype-to-genotype analyses.

1 Introduction

  • Kernel traits contribute to maize yield and carry genetic variation, but manual measurement limits breeding-scale throughput and introduces observer-dependent error.KRN is described as highly heritable, whereas KPR is more sensitive to environmental stress, resource availability, and management.Source: §1 · p. 2
  • Prior 2D projections collapse radial depth, while specialist 3D systems constrain deployment and existing consumer-accessible 3D work reports ear-level traits; the study targets joint kernel-level count, shape, color, and packing traits.The reconstructed surface describes exposed kernel caps, not complete kernels.Source: §1 · pp. 3–4
  • The pipeline extends Young et al. (2026) with training-free Cellpose-SAM segmentation of calibrated, PCA-aligned and cylindrically unwrapped ears, using triple juxtaposition to count each kernel once.The workflow contrast-enhances the unwrap with CLAHE before segmentation.Source: §1 · p. 4
  • Eleven traits span kernel count, KRN, KPR, surface area, a volume proxy, packing geometry, hue, and aspect ratio, with hue and aspect ratio profiled in 10% axial-height bands.Packing traits include the number of physically touching kernel neighbors and their mean inter-kernel distances.Source: §1 · p. 4
  • Validation uses 27 synthetic ears with known geometry, six exhaustively kernel-annotated ears, and 168 held-out manually annotated ears to isolate extraction error from physical measurement error.Source: §1 · p. 4
  • Applied to 1,091 ears with known genotype identity, the pipeline produces a multi-trait dataset for future phenotype-to-genotype association analyses.Source: §1 · p. 4

2 Materials and Methods · 2.1 Preprocessing Steps · 2.2 Ear-Level Traits

The methods transform raw 360° video into 3D ear point clouds and extract ear- and kernel-level traits through automated processing. The pipeline adapts 3D reconstruction from Young et al. (2026) and adds alignment, image-based segmentation, and trait-extraction methods.

Dataset Composition

  • The study processed 1,091 genotype-identified ears, including 268 manually labeled ears split into 100 tuning and 168 held-out validation ears.A separate six-ear subset supported exhaustive kernel-level validation, and a 27-ear synthetic dataset enabled validation against known generating geometry.Source: Dataset Composition · p. 6

Z-Axis Alignment via PCA / Cylindrical Unwrapping

  • PCA aligned each ear’s long axis with Z before cylindrical unwrapping mapped the full surface to a 2D image at 0.25 mm per pixel.The unwrap retained an index map from pixels to source 3D points; the effective cylinder radius used the 98th percentile of point radii, selected on the tuning set.Source: Z-Axis Alignment via PCA · p. 6, Cylindrical Unwrapping · pp. 6–7

Contrast Enhancement via Contrast Limited Adaptive Histogram Equalization (CLAHE)

  • CLAHE enhanced local kernel-boundary visibility by equalizing the unwrapped image’s LAB luminance channel, using a 1.0 clip limit and 8 × 8 pixel tile grid.Both parameters were tuned sequentially against manual kernel counts on the 100-ear tuning set.Source: Contrast Enhancement via Contrast… · p. 8

Triple-Juxtaposed Unwrap Strategy / Segmentation via CPSAM

  • The image was tripled horizontally to prevent seam-straddling kernels from being counted twice, then segmented with zero-shot CPSAM without corn-specific fine-tuning.Only segments centered in the middle copy were retained; tuned CPSAM settings were a cell probability threshold of 2.0, flow threshold of 0.6, and minimum segment size of 150 pixels.Source: Triple-Juxtaposed Unwrap Strategy · p. 8, Segmentation via CPSAM · p. 9

Mapping 2D Segments to 3D Point Sets / Kernel Count, Kernel Row Number, and Kernels per Row

Row Count Estimation via Adaptive FFT

  • KRN estimation used the dominant FFT frequency of binned kernel-centroid angles, with a tuned 15-bin smoothing kernel and CV threshold of 1.8725.The adaptive method smoothed only when CV ≥1.8725; odd row estimates were rounded up to even, changing Signed ME from −0.08 rows to +0.26 rows on the tuning set while lowering MAE.Source: Row Count Estimation via Adaptive FFT · pp. 10–11

Row Chain Graph and Per-Row Kernel Counts

  • A degree-two angular spanning forest linked kernel centroids into row chains, reconnecting vertically discontinuous chains and recording each final chain’s size as KPR.The number of chains was fixed to the estimated KRN, so KPR accuracy depended on correct KRN estimation; an underestimated KRN can combine distinct true rows.Source: Row Chain Graph and Per-Row Kernel… · pp. 11, 13

2.3 Kernel-Level Traits

The section derives per-kernel surface area, a visible-geometry size proxy, and mean hue from reconstructed kernel point sets. It also defines area-weighted centroids for downstream trait extraction.

Kernel Surface Area

  • BPA estimates per-kernel surface area from 3D points using consistently oriented KNN normals and ball radii tied to the 98th-percentile nearest-neighbor spacing.Area-weighted 3D centroids are used in downstream computations; Figure 8 overlays them on representative ear point clouds.Source: Kernel Surface Area · pp. 13–14, Figure 8 · p. 14

Kernel Convex Hull Volume Proxy

  • The convex hull of each kernel’s mesh vertices provides a relative size descriptor of visible cap geometry, not an estimate of true physical volume.Because the kernel interior and proximal cob-facing surface are occluded, the hull closes across the unseen region.Source: Kernel Convex Hull Volume Proxy · p. 14

Mean Kernel Hue

  • Mean kernel hue is computed from point RGB values converted to HSV, using a circular mean to handle hue wraparound at 0°/360°.Hue angles are recorded for points in each kernel’s surface vertex set, Sk.Source: Mean Kernel Hue · p. 14

3D Bounding Box Aspect Ratio

This section defines kernel shape and packing traits from 3D point clouds and describes how trait profiles and validation comparisons are constructed. The aspect ratio uses ordered dimensions of each kernel’s axis-aligned bounding box, while packing traits derive from kernel-centroid neighborhoods.

3D Bounding Box Aspect Ratio

  • Each kernel’s aspect ratio (ARk) is based on the descending edge lengths of its axis-aligned bounding box, computed from its 3D segment.The bounding box captures only the exposed outer cap, so its dimensions reflect visible surface depth and provide a relative shape descriptor for line-of-sight phenotyping.Source: 3D Bounding Box Aspect Ratio · p. 15

Mean Hue and Mean Aspect Ratio vs. Height

  • Mean hue and aspect ratio profiles were computed across ten equal-height bands along the PCA-aligned ear axis, with each kernel weighted equally.Hue was averaged using circular means, whereas aspect ratio was averaged arithmetically within each band.Source: Mean Hue and Mean Aspect Ratio vs.… · p. 15

Gabriel Graph Construction; Mean Nearest-Neighbor Distance

  • A cylindrically periodic Gabriel graph on projected kernel centroids yields per-kernel neighbor counts and mean 3D distances to direct neighbors as packing measures.Three circumferential copies are used to construct candidate edges, and only middle-copy nodes are counted; the graph is independent of row detection.Source: Gabriel Graph Construction · p. 15, Mean Nearest-Neighbor Distance · p. 15

2.4 Evaluation Metrics

  • Evaluation metrics include kernel-count agreement, tolerance-based row accuracy, end-to-end runtime, circular hue error, and population summaries of aspect ratio by ear thirds.The headline KRN tolerance uses k = 2; axial aspect-ratio summaries average per-ear means within bottom, middle, and top thirds.Source: §2.4 · p. 17

Distributional Agreement via Earth-Mover Distance

  • Earth-Mover Distance (EMD) compares automated and manual KPR and Gabriel-neighbor-count distributions separately for each ear in the six-ear annotated subset.Lower EMD indicates closer agreement; zero denotes identical distributions, and values below 1.0 indicate less than one trait unit of cumulative-distribution deviation.Source: Distributional Agreement via… · p. 18

Synthetic Ground-Truth Validation Protocol

  • Validation uses 27 procedurally generated synthetic ears with known geometry and lattice-derived ground truth, spanning a 3×3×3 sweep of row number, curvature, and missing-kernel fraction.The full pipeline runs end-to-end without access to ground truth; synthetic packing-trait references use lattice adjacency rather than the pipeline’s Gabriel graph, and hue ground truth is a population mean parameter.Source: Synthetic Ground-Truth Validation… · pp. 18–19

Kernel Matching via Hungarian Assignment

  • Predicted and ground-truth kernels are paired by globally optimal Hungarian centroid assignment, then pairs beyond a density-scaled distance cutoff are rejected.Rejected pairs are scored as false positives and false negatives, preventing distant incidental matches when predicted and ground-truth kernel counts differ.Source: Kernel Matching via Hungarian Assignment · p. 19

Trait-Specific Evaluation Notes

  • Trait-specific comparisons average errors across matched pairs, align row labels independently for KPR, and compare axial hue and aspect ratio in matched 10%-height bands.Ground-truth aspect ratio uses sorted ellipsoid semi-axes, while predicted aspect ratio uses the BPA-surviving point-set bounding box; the constructions are analogous but not identical.Source: Trait-Specific Evaluation Notes · p. 20

3 Results

The results evaluate the pipeline across synthetic ground truth, a held-out breeding-scale set, and a manually annotated subset, with a separate analysis of computational throughput.

3.1 Parameter Tuning Dataset

  • The 100-ear tuning set combined manually selected challenging curved ears with random, row-number-stratified sampling, and was deliberately more challenging than the held-out set.Kernel-count parameters were tuned sequentially across unwrapping, CLAHE, and CPSAM stages; row-count parameters were optimized jointly.Source: §3.1 · pp. 20–21

3.2 Synthetic Ground-Truth Validation

  • On the 27-ear synthetic set, pooled KRN MAE was 7.26 rows and KC achieved R2 = 0.769; geometric and color traits closely matched generating parameters.The larger row-structure errors on synthetic than real ears were attributed to regular lattice harmonics: frequency selection could lock onto a harmonic rather than the true row frequency.Source: §3.2 · pp. 21–22, Table 4 · p. 22

3.3 Held-Out Set Validation and Benchmarking

  • KRN was exact for 121 of 168 held-out ears (72.0%) and within ±2 rows for 160 of 168 (95.2%), versus 63 of 100 (63.0%) and 96 of 100 (96.0%) on the tuning set.The higher exact-match rate on the held-out set reflects the tuning set’s deliberate enrichment for highly curved ears, which challenge the FFT method’s angular-regularity assumption.Source: Figure 13 · p. 23, §3.3 · p. 23
  • The pipeline offers spatial packing data and volumetric shape proxies from consumer video, but its kernel-count error is higher than that of the best-performing projection-based methods.The paper frames Table 5 as a high-level comparison, not a controlled benchmark, because imaging protocols, crop varieties, and validation datasets differ.Source: §3.3 · p. 24, Table 5 · p. 24

3.4 Manually Annotated Subset Validation

  • Across six manually annotated ears, Gabriel graph neighbor-count EMD ranged from 0.33 to 0.84 (mean 0.57), while KPR EMD ranged from 0.75 to 7.58 (mean 3.04).KPR discrepancies concentrated in Samples 4 and 6, the same ears whose KRN was underpredicted.Source: §3.4 · p. 31, Table 8 · p. 31, Table 9 · p. 32

3.5 Computational Cost and Throughput

  • On 168 held-out ears, kernel count achieved R2 = 0.921 (MAPE = 10.33%), and KRN was within ±2 rows for 95.2% of ears.Trait extraction took approximately 10 seconds per ear on one NVIDIA A100 GPU; processing all 1,091 ears required approximately 3 hours.Source: §3.5 · p. 32

4 Discussion and Conclusions

The discussion concludes that the pipeline extracts a broad set of maize ear traits from consumer-grade 3D imaging, with useful validation results but identifiable limits in row alignment, kernel counting, and generalizability. Its 1,091-ear genotype-linked dataset is positioned for future phenotype-to-genotype analyses; no association analysis was attempted.

4 Discussion and Conclusions

  • Triple-juxtaposed unwrapping prevents seam double-counting, while non-aligned kernel rows can violate the FFT’s angular-regularity assumption and cause larger KRN errors.In the six-ear annotated subset, adaptive FFT predicted exact KRN for four ears; the other two were underpredicted by two rows. KPR errors can follow from underestimated KRN because the row-chain graph merges chains to match the predicted row count, whereas Gabriel-graph neighbor counts do not depend on KRN.Source: §4 · p. 33
  • Kernel-count residuals widen with ground-truth count, and blurred boundaries in NeRF reconstructions can cause CPSAM to merge kernels or split them into multiple segments.Across five KRN classes with more than ten ears, held-out kernel-count MAPE ranged from 8.26% (KRN=14) to 14.62% (KRN=10); estimates for KRN=18 and 20 warrant caution because their sample sizes were 13 and 1 ears.Source: §4 · p. 33
  • Consumer-video phenotyping adds packing and volumetric traits beyond projection-based methods, which report higher kernel-count accuracy (Fan et al. (2026)) and lower KRN MAE (Shi et al. (2022)).Across 1,091 ears, aspect ratio peaked in the middle third of the cob, with between-band differences smaller than within-band standard deviation; hue profiles also showed ear-specific axial gradients. These axial patterns may reflect developmental or positional variation.Source: §4 · pp. 33–34
  • Interpretation is bounded by occlusion, synthetic-data abstractions, and controlled imaging conditions that may not represent all maize morphologies or field environments.On the 27-ear synthetic set, KRN and KPR MAE were 7.26 rows and 11.28 kernels, and kernel-count R2 was 0.769; idealized regularity disrupted frequency analysis, unlike biological variation in real ears. Volume proxies are relative descriptors because hidden kernel surfaces are unobserved, and their absolute fidelity against physical measurements remains untested.Source: §4 · p. 34
  • Kernel-count MAPE was 10.33%, KRN was within ±2 rows for 95.2% of ears, and packing topology had mean EMD of 0.57 neighbors across validation tiers.Applied to 1,091 genotype-identified ears, the pipeline extracted eleven ear- and kernel-level traits without task-specific training, providing a dataset for future phenotype-to-genotype association analyses; no association analysis was attempted here.Source: §4 · p. 35

ORCID

The section provides ORCID identifiers for five authors.

ORCID

  • ORCID identifiers are provided for Ritwesh A. Kumar, Talukder Zaki Jubery, Patrick Schnable, Adarsh Krishnamurthy, and Baskar Ganapathysubramanian.Ritwesh A. Kumar: 0009-0006-1956-5273; Talukder Zaki Jubery: 0000-0003-4107-1617; Patrick Schnable: 0000-0001-9169-5204; Adarsh Krishnamurthy: 0000-0002-5900-1863; Baskar Ganapathysubramanian: 0000-0002-8931-4852.Source: ORCID · p. 35

Supplementary Material · Manual Annotation Protocol

Both manually annotated datasets used base-to-tip kernel-row numbering, but differed in annotation depth and annotator count. The six-ear subset also recorded individual kernel identities and positions for exhaustive kernels-per-row and packing validation.

Manual Annotation Protocol

  • Both datasets numbered rows from the ear base toward the tip and counted rows sequentially around the full circumference, starting from an arbitrary row designated as row 1.Kernel positions within rows followed the same base-to-tip direction.Source: Manual Annotation Protocol · p. 39, Figure A.1 · p. 39
  • The arbitrary starting row did not affect reported kernel row number or kernels-per-row values, which depended on total count and per-row kernel assignment.Source: Manual Annotation Protocol · p. 39
  • Figure A.1 illustrates the numbering convention for Sample 1, with traced rows labeled from the base and kernel positions numbered toward the tip.Source: Figure A.1 · p. 39, Manual Annotation Protocol · p. 39
  • The datasets differed in annotation depth and annotator count: the 268-ear set recorded only total kernel count and row number, while the six-ear subset included individual kernel positions.The six-ear annotations supported exhaustive kernels-per-row and packing validation reported in Section 3.4.Source: Manual Annotation Protocol · p. 39
  • A single expert annotator recorded only total kernel count (KC) and kernel row number (KRN) for each ear in the 268-ear labeled dataset.Source: Manual Annotation Protocol · p. 39
  • Three independent annotators each labeled two ears in the six-ear subset, recording full per-row kernel identity and position under the shared convention.The consistent numbering made per-row identity and position unambiguous despite each ear being labeled by one individual.Source: Manual Annotation Protocol · p. 39, Figure A.1 · p. 39

Additional Distribution Plots

Additional plots complete the tracking and packing evaluation across the six-ear manually annotated subset, showing KPR and Gabriel graph neighbor-count accuracy distributions for Samples 2–6. Corresponding Earth-Mover Distances are collected in Table 9.

Additional Distribution Plots

Corn Ear 3D Reconstruction Using Stationary Camera

The section outlines a stationary-camera workflow for reconstructing a corn ear in 3D.

Corn Ear 3D Reconstruction Using Stationary Camera

360° Video Acquisition

The pipeline captured rotating maize ears on video, reconstructed the scene as a 3D point cloud, isolated the ear, and calibrated its coordinates to millimeters using the sample holder.

360° Video Acquisition

  • Each ear completed a full rotation on a motorized turntable while a stationary camera recorded continuous 360° video against a dark background.The holder’s known diameter served as the reference for later distance calibration, and extracted frames fed the reconstruction pipeline.Source: 360° Video Acquisition · p. 41, Figure A.4 · p. 42

Camera Pose Estimation via COLMAP

  • COLMAP estimated camera parameters and poses from the video frames, producing a sparse reconstruction and poses for the subsequent NeRF step.The estimated intrinsic and extrinsic parameters describe each frame’s camera configuration.Source: Camera Pose Estimation via COLMAP · p. 41

Neural Radiance Field Reconstruction

  • NeRF trained on the video frames and camera poses generated a dense scene representation that was queried to extract a point cloud.The point cloud included both the ear and turntable.Source: Neural Radiance Field Reconstruction · p. 42

Ear Isolation via Density-Based Separation

  • A density-based separation method retained the ear’s compact, high-density cluster while excluding the more sparsely distributed turntable and surrounding structures.Source: Ear Isolation via Density-Based… · p. 42

Distance Calibration

  • A known-diameter cylindrical holder supplied the metric reference for calibrating the reconstructed ear point cloud to millimeters.Its measured diameter in the point cloud was scaled to the holder’s known diameter, and the resulting factor was applied to all point coordinates.Source: Distance Calibration · p. 42

Related papers

Loading 2609.01921v1…