Source-linked AI summary
EgoMaize: A First-Person Maize Instance Segmentation Benchmark under Severe Field Occlusion
Jiayi Li, Zihan Zhang, Erhankang Yan, Yitian Chen, Yuze Li, Chengzhang Ding, Jianxin Cao
TL;DR
Close-range maize phenotyping needs plant-level assignment of in-canopy structures, but post-seedling rows create severe overlap and hidden boundaries that can be difficult to verify. EgoMaize introduces a compact first-person benchmark and evidence-closed annotation workflow; its results show that ownership-consistent masks remain difficult for current models.
Problem
Post-seedling maize has elongated, repetitive, overlapping organs, making ownership-consistent plant masks difficult even though close-range phenotyping depends on in-canopy structures.
Source: Introduction · p. 2Method
EgoMaize provides a compact first-person maize instance segmentation dataset and labels uncertain regions as ignore while completing only locally constrained gaps supported by visual evidence.
Source: Introduction · pp. 2–3Results
Current instance segmentation models can localize maize plants to some extent, but ownership-consistent mask prediction remains difficult under elongated structures, repeated organs, same-class overlap, and ignore regions.
Source: §5 · p. 14Takeaways & Limitations
Close-range phenotyping requires plant-level association of stems, tassels, and fragmented visible regions, rather than only crop foreground detection.
Source: Introduction · p. 2, §5 · p. 14The dataset is compact and covers limited field sites, devices, and acquisition conditions; 2D images also cannot fully validate hidden plant geometry.
Source: §5 · p. 14
Abstract
from arXiv · showhide
Close-range first-person field images are important for mobile maize phenotyping because many plant-level traits depend on in-canopy structures that are difficult to ob serve from overhead views. However, post-seedling maize fields create a difficult in stance segmentation setting: stems, leaves, tassels, and neighboring plants are elon gated, repetitive, and strongly occluded. We introduce EgoMaize, a compact benchmark for first-person maize instance segmentation, where the task is to predict ownership consistent plant masks and plant-owned stem/tassel cues from close-range field images with severe same-class overlap. Existing visible-only labels can fragment one physi cal plant into disconnected supervision, while full-amodal labels may require unverifi able completion behind neighboring plants or field objects. EgoMaize therefore uses an evidence-closed annotation workflow for occluded maize regions and assigns unreli able maize regions to ignore rather than background. Baseline results show that pre trained query-based grouping, boundary refinement, and high-resolution crop refine ment help different aspects of the task, but no architecture solves the coupled chal lenges of fine structure recovery, same-class instance ownership, and occlusion reason ing; occlusion-level analysis further shows that performance decreases as plant visi bility becomes more limited. The dataset and code are publicly available at https: //github.com/JaaaaaaaD/EgoMaize.
Introduction
Close-range phenotyping needs plant-level assignment of in-canopy structures, but post-seedling maize creates severe same-class overlap that existing benchmarks do not target. EgoMaize provides a compact first-person benchmark with an evidence-closed annotation workflow and analyses of ownership-consistent segmentation.
Introduction
- The benchmark targets close-range post-seedling maize, where existing resources do not assign overlapping stems, leaves, tassels, and fragments to the correct plant instance.Plant-level phenotyping requires stable association of plant parts, not only vegetation coverage, crop/weed labels, or plant-center detection.Source: Introduction · p. 2
- EgoMaize contains 301 first-person field images, 1,276 plant instances, and 2,731 ignore regions, with plant-owned stem and tassel cues.It is described as compact and focused on a practical engineering scenario where current instance segmentation models expose failure modes.Source: Introduction · p. 2
- Visible maize regions are labeled by reliable plant ownership, uncertain visible regions go to ignore, and invisible regions are completed only for locally constrained, evidence-supported gaps.External occlusion and unverifiable hidden regions are not completed.Source: Introduction · p. 3
- The task is ownership-consistent instance delineation because one physical plant can appear as disconnected fragments while adjacent visible regions can belong to different plants.In-canopy traits such as stem diameter and ear height are difficult to observe from overhead views alone.Source: Introduction · pp. 2–3
- Baseline analyses show that approximate localization and loose instance matching are easier than high-quality ownership-consistent masks, especially as visibility becomes limited.The annotation analyses also find the evidence-closed protocol more consistent than unconstrained full-amodal completion in this field setting.Source: Introduction · p. 3
- EgoMaize is a compact egocentric benchmark for post-seedling maize instance segmentation under severe real-world field occlusion.Its evidence-driven workflow preserves continuity supported by local visual evidence while suppressing speculative completion behind external occluders.Source: Introduction · p. 3
Related Work
Prior agricultural datasets cover several perception tasks, but do not directly address first-person post-seedling maize rows with severe same-class overlap. Related annotation approaches also leave a trade-off between fragmented visible-only labels and unverifiable hidden-shape completion.
2.1 Agricultural Vision Datasets and Field Phenotyping
- Existing resources include crop–weed, overhead, seedling-stage, organ-level, and 3D benchmarks, while EgoMaize targets first-person post-seedling rows requiring ownership-aware plant instances.Ground robots and in-canopy systems collect traits such as ear height and stem diameter that are difficult to obtain from above-canopy sensing.Source: §2.1 · p. 4
2.2 Annotation under Occlusion
- Modal masks can fragment one physical object into disconnected visible parts, whereas amodal labels can make hidden boundaries subjective when real-image evidence does not constrain them.Some recent work uses 3D information to obtain more objective amodal targets [30].Source: §2.2 · p. 4
- EgoMaize uses ignore labels to exclude recognizable maize regions with uncertain ownership or unreliable boundaries from both positive instances and background.This makes ignore part of the label schema rather than merely a cleanup label.Source: §2.2 · p. 4
- Unlike compact-fruit amodal segmentation, maize annotation centers on assigning interleaved elongated plant regions to the correct instance without inventing unsupported cross-plant boundaries.Amodal tomato segmentation aims to recover complete fruit shape under occlusion.Source: §2.2 · pp. 4–5
3 EgoMaize Dataset
EgoMaize defines a first-person, post-seedling maize instance-segmentation benchmark focused on assigning overlapping, fragmented plant regions to the correct plant. Its annotation and split procedures account for uncertain ownership and varying occlusion difficulty.
3.1 Task, Challenges, and Dataset Positioning
- EgoMaize targets ownership-consistent plant masks in first-person post-seedling scenes, where severe same-class overlap makes plant ownership harder than foreground segmentation.The benchmark is positioned as complementary to datasets focused on crop–weed recognition, overhead monitoring, seedling instances, leaf annotation, or 3D geometry.Source: §3.1 · p. 5
3.2 Dataset Scope
- The dataset contains 301 close-range, field-level RGB images of post-seedling maize, with row-level and oblique views exposing dense overlap among plant organs.The release is intended as a focused finetuning and evaluation benchmark, rather than a large-scale pretraining corpus.Source: §3.2 · p. 6
3.3 Annotation Schema and Workflow
- Plant-owned stem and tassel cues are linked to their parent plant rather than treated as independent object instances.Quality control checks ownership consistency, organ-to-plant links, and whether uncertain maize regions are assigned to ignore.Source: §3.3 · pp. 6, 8
- The evidence-closed protocol labels visible maize by reliable plant ownership, completes only locally constrained gaps supported by image evidence, and assigns unreliable regions to ignore.Large hidden areas, external occlusion, long-range interruptions, and unverifiable stem structure are not completed.Source: §3.2 · p. 6, §3.3 · p. 7
- Four trained annotators use manual labeling and refinement, with interactive segmentation tools such as SAM available for initial proposals rather than automatic label generation.Ambiguous cases are reviewed against shared rules; an image takes about 25 minutes to annotate on average.Source: §3.3 · pp. 7–8
3.4 Dataset Split and Statistics
- The fixed train/validation/test split stratifies images by a difficulty score combining instance count, occlusion, and ignore-region coverage.This is intended to prevent validation or test sets from being dominated by easy, low-occlusion images.Source: §3.4 · p. 8
- Dataset statistics show that uncertain regions are common and occlusion difficulty follows a long-tailed pattern.The release reports the fixed split and dataset statistics, with Figure 4 showing image-level ignore-region and instance-level height-wise occlusion distributions.Source: §3.4 · p. 9, Table 3 · p. 9, Figure 4 · p. 9
4 Benchmark and Annotation Analysis
Section 4 evaluates EgoMaize as a benchmark rather than proposing a new segmentation architecture, combining overall and visibility-stratified model results with annotation-policy validation and reproducibility analyses. Results show different metric leaders, sharp performance loss under severe occlusion, and stronger reference agreement for evidence-closed than full-amodal annotation.
4.2 Overall Baseline Benchmark
- Mask2Former Swin-B leads mIoU and AP25, CropFormer leads AP50 and count MAE, while YOLO11m-seg has the smallest center distance but very low mask AP.The gap between AP25 and AP50 remains substantial; corrected Mask2Former Swin-B scores 0.8024 and 0.5586, respectively.Source: Table 4 · p. 10, §4.2 · p. 11
4.1 Evaluation Setup
- Models share a fixed split and ignore-mask handling, with five metrics covering foreground overlap, instance matching, counting, and localization.The benchmark includes representative one-stage, two-stage, boundary-refinement, occlusion-aware, query-based, and crop-refinement baselines.Source: §4.1 · p. 10
4.3 Occlusion-Level Breakdown
- AP25 drops from 0.7171 at r = 0 to 0.0915 in the (0.3,0.5] bin for corrected Mask2Former Swin-B.The trend is not strictly monotonic for every architecture, but no model maintains uniformly high AP25 across visibility levels.Source: Table 5 · p. 11, §4.3 · pp. 11–12
4.4 Ignore-Region Sensitivity
- Under the stricter ignore-region policy, AP25 and AP50 were only mildly affected, changing from 0.8024/0.5586 to 0.7885/0.5344 for corrected Mask2Former Swin-B.Although 22.76% of predicted foreground area lies inside ignore regions, none of the 76 predictions with more than half their area inside ignore reaches best valid-region GT IoU 0.50.Source: §4.4 · p. 12, Table 6 · p. 12
4.5 Controlled Artificial-Occlusion Validation
- On the controlled set, evidence-closed annotation achieves higher mIoU and slightly lower bounding-box center distance than full-amodal annotation.The comparison uses exposed reference views for 10 plants and is presented as a small-scale validation.Source: §4.5 · pp. 12–13, Table 7 · p. 13
4.6 Inter-Annotator Consistency
- EgoMaize annotations have mean inter-annotator IoU of 0.8203 (SD 0.0934) and mean bounding-box center offset of 71.62 pixels.The comparison also measures how consistency changes when annotators must complete invisible projections.Source: §4.6 · p. 13, Table 8 · p. 14
5 Conclusion
EgoMaize targets close-range, post-seedling maize perception with plant masks, plant-owned organ cues, and ignore regions, using an evidence-closed annotation protocol. The authors report that ownership-consistent mask prediction remains difficult and identify limits in dataset scope and 2D validation of hidden geometry.
5 Conclusion
- EgoMaize provides high-resolution RGB images, plant-level instance masks, plant-owned stem and tassel fields, and ignore regions for unreliable maize pixels.Source: §5 · p. 14
- Its evidence-closed protocol preserves locally supported same-plant continuity while avoiding completion of hidden regions with unverifiable ownership or boundaries.The authors note that this protocol may transfer to settings where partial occlusion, instance ownership, and unverifiable hidden regions must be handled jointly.Source: §5 · p. 14
- Current instance segmentation models localize maize plants to some extent, but ownership-consistent mask prediction remains difficult.The reported challenges include elongated structures, repeated organs, same-class overlap, and ignore regions.Source: §5 · p. 14
- The dataset is compact, covers limited field sites, devices, and acquisition conditions, and 2D images cannot fully validate hidden plant geometry.The release focuses on plant instances and selected organ cues rather than complete phenotyping records.Source: §5 · p. 14
- Future work will expand coverage, explore in-silico validation, add phenotype-oriented attributes, and use annotation correction traces for quality control and labeling.The proposed coverage expansion includes field conditions, cultivars, planting densities, and post-seedling growth stages; organ additions include exposed stems, leaves, and tassels.Source: §5 · p. 14