PaperlayerView exact revision on arXiv

Source-linked AI summary

PlantC2USeg: Cross-Scale Consistent Pre-Training for Few-Shot Unified Plant Point Cloud Segmentation

Yu Tian, Xintong Jiang, Jan Franklin Adamowski, Shiv O. Prasher, Shangpeng Sun

arXiv:2609.02860v1cs.CV

TL;DR

Plant point cloud segmentation requires effective adaptation despite limited annotations and distribution shifts. PlantC2USeg uses cross-scale consistency learning with information-restricted decoding and unified fine-tuning, and reports stable few-shot transfer across plant datasets and beyond agriculture.

  • Problem

    PPCS adaptation across distribution-shifted examples remains challenging because conventional methods rely on densely annotated datasets that are labor-intensive to acquire.

  • Method

    PlantC2USeg aligns representations across spatial scales, restricts masked reconstruction to visible evidence, and jointly fine-tunes semantic and instance objectives.

  • Results

    Across Soybean3D, HR3D, and SYAU-Maize, PlantC2USeg reports higher IoU and mRec than Deformation3D x10 while retaining higher IoU and mWCov as supervision decreases.

  • Takeaways & Limitations

    The framework supports stem–leaf labeling and individual-leaf partitioning for organ-level trait extraction from small annotated collections.

  • Takeaways & Limitations

    Extending the framework to complex multi-plant field scenes requires evaluation under overlap, occlusion, background vegetation, variable point density, and temporal organ correspondence.

Abstract

from arXiv · show

Modern crop breeding demands precise organ-level analysis for trait quantification, making plant point cloud segmentation (PPCS) increasingly important. However, conventional deep learning approaches rely heavily on densely annotated datasets that are labor-intensive to acquire. Unified PPCS adaptation from distribution-shifted examples with minimal additional training remains challenging. To address this, we propose PlantC2USeg, a deep transfer learning framework featuring cross-scale consistency learning to explicitly align features across spatial scales and an information-restricted decoding strategy that prevents reconstruction shortcuts and promotes robust adaptation. The resulting pre-training enables stable few-shot generalization across species and sensing conditions, while unified fine-tuning with inherited thresholds further reduces adaptation overhead. Under full supervision on Soybean3D, PlantC2USeg achieves the highest semantic IoU and instance mWCov among compared methods, at 91.91% and 94.62%. With 20 labeled samples, it leads both metrics at 89.78% and 90.27%; with only 10 samples, it retains the highest mWCov of 83.23% while achieving 83.19% IoU. Across HR3D, 10-shot transfer to tobacco, tomato, and sorghum averages 78.41% IoU and 79.42% mWCov, while 22-shot transfer to SYAU-Maize achieves the highest IoU and mRec at 92.75% and 93.51%. Furthermore, a leading category-averaged mIoU of 85.0% on ShapeNet Part demonstrates the framework's capability to handle diverse shape variations beyond agricultural domains. These results demonstrate that PlantC2USeg reduces overall adaptation effort under distribution shifts, enabling scalable plant phenotyping and transferable 3D representation learning beyond agriculture.

1. Introduction

Plant point cloud segmentation supports organ-level phenotyping, but limited annotations and distribution shifts make unified adaptation difficult. PlantC2USeg addresses this with cross-scale pre-training, information-restricted decoding, and unified few-shot fine-tuning.

  • Motivation: Limited-annotation regimes motivate models that remain stable when adapting PPCS to new test cases.Existing approaches include data augmentation, synthetic point clouds, and sparse-label propagation, but synthetic supervision can remain biased toward predefined morphological assumptions.
  • Method: Cross-scale consistency learning preserves coherence between local plant cues and higher-level organ arrangement during pre-training.The design is inspired by annotators alternating between detailed inspection and global assessment across scales.
  • Method: Information-restricted decoding conditions reconstruction on visible plant evidence to mitigate reconstruction shortcuts.The decoder design limits information available during masked reconstruction so masked regions remain dependent on visible evidence.
  • Method: Unified fine-tuning jointly optimizes stem–leaf semantic segmentation and instance regularization in feature and coordinate spaces.Progressive segmentation uses thresholds inherited from fine-tuning, eliminating dataset-specific parameter sweeping.
  • Evaluation: PlantC2USeg maintains stable performance with as few as 10 to 20 labeled samples while evaluating shifts across species, datasets, and sensing conditions.The study also extends Soybean3D with 314 samples, including 145 labeled samples, as a benchmark for organ-level segmentation.

2.1. Overview

PlantC2USeg combines cross-scale pre-training, unified fine-tuning, and progressive segmentation for stem–leaf semantic and leaf instance segmentation under limited annotation.

  • Overview: PlantC2USeg integrates cross-scale pre-training, unified fine-tuning, and progressive segmentation into one PPCS pipeline.The pipeline addresses both stem–leaf semantic segmentation and leaf instance segmentation.

2.2. Multi-scale Neighborhood Construction

The framework organizes plant point clouds into multi-scale local neighborhoods, using center-relative coordinates to preserve spatial context at different resolutions.

  • Multi-scale Neighborhood Construction: Multi-scale neighborhood construction organizes point clouds into hierarchical centers and neighborhoods for contextual encoding.For scale s, neighborhoods are represented in local reference frames relative to their center points.

2.3. Cross-scale pre-training

Cross-scale pre-training aligns visible representations of corresponding plant regions and reconstructs masked neighborhoods from restricted visible evidence. This design links multi-scale consistency with decoding that limits masked-token interactions.

  • Hierarchical feature extraction: Visible neighborhoods are encoded across scales, while masked first-scale neighborhoods are withheld as reconstruction targets.Hierarchical masking propagates visibility from a coarsest-scale mask to finer scales through stored neighborhood indices.
  • Consistency supervision: Cross-scale consistency learning encourages agreement within reference-centered groups while separating representations from other groups and samples.The loss uses relative supervision to avoid degenerate constant-vector solutions.
  • Feature projection: Scale-specific projectors map visible tokens into a shared normalized embedding space for direct cross-scale comparison.Two-layer MLP projectors produce l2-normalized hyperspherical embeddings.
  • Cross-scale token grouping: Finer-scale projected representations are grouped around coarsest-scale reference tokens through nearest-center associations.Each cross-scale group links finer-scale projections to a coarsest-scale reference token, making the association explicit for consistency supervision.
  • Reconstruction: Masked-neighborhood reconstruction anchors encoded features because consistency alone does not directly supervise the validity of final-stage features.The reconstruction target is the center-relative input-point coordinate set around each masked first-scale center.
  • Information-Restricted Decoding: Information-restricted decoding uses masked tokens as queries over projected visible representations rather than allowing unrestricted masked-token interactions.The decoder retains coarse-to-fine decoding while applying asymmetric cross-attention between masked queries and visible keys and values.

2.4. Unified few-shot fine-tuning

Unified fine-tuning combines multiscale point representations into a shared embedding for semantic and leaf-instance segmentation. Joint semantic, feature-space, and coordinate-space objectives supervise adaptation, while inherited feature-distance statistics support inference.

  • Unified encoder adaptation: Multiscale neighborhood representations are interpolated to input points, processed by scale-specific MLPs, and averaged into shared embedding F.
  • Semantic supervision: The semantic head predicts point-wise class probabilities for stem–leaf segmentation using a mean negative log-likelihood objective.
  • Instance-center regularization: Leaf instance segmentation combines feature compactness and inter-leaf separation with coordinate-space offset regression toward leaf centers.
  • Instance-center regularization: Feature regularization excludes stem points and omits the inter-instance term when a sample contains only one leaf instance.
  • Inference preparation: Aggregate feature-distance statistics from fine-tuning provide the three criteria used for progressive segmentation.

2.5. Progressive segmentation

Progressive segmentation transfers semantic, coordinate, and feature cues into leaf-instance inference. Morphology-dependent radial processing and inherited thresholds organize clustering without dataset-specific threshold sweeps.

  • Coordinate-guided inference: Semantic predictions select leaf points, and predicted offsets shift their coordinates toward estimated leaf centers.
  • Coordinate-guided inference: Planar distances determine radial eligibility, while feature embeddings guide DBSCAN partitioning, reconnection, residual assignment, and final consolidation.
  • Morphology-dependent progression: For broad-leaved or rosette plants, outer-to-inner radial progression moves from peripheral points toward the central stem region.
  • Morphology-dependent progression: For narrow, grasslike leaves, radial progression is disabled and all candidates are processed in one pass.
  • Radial eligibility: The radius decreases linearly from a morphology- and supervision-dependent initial value to zero across radial steps.
  • Cluster association: DBSCAN partitions eligible embeddings, matches later clusters to existing instances using centroid distance, and creates new labels when reconstruction exceeds the matching threshold.

3. Experiment

Experiments evaluate PlantC2USeg across agricultural and synthetic point-cloud benchmarks under full, few-shot, and unsupervised pre-training settings. The evaluation uses semantic and instance segmentation metrics to assess performance across datasets and supervision regimes.

  • 3.1. Datasets: Experiments cover Soybean3D, HR3D, SYAU-Maize, and ShapeNet Part, spanning plant species, sensing conditions, and synthetic object categories.HR3D includes tomato, sorghum, and tobacco point clouds, while ShapeNet Part covers 16 categories and 50 semantic parts.
  • 3.1. Datasets: 22 samples were used to construct the SYAU-Maize training set, with identical samples used for all compared methods.The dataset includes complete and incomplete plant structures, enabling evaluation under partial observations.
  • 3.2. Experimental settings: Pre-training used unlabeled Soybean3D data and ShapeNet pre-training for synthetic part segmentation, with inputs randomly sampled to 2048 points.Soybean3D pre-training yielded 2,890 augmented samples; ShapeNet pre-training used 57,448 synthetic shapes from 55 categories.
  • 3.2. Experimental settings: Downstream plant segmentation used fully annotated samples under full-shot and limited-label protocols, with PlantC2USeg inputs containing 4,096 points.Few-shot samples were fully annotated for both semantic and instance segmentation, while Soybean3D and SYAU-Maize were preprocessed to 16,384 points before augmentation.
  • 3.3. Evaluation metrics: Semantic performance was measured with Precision, Recall, F1-score, and IoU, while instance performance used mPrec, mRec, mCov, and mWCov.mCov averages coverage across instance-aware semantic categories, whereas mWCov weights instances according to their point counts.

4. Results

PlantC2USeg consistently improves unified stem–leaf semantic and leaf instance segmentation under full-shot and few-shot settings, including transfers across species and sparse-label protocols.

  • Soybean3D: 95.69% semantic F1 and 91.91% semantic IoU were achieved under full supervision on Soybean3D, with PlantC2USeg leading the compared methods.With the downstream architecture fixed, pre-training increased IoU by 1.72 percentage points, mainly through higher recall.
  • Soybean3D: 89.78% semantic IoU and 94.47% semantic F1 were achieved with 20 labeled samples on Soybean3D.The 20-shot IoU exceeded the full-shot results of PlantNet, PSegNet, and Eff-3DPSeg.
  • HR3D: 81.50% aggregate IoU was achieved on HR3D at 20 shots, exceeding Deformation3D by 15.38 percentage points across tobacco, tomato, and sorghum.The advantage persisted across the evaluated HR3D species and also remained competitive against full-shot general-purpose methods.
  • SYAU-Maize: 92.75% IoU was achieved under the 22-sample SYAU-Maize protocol, with PlantC2USeg remaining higher than both Deformation3D variants in F1 and IoU.Increasing Deformation3D data generation 100-fold raised mWCov but lowered IoU by 0.60 percentage points.
  • Leaf instance segmentation: PlantC2USeg preserved broad leaves and separated closely grouped leaves more effectively than several baselines in qualitative full-shot and few-shot comparisons.At 20 and 10 shots, it retained broad leaves as single instances while Soybean-PCMAE repeatedly fragmented them.
  • Leaf instance segmentation: 94.62% mWCov was achieved under full-shot Soybean3D instance segmentation, compared with 87.89% for PSegNet.The mRec margin reached 10.20 percentage points, indicating the largest gain concerned recovery of ground-truth leaves.

4.3. Object part segmentation

PlantC2USeg extends part-segmentation evaluation beyond plants and maintains strong performance across diverse ShapeNet Part object categories.

  • ShapeNet Part: 85.0% category-averaged mIoU and 86.4% instance-averaged mIoU were achieved on ShapeNet Part.PlantC2USeg exceeded baselines trained from scratch and ranked first among single-modal pre-training methods in category-averaged mIoU.
  • ShapeNet Part: Accurate part segmentation was maintained across a wide range of generic object categories.The evaluation therefore extends the reported segmentation capability beyond plant-specific data.

5. Discussion

The discussion attributes PlantC2USeg’s transfer performance to cross-scale representation coherence, information-restricted reconstruction, unified adaptation, and inherited segmentation thresholds, while also examining practical efficiency.

  • Ablation study: Cross-scale consistency learning raised IoU by 1.25 percentage points with the standard decoder and 0.90 percentage points with the information-restricted decoder.The combined design reached 95.69% F1 and 91.91% IoU while leading all four instance metrics.
  • Ablation study: Information-restricted decoding conditions reconstruction on visible evidence, complementing the cross-scale encoder constraint.Unrestricted masked-token attention allocated mass to masked keys, providing an internal reconstruction route.
  • Attention distribution across scales during encoding: Attention remained local at early stages and reached a broader portion of the connected plant at s=3.Because attention is normalized over keys and key counts differ, larger displayed weights at s=3 do not alone establish stronger or more concentrated attention.
  • Progressive segmentation: Inherited feature-distance thresholds kept mCov and mWCov within 1.04 and 0.51 percentage points of their observed maxima across criteria and supervision regimes.The thresholds avoided isolated precision maxima associated with lower coverage or recall.
  • Adaptation: Pre-training raised semantic IoU by 2.63 percentage points and mWCov by 8.01 percentage points over random initialization at 20 shots on Soybean3D.The aligned benefit also extended to 10-shot supervision.
  • Annotation efficiency and practical considerations: Deformation3D x1k increased generated clouds 100-fold, raised mWCov by 3.38 percentage points, and lowered IoU by 0.60 percentage points.The comparison shows a trade-off between size-weighted instance coverage and semantic overlap.
  • Annotation efficiency and practical considerations: PlantC2USeg recorded the shortest network-core time among five evaluated methods at 38.60 ms for one Soybean3D sample.The measured model performs semantic and instance segmentation in one shared network, unlike the cited HAIS measurement covering only the Deformation3D instance network.

6. Conclusion

PlantC2USeg provides reusable pre-training for jointly adapting stem–leaf semantic and leaf instance segmentation with limited annotations. Its reported scope includes cross-distribution transfer and organ-level trait extraction, while complex multi-plant field scenes remain an evaluation boundary.

  • PlantC2USeg jointly adapts stem–leaf semantic segmentation and leaf instance segmentation through reusable pre-training.The framework reduces dependence on target annotations and supports both tasks through a shared adaptation model.
  • Higher IoU and mRec than Deformation3D x10 on SYAU-Maize support reusing the pre-trained representation under different target distributions.
  • Limited-label regimes across three plant datasets produce stem–leaf labels and individual-leaf partitions needed for later organ-level trait extraction.
  • Complex multi-plant field scenes remain unevaluated, where overlap, occlusion, background vegetation, reconstruction completeness, and point density complicate plant and organ association.
  • Stable organ correspondence across repeated 3D observations would enable longitudinal measurement of organ-specific growth trajectories.

CRediT authorship contribution statement

The contribution statement assigns data curation, investigation, methodology, visualization, formal analysis, writing, supervision, project administration, resources, and funding acquisition across the listed authors.

  • Yu Tian contributed data curation, investigation, methodology, visualization, formal analysis, and manuscript writing.
  • Xintong Jiang contributed data curation, investigation, and manuscript review and editing.
  • Jan Franklin Adamowski and Shiv O. Prasher contributed manuscript review and editing.
  • Shangpeng Sun contributed conceptualization, resources, manuscript review and editing, supervision, project administration, and funding acquisition.
Loading 2609.02860v1…