PaperlayerView exact revision on arXiv

Source-linked AI summary

A CNN Approach to Simultaneously Count Plants and Detect Plantation-Rows from UAV Imagery

Lucas Prado Osco, Mauro dos Santos de Arruda, Diogo Nunes Gonçalves, Alexandre Dias, Juliana Batistoti, Mauricio de Souza, Felipe David Georges Gomes, Ana Paula Marques Ramos, Lúcio André de Castro Jorge, Veraldo Liesenberg, Jonathan Li, Lingfei Ma, José Marcato Junior, Wesley Nunes Gonçalves

arXiv:2012.15827v3cs.CV

TL;DR

High-density plant counting and row detection from UAV imagery remain challenging across crops. This paper uses a CNN with mutually informing plant and row branches; it reports results across corn and citrus, including better corn-counting results than three evaluated networks.

  • Problem

    High-density plant counting is challenging, and simultaneous plant counting and row mapping across cultivars remained an unproposed task.

    Source: §1 · p. 4

  • Method

    A CNN estimates plant and row confidence maps, with two branches exchanging information to refine predictions and geolocate detections.

    Source: §1 · p. 4, §2.2 · p. 5, §2.2.5 · p. 9

  • Results

    Across corn and citrus, the method counted plants and detected rows; corn plant-counting MAE was 6.224 plants per image patch, outperforming HRNet, Faster R-CNN, and RetinaNet.

    Source: §5 · pp. 20–21

  • Takeaways & Limitations

    The authors present the approach as a feasible alternative to visual inspection for precision farming and as applicable to crops beyond those tested.

    Source: §4 · pp. 18, 20

    Weed variety was not a main concern in the study area, and adding datasets with more weed variety is identified as future work.

    Source: §4 · p. 19

Abstract

from arXiv · show

In this paper, we propose a novel deep learning method based on a Convolutional Neural Network (CNN) that simultaneously detects and geolocates plantation-rows while counting its plants considering highly-dense plantation configurations. The experimental setup was evaluated in a cornfield with different growth stages and in a Citrus orchard. Both datasets characterize different plant density scenarios, locations, types of crops, sensors, and dates. A two-branch architecture was implemented in our CNN method, where the information obtained within the plantation-row is updated into the plant detection branch and retro-feed to the row branch; which are then refined by a Multi-Stage Refinement method. In the corn plantation datasets (with both growth phases, young and mature), our approach returned a mean absolute error (MAE) of 6.224 plants per image patch, a mean relative error (MRE) of 0.1038, precision and recall values of 0.856, and 0.905, respectively, and an F-measure equal to 0.876. These results were superior to the results from other deep networks (HRNet, Faster R-CNN, and RetinaNet) evaluated with the same task and dataset. For the plantation-row detection, our approach returned precision, recall, and F-measure scores of 0.913, 0.941, and 0.925, respectively. To test the robustness of our model with a different type of agriculture, we performed the same task in the citrus orchard dataset. It returned an MAE equal to 1.409 citrus-trees per patch, MRE of 0.0615, precision of 0.922, recall of 0.911, and F-measure of 0.965. For citrus plantation-row detection, our approach resulted in precision, recall, and F-measure scores equal to 0.965, 0.970, and 0.964, respectively. The proposed method achieved state-of-the-art performance for counting and geolocating plants and plant-rows in UAV images from different types of crops.

1 Introduction

Plant counting and row detection matter for yield estimation and crop monitoring, but high-density plantations remain challenging. The paper proposes a shared-branch CNN approach for counting plants and detecting rows across corn and citrus from UAV imagery.

1 Introduction

  • Plant counts support yield estimation and growth monitoring, while row detections can help assess missed plants and crop production.The introduction also notes that early assessment can support rapid decisions.Source: §1 · pp. 3–4
  • Prior corn studies had not addressed plantation-row detection, while simultaneous plant counting and row mapping across cultivars remained unproposed.The paper identifies these gaps in work on corn and citrus, respectively.Source: §1 · p. 4
  • High-density plant counting is challenging, and prior investigations were few and often limited to one cultivar.Early-stage counts can support rapid decisions, while row detection can help assess missed plants and production.Source: §1 · p. 4
  • The proposed CNN uses two branches that exchange information to count plants and detect plantation-rows simultaneously from UAV imagery.Plant detections are refined toward detected rows, while row predictions adjust to plant positions.Source: §1 · p. 4
  • The study uses RGB imagery as a lower-priced, readily replicable option for detecting plants and rows across crops and plantation densities.The stated scope includes corn and citrus while preserving geolocation information.Source: §1 · pp. 4–5

2 Materials and Method

The framework proceeds from UAV image acquisition through annotation and dataset splitting to model evaluation and task-specific error calculation.

2 Materials and Method

  • The five-step framework begins by acquiring RGB images from corn and citrus fields using a camera mounted on a UAV.The steps then cover specialist labeling, hold-out dataset splitting, evaluation of plant and row detection, and calculation of error metrics.Source: §2 · p. 5
  • A specialist labels plantation-rows as line features and plants as point features in a GIS environment.These annotations follow image preprocessing.Source: §2 · p. 5
  • The data are divided into training, validation, and testing sets by hold-out, then evaluated for row detection and plant counts.The framework calculates error metrics for each task performed by the CNN.Source: §2 · p. 5

2.1 Study Area and Data

The study evaluates corn at young and mature growth stages and mature citrus trees in separate Brazilian sites using UAV imagery.

2.1 Study Area and Data

  • Corn was studied at the Federal University of Mato Grosso do Sul field, with V3 and mature plants spaced 30 × 50 cm and averaging 4–5 plants per square meter.The approximately 7,435 m² field had rows of two lengths and directions.Source: §2.1 · p. 5
  • Corn imagery came from a Phantom 4 Advanced UAV with an RGB camera, collected at 1.55 cm GSD with 80% forward and 60% lateral overlap.The images were processed with Pix4D using structure-from-motion and multi-view stereo.Source: §2.1 · p. 5
  • Mature Pera citrus trees were studied in an approximately 10,000 m² orchard in Boa Esperança do Sul, where spacing and density varied.An X7–Spire II UAV collected RGB imagery at 80 m altitude and 2.28 cm GSD.Source: §2.1 · p. 5

2.2 Convolutional Neural Network

The CNN processes RGB imagery into plant and row confidence maps through contextual feature enhancement and multi-stage, mutually informed refinement. It extracts plant locations from map peaks and row centerlines through skeletonization.

2.2 Convolutional Neural Network

  • The CNN uses a two-branch MSM to produce plant and row predictions with geolocation, exchanging branch information across stages to refine both maps.Image coordinates are added to the final prediction map from a separate file.Source: §2.2 · pp. 5–6

2.2.1 Feature Map Extraction

  • A VGG19-based feature extractor processes 256 × 256 RGB inputs through eight convolutional layers, two pooling layers, and upsampling to a 128 × 128 feature map.The convolutional layers use 64, 128, and 256 filters of size 3 × 3.Source: §2.2.1 · pp. 6–7

2.2.2 Pyramid Pooling Module – PPM

  • The PPM combines global and local context by pooling the feature map at four scales, then concatenating the upsampled features with the input map.The pooling resolutions are 1 × 1, 2 × 2, 3 × 3, and 6 × 6.Source: §2.2.2 · p. 7

2.2.3 Multi-Stage Module Refinement and Co-Shared Information

  • The MSM has separate plant and row branches, each generating confidence maps and refining predictions over successive stages.At each later stage, the previous predictions are concatenated with the PPM feature map; branch information lets row predictions inform plant locations and plant predictions inform rows.Source: §2.2.3 · pp. 7–8

2.2.4 Confidence Map

  • Ground-truth confidence maps place Gaussian responses at plant centers and along row extents, with stage-specific σ values controlling peak spread.The first stage uses the maximum σ and the last the minimum; intermediate values are equally spaced, with separate settings for plants and rows.Source: §2.2.4 · p. 9, Figure 4 · p. 9

2.2.5 Object and Row Localization and Extraction

  • Plant locations are extracted as local maxima in a four-neighbor confidence-map neighborhood, while row maps are skeletonized to obtain centerlines.The peak threshold is T = 0.35, with a minimum distance of 1 pixel between detections.Source: §2.2.5 · pp. 9–10, Figure 5 · p. 10

2.3 Experimental Setup

The setup evaluates corn and citrus imagery through patch-based datasets, using random train/validation/test splits and comparisons with three deep networks. It also defines the training procedure and evaluation metrics.

2.3 Experimental Setup

  • 564 corn patches and 635 citrus patches were used; training included 33,360 corn plants and 224 rows, plus 14,810 citrus trees and 213 rows.Each patch measured 256 × 256 pixels, with no overlap.Source: §2.3 · p. 10
  • Figure 6 depicts the row confidence map, its skeletonization, and the predicted rows over the image.Source: Figure 6 · p. 11
  • The study used differing plant phonologies, sites, and sensor characteristics to assess the approach’s generalization and robustness.Source: §2.3 · p. 11
  • Patches were randomly divided into 60% training, 20% validation, and 20% testing, with no patch shared across subsets.The network used ImageNet-pretrained weights, SGD with momentum 0.9, a 0.001 learning rate, and 100 epochs; validation guided learning-rate and epoch adjustments.Source: §2.3 · p. 11
  • The CNN was compared with HRNet, Faster R-CNN, and RetinaNet using the same dataset characteristics and sampling conditions.Source: §2.3 · p. 11
  • Evaluation used MAE, MRE, MSE, precision, recall, and F1-measure; regression errors compare labeled counts with predicted positions for each patch.Source: §2.3 · p. 11

3 Results

The results evaluate refinement settings, branch interactions, and performance across corn growth stages and citrus. The proposed CNN outperforms the evaluated object detectors on corn plant detection and also detects plantation-rows and citrus plants.

3.1.1 Parameter Analysis in Plant and Plantation-Row Detection

  • T = 6 produced plant-detection MAE 7.778 and F-measure 0.894, while row precision, recall, and F-measure reached 0.934, 0.983, and 0.956.T = 8 was slightly better but had substantially higher computational cost, so it was not adopted for later experiments.Source: §3.1.1 · p. 12, Table 1 · p. 12
  • For plant detection, σmax = 3 and σmin = 1 yielded F1-measure 0.894 and MAE 7.778 plants per patch; row detection performed best with σmin,row = 0.5 and σmax,row = 3.0.The row settings reached precision 0.950, recall 0.983, and F-measure 0.965.Source: §3.1.1 · pp. 12–13, Table 3 · p. 12, Table 4 · p. 13

3.1.2 Plant and Plantation-Rows Extraction

  • The model produced similar metrics for V3 and mature corn, with V3 slightly better on classification and row detection but slightly worse on patch-level error metrics.The authors relate V3’s classification advantage to smaller plants and fewer occlusions, while hard-to-detect plants near patch edges increase error metrics.Source: §3.1.2 · p. 13, Table 6 · p. 14
  • The proposed method correctly predicted most V3 plant positions, including overlapping plants, across different density conditions.Its main plant-detection challenges were heavily occluded plants at patch borders and overlapping plants in dense rows.Source: §3.1.2 · pp. 13–14, Figure 7 · p. 14, Figure 8 · p. 15
  • The method detected curved rows, rows with widely spaced plants, and isolated plants without assigning those isolated plants to a true row.The authors attribute curve adaptation to information exchange between plant and row branches, where detected plant positions influence predicted row shape.Source: §3.1.2 · p. 14, Figure 9 · p. 16

3.1.3 Comparative Results with State-of-the-Art Deep Networks

  • Corn plantation-row detection reached precision 0.913, recall 0.941, and F-measure 0.925; the bounding-box baselines could not be evaluated properly for this task.The method performs simultaneous plant counting and row detection in a one-step architecture.Source: §3.1.3 · p. 15
  • The CNN achieved better performance than HRNet, Faster R-CNN, and RetinaNet on corn plantation (V3 and matures) datasets, with precision gains of +9.5%, +12%, +16.3% and MAE differences of -8,665, -11,021, -14,026, respectively.Source: §3.1.3 · p. 14, Table 7 · p. 15

3.2 Experiments in the Citrus Plantation Dataset

  • On citrus, the method generalized with minimal parameter adjustment and maintained high performance in high-density plantations with overlapping canopies.The citrus evaluation used a 15-pixel prediction distance rather than the 8-pixel distance used for corn; the paper reports results in Table 9 without specifying its metric values in the supplied passage.Source: §3.2 · pp. 16–17, Table 9 · p. 18

4 Discussion

The discussion presents the CNN as a feasible approach to counting plants and detecting plantation-rows in dense crops, with information exchanged between its branches. The authors highlight results across corn and citrus, while noting challenges involving occlusion, weeds, and plantation patterns.

4 Discussion

  • Plant-and-row detection was presented as a one-step alternative that may be replicated in other crops, including for identifying high-density canopies.The discussion also notes that RGB imagery is a lower-cost alternative to LiDAR or multispectral and hyperspectral sensors.Source: §4 · pp. 18–19
  • 0.876 F-Measure and 6.224 MAE were reported for corn; the authors describe results as similar to or better than existing literature despite dense conditions.The corn dataset reached up to 4 or 5 plants per square meter, and the study also notes that no cited comparison method implemented plantation-row detection.Source: §4 · pp. 18–20
  • The citrus experiments tested generalization to a denser orchard setting, where the authors report high accuracy despite the detection challenge.The discussion compares the citrus dataset with prior work but cautions that datasets with different characteristics make direct comparison difficult.Source: §4 · p. 19
  • The discussion identifies diverse weeds as an area for further exploration because they were not a main concern in the evaluated area.The authors suggest that sharing information between branches can reduce detection of objects such as weeds that have similar spectral information but differ spatially from target plants.Source: §4 · p. 19
  • The two-branch CNN exchanges plant and row information, and the authors report higher overall accuracy for both predictions when using this strategy.The discussion describes confidence-map refinement as helping detect overlapped and individual plants, and row refinement as potentially helping identify seedling problems or plantation gaps.Source: §4 · pp. 18, 20
  • Reported accuracy challenges include plant occlusion and proximity, isolated plants outside rows, unusual spacing, and curved plantation-rows.The authors also report greater processing time than plant-only bounding-box methods because their network computes both plants and plantation-lines.Source: §4 · p. 20

5 Conclusion

The conclusion presents a UAV RGB-based CNN for jointly counting plants and detecting plantation-rows in cornfields and citrus orchards. It reports strong results across both crops and recommends the one-step approach for estimating plantation patterns and errors.

5 Conclusion

  • 0.876 F-measure and 6.224 MAE were reported for corn plant counting, outperforming HRNet, Faster R-CNN, and RetinaNet on the same task and dataset.The reported metrics were 0.1038 MRE, 0.856 precision, and 0.905 recall.Source: §5 · p. 20
  • For citrus-tree counting, recall was 0.911 and F-measure was 0.965; citrus plantation-row detection achieved precision 0.965, recall 0.970, and F-measure 0.964.The proposed method uses an RGB-based sensor as a reasonable cost alternative.Source: §5 · p. 21
  • The conclusion describes the RGB-based method as a reasonable-cost alternative that may support decision-making in agricultural management.The authors also state that they intend to implement new features to address plantation-pattern challenges.Source: §5 · p. 21
  • The authors recommend the one-step, two-branch approach for jointly counting plants and detecting plantation-rows to estimate plantation patterns and errors.The method exchanges information between branches and uses confidence maps rather than bounding-box detection for individual plants.Source: §5 · p. 21

Funding

The authors acknowledge support from EMBRAPA, CNPq, CAPES-Print, Fundect, and FAPESC.

Funding

  • The authors received support from EMBRAPA and CNPq.The passage lists grant numbers for both funders.Source: Funding · p. 21
  • CAPES-Print and Fundect also supported the authors.Source: Funding · p. 21
  • V. Liesenberg was supported by FAPESC.Source: Funding · p. 21

Related papers

Loading 2012.15827v3…