PaperlayerView exact revision on arXiv

Source-linked AI summary

A Survey of Deep Learning Techniques for Weed Detection from Images

A S M Mahmudul Hasan, Ferdous Sohel, Dean Diepeveen, Hamid Laga, Michael G. K. Jones

arXiv:2103.01415v1cs.CVcs.LG

TL;DR

Weed detection and species recognition are difficult because crops and weeds can share visual characteristics, yet they matter for automated weed management. This paper surveys deep learning research across data, preparation, detection, localisation, classification, and evaluation. The survey finds that fine-tuned pretrained models and sufficient labelled data are associated with high reported accuracy, although those results remain limited to narrow experimental settings.

  • Problem

    Weed and crop imagery can contain similar colours, textures, and shapes, while existing deep learning literature required a comprehensive synthesis.

  • Method

    The paper conducts a comprehensive literature survey and taxonomy of deep learning techniques for detecting, localising, and classifying weeds in crops.

  • Results

    High classification accuracy was achieved by fine-tuning pretrained models on plant datasets, especially when sufficient labelled data for each class was available.

  • Takeaways & Limitations

    Deep learning techniques show strong potential for automated weed detection, localisation, and classification within the surveyed research scope.

  • Takeaways & Limitations

    High accuracy was achieved mainly in limited experimental setups using small datasets and selected crop and weed species.

Abstract

from arXiv · show

The rapid advances in Deep Learning (DL) techniques have enabled rapid detection, localisation, and recognition of objects from images or videos. DL techniques are now being used in many applications related to agriculture and farming. Automatic detection and classification of weeds can play an important role in weed management and so contribute to higher yields. Weed detection in crops from imagery is inherently a challenging problem because both weeds and crops have similar colours ('green-on-green'), and their shapes and texture can be very similar at the growth phase. Also, a crop in one setting can be considered a weed in another. In addition to their detection, the recognition of specific weed species is essential so that targeted controlling mechanisms (e.g. appropriate herbicides and correct doses) can be applied. In this paper, we review existing deep learning-based weed detection and classification techniques. We cover the detailed literature on four main procedures, i.e., data acquisition, dataset preparation, DL techniques employed for detection, location and classification of weeds in crops, and evaluation metrics approaches. We found that most studies applied supervised learning techniques, they achieved high classification accuracy by fine-tuning pre-trained models on any plant dataset, and past experiments have already achieved high accuracy when a large amount of labelled data is available.

1 Introduction

Weed management is important but difficult because weeds and crops can look alike under varied field conditions. This survey examines deep learning methods and the data, processing, detection, classification, and evaluation choices supporting automated weed recognition.

  • Motivation: 17?
  • Motivation: Weed control is increasingly desirable as labour costs and health and environmental concerns rise.Automated systems may reduce labour costs and minimise herbicide use through selective spraying.
  • Detection challenge: Weed detection is challenging because crops and weeds often share similar colours, textures, and shapes.Occlusion, lighting variation, growth-stage differences, motion blur, noise, geography, weather, and soil conditions add difficulty.
  • Deep learning rationale: Deep learning can address feature-extraction difficulty by learning discriminative features directly from imagery.Traditional machine learning requires manually extracting and selecting features when crops and weeds are visually similar.
  • Survey scope: The survey provides a taxonomy of deep learning techniques and reviews data collection, preparation, representation, and evaluation metrics.It focuses on detecting, localising, and classifying weeds in crops.

2 Related Surveys

Earlier surveys addressed weed monitoring, sensing, selective spraying, and deep learning, but coverage was often narrow or incomplete. Reported gaps include limited datasets, insufficiently generalised models, and inadequate breadth of reviewed literature.

  • Deep learning relevance: Deep learning was reported to outperform traditional image-processing methods and support applications beyond weed management.Proposed applications included fertilisation, irrigation, and phenotyping.
  • Prior review scope: Earlier reviews covered ground-based detection, remote sensing, weed monitoring, and selective spraying from different perspectives.
  • Prior findings: Prior work identified limitations in using only spectral or spatial features and preferred combining both.
  • Research gaps: Earlier deep learning reviews reported a lack of large crop-weed datasets, acceptable classification accuracy, and generalised models.
  • Research gaps: One earlier review covered only a handful of publications and therefore did not adequately cover the literature’s breadth and depth.

3 Traditional ML- vs DL-based Weed Detection Methods

Traditional machine learning depends on engineered features, whereas deep learning learns task-relevant representations from raw data. The surveyed literature describes convolutional and recurrent architectures and reports stronger deep learning performance with large datasets.

  • Traditional ML: Traditional machine-learning weed classification typically involves image acquisition, preprocessing, feature extraction or selection, and classification.
  • Traditional ML: Traditional studies combined shape, colour, texture, spectral, or sensor features and evaluated classifiers including SVM, LDA, Random Forest, and perceptrons.
  • Traditional ML: Colour or texture alone may be inadequate for distinguishing wheat from Bidens pilosa, motivating multimodal features such as near-infrared cues.
  • Deep learning: Deep learning automatically discovers discriminative features from raw data for classification and object detection.This representation-learning approach reduces dependence on manually constructed feature extractors.
  • Deep learning architectures: CNNs are widely used for image analysis, while RNNs incorporate prior inputs and LSTMs retain important prior information.
  • Comparative performance: Deep learning techniques show superior performance compared with traditional machine learning when large datasets are used.
  • Workflow: The surveyed workflow comprises data acquisition, dataset preparation or image preprocessing, classification, and evaluation.Not every method includes every outlined step.

4 Paper Selection Criteria in this Survey

The survey used a two-step process to search for and analyse related studies addressing deep learning for weed detection, localisation, and classification. Publication counts were examined to identify the relevant period, with studies from 2016 onward selected as the survey base.

  • The review process comprised searching and selecting related studies, followed by detailed analysis of those studies.
  • The main research question asked how deep learning techniques support detecting, localising, and classifying weeds in crops.
  • The authors searched multiple scholarly databases using keywords covering publications from 2010 to 30 August 2020.
  • Before 2016, publications on deep-learning weed detection were very few, while publication numbers increased from 2016 onward.
  • The survey therefore used articles published from 2016 onward as its review corpus.

Detection Approaches

The survey organises deep-learning weed-detection research through a taxonomy covering the data pipeline and model approaches. It compares studies by acquisition, preparation, architectures, and whether they generate weed maps or classify individual plants.

  • The survey presents an overall taxonomy for categorising deep-learning techniques used in weed detection.
  • The taxonomy covers data acquisition, sensors and mounting vehicles, dataset preparation, and deep-learning approaches.
  • Related works either generate a weed map for the target site or classify individual plants as crops or weeds.
  • The reviewed classifiers use supervised, unsupervised, or semi-supervised learning approaches alongside varied deep-learning architectures.
  • The reviewed studies are compared by crop and weed species, dataset collection and preparation, and applied deep-learning methods.

6 Data Acquisition

Weed imagery is acquired through varied sensors and platforms, including UAVs, field robots, ATVs, handheld cameras, and satellites. Public datasets provide crop–weed imagery with diverse annotations, modalities, species, and access conditions.

  • The survey describes weed-data collection using different modalities, sensors, and mounting platforms.
  • UAV acquisition: UAVs collect RGB or multispectral imagery for mapping weed density, identifying crop rows, and covering large field areas.
  • Ground acquisition: Field robots, ATVs, mobile platforms, tripods, and handheld cameras have all been used to capture crop and weed images.
  • Sensors and modalities: Researchers have collected RGB, multispectral, hyperspectral, monochrome, and satellite imagery for weed-related tasks.
  • Public datasets: The survey lists nineteen publicly available crop and weed datasets for training deep-learning models.
  • Public datasets: Most datasets contain RGB images from food-crop and weed settings, while some use multispectral or hyperspectral sensors.
  • Public datasets: The CNU Weed Dataset contains 208,477 images and twenty-one weed species from five families, but exhibits class imbalance.
  • Public datasets: Dataset annotations include image-level, pixel-wise, and bounding-box labels, with most datasets manually labelled.

7 Dataset Preparation

Dataset preparation commonly combines preprocessing, resizing, augmentation, synthetic-data generation, and annotation before training, testing, and validation. Studies adapt image representations and scale to model requirements while addressing lighting, background, noise, and computational constraints.

  • Preparing data for deep-learning experiments includes image processing, labelling, augmentation, and synthetic-data generation.
  • Image preprocessing: Common preprocessing operations include background removal, resizing, vegetation segmentation, de-noising, enhancement, and colour-model changes.
  • Image resizing: A 60×60-pixel lower patch size achieved good accuracy and required less model-training time in one study.
  • Image resizing: Most studies resize images to match network requirements and reduce processing time or computational complexity.
  • Image resizing: High-resolution imagery is often divided into patches or tiles to reduce computational complexity before model input.
  • Image preprocessing: Preprocessing methods address illumination, noise, blur, soil, and background separation using thresholding, vegetation indices, morphology, and enhancement.
  • Training data generation: Geometric transformations were applied in most studies, while synthetic data were identified as useful when labelled data were insufficient.

8 Detection Approaches

Weed detection approaches either identify and classify individual plants or map weed density across fields. These outputs support real-time, selective, and site-specific weed management.

  • Detection approaches: Studies use two broad approaches: classifying individual plants after localisation, or mapping weed density across a field.Some studies add further classification of weed species.
  • Individual plant detection: Individual plant classification labels each plant as a crop or weed, supporting real-time weeding and selective spraying.A robotic system used detected weeds for knife-based removal, while selective spraying requires identifying individual crop plants and weeds.
  • Species classification: Species identification enables specific treatments, including appropriate herbicides and doses.The reviewed studies report that identifying weed species is important for targeted control.
  • Density mapping: Weed-density mapping supports site-specific management, potentially reducing herbicide use and enabling location-specific applications.A density map can guide the amount of herbicide applied to a specific field site.
  • Field monitoring: Distribution maps can help farmers monitor weed spread and act accordingly.Deep learning has been used to generate weed distribution maps and automatically monitor field conditions.

9 Learning Methods

The reviewed studies mainly use supervised learning, while unsupervised and semi-supervised approaches provide alternatives when labels or data structures differ. Unsupervised methods include clustering and pre-training with plant-image features.

  • Supervised learning: Supervised learning trains models on labelled images and corresponding outputs, and most related weed-detection studies use this approach.The model learns a mapping from inputs to outputs based on labelled training data.
  • Unsupervised learning: Unsupervised learning uses unlabelled data to learn structure, extract features, and group objects into clusters.Clustering separates objects whose features are similar within a group and different from other groups.
  • Unsupervised learning: A semi-automatic labelling approach combined JULE and DeepCluster with AlexNet and VGG-16 architectures initialized by pre-trained weights.The approach achieved 97% accuracy in classifying weeds in crops and reduced manual labelling cost.
  • Unsupervised learning: K-means clustering was used as a pre-training step to generate a feature dictionary for initializing CNN weights.The authors claimed improved feature-extraction generalization and better accuracy than SVM, back-propagation neural networks, and randomly initialized CNNs.
  • Semi-supervised learning: Semi-supervised learning combines supervised and unsupervised learning, with GCNs using graph-structured rather than regular structured input.The reviewed work distinguishes CNNs for regular structured data from GCNs for graph data.

10 Deep Learning Architecture

The literature applies diverse deep learning architectures according to dataset and research goal, commonly comparing them with other deep learning or traditional machine learning models. Reported results span classification, detection, segmentation, speed, and hardware trade-offs.

  • Architecture overview: Researchers select different architectures according to the dataset and research goal, often comparing proposed models with deep learning or traditional machine learning approaches.CNNs commonly combine feature extraction and classification, while many studies use established architectures such as VGGNet.
  • CNN architectures: Pre-trained weights often produced strong classification results, including 90% for AlexNet on Rumex and 92.1% for VGG-16 in another Rumex study.The studies reached different conclusions about which architecture performed best in grassland weed detection.
  • CNN architectures: Reported classification results included 95.7% for ResNet-50 versus 95.1% for Inception-v3, and 99% for k-FLBPCM on barley, canola, and wild radish.Other studies reported accuracies from 33% to 98% across plant species.
  • Real-time detection: Reducing cNET to 16 filter layers made it 2.5 times faster while remaining adequate for pre-processed crop-and-weed classification.The reduced architecture was used for real-time weed detection.
  • Object detection: Tiny YOLO-v3 was favored for lower-cost hardware, while Faster R-CNN showed better weed-detection accuracy and inference performance in another comparison.Tiny or Fast YOLO-v3 can improve detection speed, but hardware configuration affects processing speed.
  • Segmentation: SegNet achieved 92.7% accuracy, exceeding FCN at 89.5% and U-Net at 70.8% for rice-field weed segmentation.Segmentation accuracy also depended on dataset size, and image transformations or additional spectral channels improved reported performance.
  • Graph-based architectures: Graph-based models reported classification accuracies of 97.80%, 99.37%, 98.93%, and 96.51%, while a DenseNet-202 graph model achieved 98.1%.The review also covers GWN and a ResNet-101-based graph convolutional network.

11 Performance Evaluation Metrics

The review describes evaluation metrics for classifier quality and model comparison, with classification accuracy, F1 score, and mIoU used most frequently. Runtime, memory, and hardware requirements are also relevant for real-time systems, but differing metrics make studies difficult to compare.

  • Metric selection: Evaluation metrics quantify classifier performance and can measure model quality or compare trained models for selection.Researchers choose metrics according to their research needs and often use multiple metrics before drawing conclusions.
  • Comparison limitations: Different evaluation metrics, models, goals, datasets, and detection settings make it difficult to compare related studies directly.The review identifies metric heterogeneity as an explicit comparison limitation.
  • Common metrics: Classification accuracy, F1 score, and mIoU are the most frequently used evaluation metrics, while confusion metrics are preferred for plant-species classification.Table 6 summarizes the metrics applied in related studies.
  • Efficiency metrics: Runtime and inference time are evaluated when systems must identify crop and weed classes quickly for real-time operation.Studies also measure training and identification time when developing real-time classifiers.
  • Efficiency metrics: Increasing input patch size reduced model training time in one study.Processing time was also used to compare different deep learning architectures.
  • Accuracy-efficiency trade-offs: CNNs can achieve higher accuracy while requiring more training time and memory, whereas reducing model layers can speed detection and identification.Processing time was used as an additional criterion when choosing CNN architectures.

12 Discussion

The review finds strong performance in deep-learning weed detection while identifying data, generalisation, computational, and class-imbalance challenges that require further work.

  • Deep-learning models offer high performance for weed detection and classification, with accuracy commonly used as the primary evaluation parameter.
  • Studies typically acquire RGB imagery with digital cameras, preprocess images, annotate plants, and train models using supervised learning and varied deep-learning techniques.
  • Emerging technologies and spectral indices may improve the accuracy and speed of automatic weed detection systems, including under green-on-green visual similarity.
  • Large benchmark datasets should represent diverse crops, weeds, locations, weather conditions, and growth stages, but collecting and annotating them is expensive.
  • Semi-supervised, weakly supervised, self-supervised, unsupervised, synthetic-data, and few-shot or zero-shot approaches could reduce dependence on extensive labelled data.
  • Class imbalance may bias models and cause over-fitting, motivating data redistribution, cost-sensitive learning, or class-balancing classifiers.

13 Conclusion

This survey examines 70 papers on deep-learning weed detection and classification, covering data, methods, datasets, and evaluation. It finds strong reported accuracy with supervised learning and fine-tuned pretrained models, but limited generalisation and deployment readiness.

  • The survey examines 70 papers covering data acquisition, dataset preparation, detection and classification methods, and model evaluation.
  • It provides a taxonomy of research studies and highlights publicly available datasets for prospective researchers.
  • Most reviewed studies applied supervised learning techniques for weed detection and classification.
  • Fine-tuning pretrained deep-learning models on plant datasets was associated with better performance and classification accuracy.
  • High accuracy was achieved mainly when sufficient labelled data for each class was available, while evidence remained limited to small datasets and selected crop and weed species.
  • Real-time deployment remains constrained by recognition speed, and future work calls for efficient models, larger generalised datasets, class-imbalance solutions, growth-stage identification, and field trials.
Loading 2103.01415v1…