PaperlayerView exact revision on arXiv

Source-linked AI summary

Automatic Model Based Dataset Generation for Fast and Accurate Crop and Weeds Detection

Maurilio Di Cicco, Ciro Potena, Giorgio Grisetti, Alberto Pretto

arXiv:1612.03019v3cs.CVcs.RO

TL;DR

Pixel-wise agricultural annotations are time-consuming to produce, limiting the practical training-data supply for crop-and-weed detection. The paper procedurally generates labeled scenes from modeled environmental parameters and real-world textures; its synthetic data achieves similar performance to real annotated images, and can supplement limited real data.

  • Problem

    Training appearance-based crop-and-weed models requires large annotated datasets spanning plant growth stages and weather conditions, but pixel-wise annotation is extremely time-consuming.

    Source: §I · p. 1

  • Method

    The paper models agricultural scenes and procedurally generates varied synthetic training data by randomizing environmental parameters and using a few real-world textures.

    Source: §I · p. 2, B. Contributions · p. 3, §II · p. 3

  • Results

    Synthetic-data-trained models show similar performance to models trained on real-world human-annotated images, and synthetic data can supplement limited real datasets.

    Source: §IV · p. 8

  • Takeaways & Limitations

    Synthetic generation removes human intervention in the labeling phase and reduces the time and effort needed to train a visual inference model.

    Source: §IV · p. 8

    Realistic NIR image rendering remains future work, for which the authors propose investigating a chlorophyll model.

    Source: §IV · p. 8

Abstract

from arXiv · show

Selective weeding is one of the key challenges in the field of agriculture robotics. To accomplish this task, a farm robot should be able to accurately detect plants and to distinguish them between crop and weeds. Most of the promising state-of-the-art approaches make use of appearance-based models trained on large annotated datasets. Unfortunately, creating large agricultural datasets with pixel-level annotations is an extremely time consuming task, actually penalizing the usage of data-driven techniques. In this paper, we face this problem by proposing a novel and effective approach that aims to dramatically minimize the human intervention needed to train the detection and classification algorithms. The idea is to procedurally generate large synthetic training datasets randomizing the key features of the target environment (i.e., crop and weed species, type of soil, light conditions). More specifically, by tuning these model parameters, and exploiting a few real-world textures, it is possible to render a large amount of realistic views of an artificial agricultural scenario with no effort. The generated data can be directly used to train the model or to supplement real-world images. We validate the proposed methodology by using as testbed a modern deep learning based image segmentation architecture. We compare the classification results obtained using both real and synthetic images as training data. The reported results confirm the effectiveness and the potentiality of our approach.

I. INTRODUCTION

Crop-and-weed detection is important for agricultural robots, but training data with pixel-level annotations is costly to collect. The paper proposes procedurally generated agricultural scenes as training data and reports results approaching those from real data.

I. INTRODUCTION

  • Large annotated datasets are costly because images must cover plant growth stages and weather conditions and carry accurate pixel-wise labels.Manual pixel-wise annotation is described as extremely time consuming.Source: §I · p. 1
  • Procedural generation uses a few real-world textures and randomized environmental parameters to create synthetic agricultural scenes without manually crafting each scene.The parameters include such features as weather conditions and plant size.Source: §I · p. 2
  • Synthetic training data can approach the accuracy reached with real data, and adding a small number of real images yields even better results.The reported synthetic-data result holds even when the virtual scene does not contain all weed species.Source: §I · p. 2

1) Plants Classification:

The section reviews plant classification and approaches to reducing the effort of creating training datasets. It describes the difficulty of distinguishing similar plant species and surveys synthetic-data, transfer-learning, and dataset-reduction strategies.

1) Plants Classification:

  • Plant classification is a fine-grained recognition problem because differences between similar species can be minimal.The section frames plant species as subordinate categories in fine-grained visual classification.Source: 1) Plants Classification · p. 2
  • Earlier crop-and-weed classifiers used texture, multispectral reflectance, neural networks, random forests, and CNNs, with reported classification rates including 80% for sugar beet and 91% for weeds.A cell-based neural-network approach reported accuracy up to 75%; later random-forest work extended features and added a plant-arrangement prior,.Source: 1) Plants Classification · p. 2

2) Synthetic Dataset Generation:

  • Synthetic-data studies used video games and handcrafted virtual worlds, but closed-source games limit customization and manually crafting worlds still takes time,,,.Prior work reported gains from combining synthetic and real images, as well as generalization to real data using synthetic training data alone.Source: 2) Synthetic Dataset Generation · p. 2
  • Transfer learning is another approach to mitigating labeling effort, and feature transferability varies with the distance between base and target tasks,.Source: 2) Synthetic Dataset Generation · p. 2
  • Dataset reduction can select a subset of images that maximizes visual entropy from a larger dataset.Source: 2) Synthetic Dataset Generation · pp. 2–3

B. Contributions

The approach models relevant agricultural-scene aspects without manually building the entire scene, then randomizes model parameters to generate many scenes procedurally.

B. Contributions

  • The approach explicitly models relevant aspects without manually creating the entire scene.Source: B. Contributions · p. 3
  • Modulating and randomizing model parameters enables rapid procedural generation of a large number of scenes.Source: B. Contributions · p. 3

II. PROCEDURAL DATASET GENERATION

The procedural pipeline aims to produce realistic, varied, fully labeled agricultural scenes. Its leaf model varies plant forms through a kinematic skeleton, randomized joint angles, and real-world textures.

II. PROCEDURAL DATASET GENERATION

  • The pipeline treats realism and variety as its two goals, sampling fully labeled training images from a randomized rendering process.It models leaves, assembles plants, and scatters them on realistic soil before rendering the field.Source: §II · p. 3

A. Description of the Leaf Model

  • A leaf’s kinematic tree runs from the stem through the principal vein to the tip, with mirrored branches following secondary veins.The leaf skeleton is attached to a planar surface mesh.Source: A. Description of the Leaf Model · p. 3, Figure 2 · p. 3
  • Joint posture uses a single rotation around the z axis, while joint counts and distances vary leaf shape and growth stage.Changing chain angles can bend leaves to resemble physical effects such as gravity; random angles produce a wider range of forms.Source: A. Description of the Leaf Model · p. 3
  • Leaves use high-definition RGBA textures from real-world pictures, with the alpha channel serving as an opacity mask.The skeleton is associated with a planar mesh and physically based shading.Source: A. Description of the Leaf Model · p. 3
  • Normal, ambient-occlusion, and height maps add surface detail, light approximation, and definition to the leaf model.The figure also shows the mesh, maps, shading, and skeleton.Source: A. Description of the Leaf Model · pp. 3–4, Figure 3 · p. 3

B. Plant Modeling

Plants are procedurally modeled with a multi-layer radial arrangement of leaves. Species-specific leaf and layer counts are fixed, while other parameters vary plant instances and growth stages.

B. Plant Modeling

  • A generic multi-layer radial distribution models plant species using leaf count, leaf layers, and leaf size as structural properties.The model also parameterizes the growth-stage axis by its position and direction relative to gravity.Source: B. Plant Modeling · p. 4
  • The system fixes species-specific leaf count, layer count, and size, while modulating position, direction, arrangement offset, and leaf-size variation.The fixed values are chosen according to.Source: B. Plant Modeling · p. 4
  • Each generated dataset includes plants from two growth stages to increase variation.The passage notes that the initial growth stage is especially important for the targeted treatments.Source: B. Plant Modeling · p. 4
  • The terrain-generation example blends two soil textures using a Perlin-noise intensity image as the selector.The caption describes this as linear interpolation between the input textures.Source: Figure 6 · p. 5

C. Virtual Dataset Generation

The virtual environment combines varied soil textures, lighting, and randomly spawned objects. Perlin-noise blending varies terrain, while camera parameters approximate the real imaging setup.

C. Virtual Dataset Generation

  • Soil is generated by blending real-world textures with Perlin-noise linear interpolation and associated surface maps.The textures represent soil types such as dirt, cracked dirt, and stony ground.Source: C. Virtual Dataset Generation · p. 5
  • Perlin-noise parameters vary terrain appearance and vertex displacement, while different light sources create varied illumination and shadowing.Height and normal maps are blended to produce the displacement.Source: C. Virtual Dataset Generation · p. 5
  • Plants, weeds, rocks, and sticks are randomly spawned, oriented from the local soil normal with noise, then projected using camera parameters approximating the real camera.Source: C. Virtual Dataset Generation · p. 5

D. Artificial Ground Truth Generation

Ground-truth masks are produced by assigning each target object its class color as an emissive component. The rendered example pairs an RGB scene with a color-coded label image.

D. Artificial Ground Truth Generation

III. EXPERIMENTS

The experiments test whether procedural synthetic data generated with a few real-world textures can train a modern learning framework comparably to real data, reducing human effort for data acquisition and labeling.

III. EXPERIMENTS

  • The experiment tests whether procedural synthetic data can produce results comparable to real-data training and reduce the effort required to acquire and label data.The authors also propose that synthetic data can supplement small real datasets.Source: §III · p. 5

A. Experimental Setup

  • Real A and Real B contain 700 and 900 manually labeled images from early and later sugar beet growth stages, respectively.Images were captured at 1 Hz with a downward-facing camera on a BOSCH Bonirob; annotation typically took 5 to 30 minutes per image.Source: A. Experimental Setup · pp. 5–6
  • Four synthetic datasets of 1,300 images vary weed species; Real-Augmented adds 100 images each from Real A and Real B to Synthetic D.Synthetic data uses RGB only; Test Sets A and B are held-out subsets of the real datasets, and images are resized to 480 × 360 pixels.Source: A. Experimental Setup · p. 6
  • Evaluation reports global and per-class accuracy, average intersection over union, precision, and recall as percentages.The datasets were collected using a four-channel RGB-NIR camera, although the synthetic-data comparison uses RGB images.Source: A. Experimental Setup · p. 6

B. SegNet

The paper evaluates its synthetic datasets using SegNet and an eight-layer smaller variant, Basic SegNet.

B. SegNet

  • SegNet performs pixel-wise semantic classification with an encoder, decoder, and pixel-wise softmax layer.Unlike the authors’ earlier patch-and-voting approach, it can discriminate overlapping plants.Source: B. SegNet · p. 6
  • The evaluation compares 26-layer SegNet with Basic SegNet, which has eight layers.The larger network targets many output classes and may overfit when only a few classes are used.Source: B. SegNet · p. 6
  • The authors use the two network variants to address overfitting risk in the small-output-class setting.SegNet is described as particularly suitable because it can classify pixels even when plants overlap.Source: B. SegNet · p. 6

C. Crop/Weed Classification and Vegetation Detection

Real-Augmented often outperforms real-data training, while synthetic-only training can yield noteworthy results and approximate real-data performance. Vegetation detection results also favor Basic SegNet over the larger SegNet.

C. Crop/Weed Classification and Vegetation Detection

  • Figure 8 shows input real RGB images, ground-truth labels, and segmentation results from an RGB Basic SegNet trained on synthetic datasets.The figure presents qualitative examples rather than additional numerical comparisons.Source: Figure 8 · p. 7
  • 91.3% per-class average accuracy on Test Set B was achieved by Real-Augmented, which often outperformed real-data training.The authors also report very good average intersection over union for both test sets.Source: C. Crop/Weed Classification and… · p. 7
  • Synthetic B and Synthetic D produced noteworthy results when RGB Basic SegNet was trained solely on synthetic data.The real datasets contain mostly Capsella Bursa-Pastoris and few Galium Aparine instances, a distribution the authors suggest may explain differing synthetic-dataset results.Source: C. Crop/Weed Classification and… · p. 7
  • Adding weed species incrementally to synthetic datasets yielded a monotone positive performance trend, approaching real-data performance when real images were added.The full RGB SegNet generally performed poorly, including on real data, mainly due to overfitting.Source: C. Crop/Weed Classification and… · p. 7
  • Synthetic datasets approximated real-dataset results for soil-vegetation classification, and RGB Basic SegNet performed better than RGB SegNet across cases.The evaluation did not use NIR; the authors are investigating a model to generate synthetic NIR for vegetation detection.Source: C. Crop/Weed Classification and… · p. 7, Table II · p. 8

IV. CONCLUSIONS

The paper procedurally generates annotated plant data by explicitly modeling the target environment and using a few real-world textures. The authors report performance similar to training on human-annotated real images, with synthetic data also serving to supplement limited real datasets.

IV. CONCLUSIONS

  • The approach models the target environment and uses a few real-world textures to procedurally generate varied annotated plant data.The stated aim is to minimize human effort in training visual classification systems.Source: §IV · p. 8
  • Synthetic training data removes human intervention in the labeling phase and reduces the time and effort needed to train a visual inference model.The authors report performance similar to a model trained with human-annotated real-world images.Source: §IV · p. 8
  • Synthetic data can supplement a limited real dataset, and the authors suggest a virtual dataset containing all weeds found in a real field could reach real-image accuracy.The all-weeds outcome is presented as a suggestion, not a demonstrated result.Source: §IV · p. 8
  • Generating realistic NIR images is future work; the authors plan to investigate a chlorophyll model for this purpose.They identify NIR as a common and useful data source in precision agriculture.Source: §IV · p. 8
Loading 1612.03019v3…