Source-linked AI summary
Vine disease detection in UAV multispectral images with deep learning segmentation approach
Mohamed Kerkech, Adel Hafiane, Raphael Canals
TL;DR
The paper addresses UAV-based mapping of vine disease to support more efficient vineyard monitoring and reduce reliance on chemical inputs. It registers and combines visible and infrared imagery, then applies deep-learning segmentation to classify vineyard pixels. The method reports more than 92% detection at grapevine level and 87% at leaf level.
Problem
Vine disease monitoring is laborious and costly, while disease can reduce productivity and pesticide use is under increasing scrutiny.
Method
The method registers visible and infrared UAV images, segments four semantic classes with SegNet, and fuses the segmentation outputs into a disease map.
Results
The proposed method enables vine-symptom detection using visible and infrared UAV imagery and produces fused disease maps.
Takeaways & Limitations
The study supports the potential of machine-learning approaches and UAV remote sensing technology for disease mapping in vineyards.
Takeaways & Limitations
Detection is more difficult for smaller diseased areas, and accurate ground-truth creation limits the results.
Abstract
from arXiv · showhide
One of the major goals of tomorrow's agriculture is to increase agricultural productivity but above all the quality of production while significantly reducing the use of inputs. Meeting this goal is a real scientific and technological challenge. Smart farming is among the promising approaches that can lead to interesting solutions for vineyard management and reduce the environmental impact. Automatic vine disease detection can increase efficiency and flexibility in managing vineyard crops, while reducing the chemical inputs. This is needed today more than ever, as the use of pesticides is coming under increasing scrutiny and control. The goal is to map diseased areas in the vineyard for fast and precise treatment, thus guaranteeing the maintenance of a healthy state of the vine which is very important for yield management. To tackle this problem, a method is proposed here for vine disease detection using a deep learning segmentation approach on Unmanned Aerial Vehicle (UAV) images. The method is based on the combination of the visible and infrared images obtained from two different sensors. A new image registration method was developed to align visible and infrared images, enabling fusion of the information from the two sensors. A fully convolutional neural network approach uses this information to classify each pixel according to different instances, namely, shadow, ground, healthy and symptom. The proposed method achieved more than 92% of detection at grapevine-level and 87% at leaf level, showing promising perspectives for computer aided disease detection in vineyards.
1 Introduction
Vine disease detection is motivated by the costs of disease and pesticide-heavy monitoring, while UAV remote sensing and deep-learning segmentation offer a route to multispectral disease mapping. The proposed method registers visible and infrared imagery, segments semantic classes, and fuses the outputs.
- Motivation: Vine diseases reduce productivity and create economic losses, while conventional leaf inspections require multiple experts over many days.The motivation includes reducing monitoring costs and environmental impacts associated with pesticide use.
- Remote sensing: UAV remote sensing has expanded agricultural applications, including crop monitoring, weed detection, defective-crop detection, and disease detection.Its growing use is linked to technological progress and relatively low manufacturing costs.
- Multispectral imaging: Visible and infrared spectra are both relevant for disease detection, but separate sensors introduce spatial shifts that complicate information processing.The registration problem arises because the two modalities are acquired by different sensors.
- Deep-learning segmentation: Segmentation is framed as preferable to sliding-window CNN detection because it can provide more precise disease-region boundaries.Sliding-window methods generally produce fuzzy boundaries in image regions.
- Proposed methodology: The proposed system registers visible and infrared images, applies SegNet to classify shadow, ground, healthy, and symptomatic vines, and fuses the segmentation outputs.Data were collected under real conditions on two vineyard plots, with experiments assessing contributions of different system components.
2 Related work
Prior work covers image registration and multispectral disease detection, while deep learning and feature-based methods address limitations of conventional approaches. The paper identifies a gap in combining visible and infrared UAV imagery with deep-learning segmentation for vine disease detection.
- Image registration: Image registration research is commonly divided into area-based and feature-based methods.The literature review spans medical imaging, computer vision, remote sensing, and other applications.
- Image registration: Area-based registration can be sensitive to brightness variation and image noise, although automatic cross-correlation has been applied to multimodal images.The reviewed agricultural application illustrates efforts to adapt area-based methods to precision agriculture.
- Image registration: Feature-based UAV registration using SIFT variants outperformed cross-correlation in aerial-image registration studies in both results and runtime.The reviewed comparisons also include SURF-based standard registration methods.
- Disease detection: Multispectral disease studies report that symptoms can appear in infrared imagery several days before visible symptoms, motivating early disease-detection research.Prior work also investigated UAV detection of Flavescence dorée and symptomatic or asymptomatic vines.
- Disease detection: Prior studies used hyperspectral indices, neural-network classifiers, MobileNet, and LeNet5 for vine disease detection, with reported accuracies including 90% and 95%.These studies addressed diseases including Flavescence dorée and Esca using leaf-scale, vehicle-mounted, and UAV RGB imagery.
- Research gap: No prior research had combined visible and infrared UAV images with a deep-learning segmentation approach for vine disease detection.This gap defines the paper’s stated novelty.
3 Study areas and materials
The study used two vineyard plots in France, including diseased and healthy conditions, and acquired high-resolution visible and infrared imagery with a quadcopter UAV. The plots and acquisition setup define the real-world test conditions.
- Study areas: The study used two vineyard plots in France: P1 covered 1.8 hectares and P2 covered 1.5 hectares.P1 had silty sand soil and a 7% slope, whereas P2 had sandy loam soil and was flat.
- Study areas: P1 was divided into treated and untreated areas to obtain healthy and diseased samples, while P2 was treated globally and used for qualitative validation.Late blight spread through the untreated areas of P1.
- Documentation: The study plots and acquisition materials were documented through satellite, ground-view, and UAV-system figures.The supplied captions identify P1 and P2 in satellite imagery and the quadcopter and Survey2 sensor in the materials figure.
- Materials: The acquisition platform was a Scanopy quadcopter carrying two 16-megapixel MAPIR Survey2 sensors.The sensors captured visible RGB and infrared-related channels, including NIR, red, and NDVI; the NIR wavelength was 850 nm.
- Data acquisition: The UAV flew at 25 meters altitude and 10 km/h, capturing images every 2 seconds with more than 70% overlap.The ground resolution was 1 cm2/pixel, and average flight autonomy was 20 minutes.
4 Methods
The method registers visible and infrared UAV images, segments both modalities into four classes, and fuses their outputs into a vineyard disease map.
- Overall system: The system registers visible and infrared images, segments each modality with SegNet, and merges the segmented images into a disease map.Registration enables pixel-wise superposition before segmentation fusion.
- Image registration: The registration algorithm uses a non-rigid model because UAV vibrations, rolling-shutter acquisition, and optics can distort images.A rigid translation-and-rotation model is considered inappropriate for these distortions.
- Image registration: AKAZE extracts and describes corresponding features from normalized visible green and infrared near-infrared channels.The channels are selected for vegetation texture information and normalized to improve contrast.
- Image registration: Dynamic RANSAC thresholding validates a homographic transformation by projected-coordinate distance before producing the registered infrared image.The threshold is adjusted until the geometric correction is considered coherent.
- Image registration: An iterative RMSE-based phase repeats registration when the error decreases, using matched visible and infrared feature points.RMSE is computed from coordinate discrepancies between corresponding points.
- Segmentation and fusion: SegNet classifies pixels as shadow, ground, healthy, or symptomatic in visible and infrared imagery, then fusion uses intersection or union rules.Fusion AND requires symptoms in both modalities, whereas Fusion OR combines detections from either modality.
5 Experimentation
The experiments evaluate registration and segmentation, using semi-automatic labels, patch-based SegNet training, blockwise testing, and leaf- and grapevine-level metrics.
- Experimental design: The experiments comprise separate evaluations of visible–infrared image registration and the subsequent segmentation and fusion stages.The experimentation section is explicitly divided into these two subsections.
- Image registration: Registration performance is measured with RMSE between matched points to quantify geometric correction and residual shift.The results include standard and optimized registration statistics for 150 images.
- Data preparation: Semi-automatic labelling combines LeNet5 pre-labelling with manual correction reinforced by technicians’ field ground truth.The procedure addresses the volume of data and difficulty of producing accurate labels.
- Data preparation: Each visible and infrared patch dataset contains 70,560 32 × 32 samples across shadow, ground, healthy, and symptomatic classes.Each class contributes 17,640 samples, including 14,994 for training and 2,646 for validation.
- Model training: SegNet training uses 360 × 480 labelled patches for 100,000 iterations with batches of five randomly selected patches.Visible and infrared models are trained separately.
- Evaluation procedure: Testing divides 4608 × 3456 UAV images into non-overlapping 360 × 480 blocks, segments them, and stitches the blocks back together.The stitched output restores the original image size for comparison with ground truth.
- Performance measurement: Performance is assessed at leaf level with recall, precision, F1/Dice, and accuracy, and at grapevine level using 64 × 64 windows.Grapevine-level evaluation compares the dominant ground-truth class with the SegNet estimate.
6 Discussion
The discussion evaluates multispectral registration, segmentation, and fusion for vine disease detection, showing improved alignment and strong grapevine-level performance while identifying runtime and resolution-related constraints.
- 6.1 Image registration: After seven iterations, dynamic-threshold correspondence detection achieved good alignment of visible and infrared vine rows.
- 6.1 Image registration: The optimized registration reduced average RMSE from 3.29 pixels to 2.43 pixels across 150 images.The 0.86-pixel reduction was attributed to newly detected correspondence points and updated homographic matrices.
- 6.1 Image registration: The optimized method increased average runtime from 92 seconds to 139 seconds because of additional processing and 3.12 iterations on average.
- 6.1 Image registration: AKAZE produced slightly lower error than SIFT with runtimes 2 to 10 times shorter, while other algorithms often failed because of modality differences.
- 6.2 Image segmentation and fusion: 90.23% accuracy was obtained with union fusion, exceeding visible-only accuracy of 85.13% and infrared-only accuracy of 78.72%.Intersection fusion reached 82.20%, reflecting its dependence on detections shared across modalities.
- 6.2 Image segmentation and fusion: More than 92.81% grapevine-level detection was achieved with union fusion, compared with 91.50% for visible imagery and 81.66% for infrared imagery.The evaluation included recall, precision, F1-Score/Dice coefficient, and accuracy at leaf and grapevine levels.
- 6.2 Image segmentation and fusion: SegNet struggled more with small diseased areas, partly because downsampling and upsampling can lose resolution information.Accurate ground truth and low-resolution imagery at high UAV altitude also limited results.
- 6.2 Image segmentation and fusion: Fusion by intersection identifies locations detected in both modalities, indicating higher system confidence, while also exposing modality-specific detections.
7 Conclusion
The study proposes a three-step method for detecting vine disease in multimodal UAV images by aligning visible and infrared data, segmenting four classes, and fusing the results into disease maps.
- The method aligns visible and infrared UAV images using an iterative interest-point-based algorithm.
- SegNet segments visible and infrared images into shadow, ground, healthy vine, and symptomatic vine classes.
- A disease map is generated by fusing the segmentations from the visible and infrared images.
- The method detects vine symptoms using information from visible and infrared spectra and provides a framework for exploring earlier disease detection and mapping.
- The small training sample reduced deep-learning segmentation performance, motivating future dataset enrichment and method improvements.