Context-Aware Adaptive Pesticide Spraying for Agricultural Robots under Changing Weather and Terrain Using Vision-Language Models

1. Introduction

• Manual operators can adapt spraying to crop conditions and weather, but this flexibility relies on experience, is physically demanding, and can produce inconsistent performance.Mechanized and semi-automated systems commonly apply fixed rates without accounting for spatial variability, contributing to excessive deposition and chemical waste.Source: §1 · p. 1
• Prior work addresses crop detection, targeted spraying, and flow or nozzle adjustments, but challenges remain in adapting strategies to varying conditions.Examples include canopy-volume-based flow adjustment and active nozzle orientation to reduce off-target deposition.Source: §1 · pp. 1–2
• Farmers adapt pesticide amounts, movement direction, and speed to plant density, canopy structure, disease status, weather, and pesticide characteristics.Prior studies have largely overlooked weather and pesticide factors and assumed fixed movement trajectories.Source: §1 · p. 2
• Crop-row detection using VLMs improves accuracy by at least 30% over unsupervised algorithms and manual adjustments.The framework is evaluated in two field environments and generates adaptive strategies while helping reduce pesticide loss caused by environmental factors.Source: §1 · p. 2
• The proposed framework uses VLMs to generate adaptive pesticide strategies based on crop types, operational context, and environmental factors such as wind direction and temperature.An MPPI-based controller tracks spraying trajectories while avoiding obstacles during navigation.Source: §1 · p. 2

2. System Design

• The robot uses a tracked Bunker Mini chassis for stable operation on uneven agricultural terrain.The platform is illustrated in Fig. 1.Source: §2 · p. 2
• A ZED X camera supports crop detection, while a Livox Mid-360 3D LiDAR supports spatial mapping; both connect to the Jetson Orin onboard computer.The Jetson Orin performs sensor processing and algorithm execution.Source: §2 · p. 2
• The Jetson Orin sends velocity commands and receives status information over a CAN bus.Figure 1 also depicts the processor generating mobile-base movement commands.Source: §2 · p. 3, Figure 1 · p. 3
• Two independent side-mounted spray modules enable separate left- and right-side decisions, with adjustable nozzles and pump speeds controlled through an Arduino Uno.The Arduino receives Jetson commands and converts them to PWM signals for the ESCs; the pumps draw from a 20-liter tank.Source: §2 · p. 3

3.1. Architecture Overview

• Fast-LIO2 builds a global map from LiDAR/IMU data, while YOLOv8 locates trees for the working map and subsequent VLM planning.The architecture uses the map, tree positions, environmental images, weather, and pesticide type to inform planning and spraying decisions.Source: §3.1 · pp. 3–4

3.2. VLM-Based Global Path Planning

• The VLM classifies rows from a compact 2D tree-coordinate and ID representation, then adjacent-row centerlines form a connected zig-zag path.Few-shot prompts constrain row identification to supplied IDs and prescribe the output format.Source: §3.2 · pp. 4–5

3.3. VLM-Based Spraying Strategy

• The VLM combines onboard RGB, a row-and-path visualization, wind, crop and pesticide information, and weather conditions to choose travel speed and nozzle flow rate.RGB imagery informs terrain and canopy assessment; auxiliary context includes cultivation location, season, humidity, temperature, and wind speed.Source: §3.3 · pp. 5–6
• Each plant receives VLM-selected speed and spray volume, with spraying triggered when the robot is within a distance threshold and on the appropriate side.The speed command is forwarded to the path-following controller.Source: §3.3 · p. 6

3.4. Path Following Controller

• MPPI tracks the global path amid terrain and other disturbances, combining tracking, local-costmap obstacle-avoidance, and VLM-speed costs.The controller uses CUDA parallel computation and runs at 100 Hz.Source: §3.4 · p. 6

4. Experiments Results and Discussion

• Evaluation covers global-planning crop detection, VLM spraying in two real-world environments, and controller performance, followed by discussion of insights and improvement areas.Source: §4 · p. 6

4.1. Crop Row Detection Evaluation

• The comparison uses crop-distribution data from 20 environments, repeats each method 10 times, and evaluates six VLMs.The six models are Gemini 2.0 Flash Lite, Gemini 2.5 Flash Lite, Gemini 2.5 Flash, GPT-4.1 Nano, GPT-4.1 Mini, and GPT-4.1.Source: §4.1 · pp. 6–7
• Approximately 30–40% performance gap separates VLMs from clustering methods; Kmeans scores 60.80% accuracy and DBSCAN 59.96%.The reported comparison shows VLMs outperforming traditional clustering-based methods.Source: Figure 5 · p. 7, §4.1 · p. 7
• GPT-4.1 Nano reaches 91.87% accuracy, below larger VLMs but still significantly above traditional methods.The reported accuracy counts correctly identified trees belonging to tree rows.Source: §4.1 · p. 7
• 2.92 s is the lowest average latency, for Gemini 2.5 Flash Lite, while GPT-4.1 averages 11.65 s.Gemini 2.0 Flash Lite averages 3.88 s, GPT-4.1 Nano 3.93 s, GPT-4.1 Mini 5.38 s, and Gemini 2.5 Flash 7.28 s.Source: Figure 6 · p. 7, §4.1 · p. 7
• Gemini 2.5 Flash was selected for high recognition accuracy while maintaining efficient inference performance.Source: §4.1 · p. 7

4.2.1. Experimental Setup

• Tests used an 8,000 m2 woodland section and a 5,000 m2 uneven mango farm, with API wind direction and anemometer speed informing adaptive decisions.The anemometer was mounted at nozzle height; speed and spray flow were automatically determined at three time points in each environment.Source: §4.2.1 · pp. 7–8

4.2.2. Experimental Results

• The VLM identified tree rows in both environments, and the generated global paths aligned with the rows requiring spraying.The maps captured irregular tree distributions, including tilted or misaligned trunks and mango rows of varying lengths.Source: §4.2.2 · p. 8
• At 2.0–2.3 m/s wind, flow rates stayed near 2.85–2.90 L/min; at 3.5–4.5 m/s, they varied from about 2.7 to 3.0 L/min.Flow adjustments also reflected wind direction and individual tree position and density.Source: §4.2.2 · pp. 8–9
• 26.4% average surface coverage and 62 droplets/cm2 were reported for the VLM strategy, versus 61.7% average surface coverage for the fixed-rate baseline.The baseline showed excessive overlap and clustering, while the VLM strategy produced more uniform, adaptive deposition.Source: §4.2.2 · pp. 9–10
• As wind speed rose from 2.3 to 3.5 m/s, Environment 1's average travel speed fell from 0.93 to 0.55 m/s; Environment 2 ranged from 0.71 to 0.47 m/s.The robot maintained lower speeds in Environment 2, where uneven terrain and dense vegetation required continued adjustment.Source: §4.2.2 · p. 10

4.3. Controller Evaluation

• In Environment 2, lateral error |e_y| rose from 3.8±0.36 cm at low speed to 6.9±0.54 cm at 1.5 m/s, while spraying distance stayed within 2.65–2.76 m.Lateral error exceeded longitudinal error, and errors were consistently higher in Environment 2.Source: §4.3 · p. 10
• Tracking error remained below 8 cm across operating speeds, with stable standard deviation; low- and medium-speed errors were sufficiently small for precision spraying.The evaluation tested 0.5, 1.0, and 1.5 m/s in two environments.Source: §4.3 · pp. 10–11
• Operating speed and environment significantly affected lateral error (p<0.05), while longitudinal error was mainly influenced by speed and spraying-distance differences were statistically significant (p<0.05).The effect of speed on spraying distance was minor and depended mainly on terrain characteristics.Source: §4.3 · p. 11

4.4.1. Performance of VLMs in Crop-Row Detection

• VLM-based row detection improves accuracy by approximately 30% to 40% over K-means and DBSCAN.Source: §4.4.1 · p. 11

4.4.2. VLM-Driven Context-Aware Spraying Strategy

• The framework uses wind, foliage density, and terrain to adjust pesticide flow rates and operating speed, reducing chemical waste while maintaining coverage.Source: §4.4.2 · p. 11
• Slight prompt phrasing changes do not significantly alter speed or flow-rate outputs when environmental parameters remain consistent.Few-Shot and CoT prompting are credited with supporting this stability; system reliability also requires hardware-compatible output formats.Source: §4.4.2 · p. 11
• Higher VLM latency does not prevent field deployment in the tested low-speed setting, where lower-frequency planning is paired with higher-frequency MPPI tracking and collision avoidance.Reducing latency remains a future direction, particularly for applications needing faster reactive responses.Source: §4.4.2 · p. 11

4.4.3. System Limitations and Future Work

• Evaluation covered a restricted number of field environments, farms, and locations, leaving generalizability across diverse agricultural settings to further testing.Source: §4.4.3 · p. 11
• Controller performance remains sensitive to terrain variability; mango-orchard wheel–terrain interactions increased lateral tracking errors.The authors propose terrain-aware trajectory optimization for smoother traversable regions and note that stabilization or suspension improvements may help in rugged fields.Source: §4.4.3 · pp. 11–12
• The complete source code and full datasets are not publicly available, although detailed pseudocode for the core methodology is provided.Source: §4.4.3 · p. 12

• The VLM framework synthesizes imagery with weather, time, pesticide type, location, and crop variety to select adaptive spraying parameters.Source: §5 · p. 12
• An MPPI controller supports precise and safe trajectory tracking during field operations.Source: §5 · p. 12
• Crop-row detection accuracy improves by at least 30% over conventional methods in evaluations across two agricultural environments.Source: §5 · p. 12
• Future work will validate the framework in more complex, unstructured settings and deploy VLMs locally on edge devices to reduce latency.Source: §5 · p. 12

• The authors used ChatGPT and Gemini for readability edits, then reviewed and edited the manuscript and took responsibility for its final content.Source: Declaration of Generative AI and… · p. 12

• Cong-Thanh Vu is credited with the original draft, methodology, investigation, formal analysis, data curation, and conceptualization.Source: CRediT authorship contribution statement · p. 12
• Yen-Chen Liu is credited with review and editing, supervision, funding acquisition, and conceptualization.Source: CRediT authorship contribution statement · p. 12