Source-linked AI summary
The New Agronomists: Language Models are Experts in Crop Management
Jing Wu, Zhixin Lai, Suiyao Chen, Ran Tao, Pan Zhao, Naira Hovakimyan
TL;DR
Crop management must optimize yields and profitability while limiting environmental impacts, yet current guidelines may be uncertain under changing conditions and earlier RL agents relied on limited neural-network architectures. The paper combines deep RL, a language model, and DSSAT-based crop simulations, converting state variables into descriptive language for the agent. In maize simulations in Florida and Zaragoza, the framework outperformed previous state-of-the-art approaches and empirical baselines across key management metrics.
Problem
Crop-management guidelines may be uncertain under changing climate and market conditions, while earlier crop-management RL agents used MLPs with limited fitting power and descriptive-state representation.
Source: §1 · pp. 1–2Method
The framework uses DQN with a language model to encode simulator states converted into descriptive sentences, training crop-management policies with DSSAT and Gym-DSSAT simulations.
Source: §1 · p. 2, §3.3 · p. 4Results
In maize simulations in Florida and Zaragoza, LM-based RL outperformed previous state-of-the-art approaches and empirical baselines across yield, resource use, and environmental impact.
Source: §1 · p. 2, §4.3 · p. 6, §6 · p. 8Takeaways & Limitations
The authors conclude that LM-based agents can adjust strategies to different reward designs and achieve high rewards across varied crop-management scenarios.
Source: §6 · p. 8Applying RL policies refined in simulations to practical environments faces the sim-to-real gap.
Source: §5 · p. 8
Abstract
from arXiv · showhide
Crop management plays a crucial role in determining crop yield, economic profitability, and environmental sustainability. Despite the availability of management guidelines, optimizing these practices remains a complex and multifaceted challenge. In response, previous studies have explored using reinforcement learning with crop simulators, typically employing simple neural-network-based reinforcement learning (RL) agents. Building on this foundation, this paper introduces a more advanced intelligent crop management system. This system uniquely combines RL, a language model (LM), and crop simulations facilitated by the Decision Support System for Agrotechnology Transfer (DSSAT). We utilize deep RL, specifically a deep Q-network, to train management policies that process numerous state variables from the simulator as observations. A novel aspect of our approach is the conversion of these state variables into more informative language, facilitating the language model's capacity to understand states and explore optimal management practices. The empirical results reveal that the LM exhibits superior learning capabilities. Through simulation experiments with maize crops in Florida (US) and Zaragoza (Spain), the LM not only achieves state-of-the-art performance under various evaluation metrics but also demonstrates a remarkable improvement of over 49\% in economic profit, coupled with reduced environmental impact when compared to baseline methods. Our code is available at \url{https://github.com/jingwu6/LM_AG}.
1. Introduction
Crop management must balance production with environmental impacts, while existing guidelines may be uncertain under changing climate and market conditions. This paper asks whether language models can improve crop-management RL and introduces an LM-based agent evaluated in maize simulations.
1. Introduction
- Crop-management choices, especially nitrogen fertilization and irrigation, affect both yields and environmental health, while existing practices face uncertainty under changing conditions.The paper frames adaptable management as important for goals such as maximizing economic profit.Source: §1 · p. 1
- The paper asks whether language models can address limitations of MLP-based crop-management RL agents and serve as viable alternatives.Prior agents had limited fitting power and difficulty incorporating descriptive environmental, soil, and crop-growth features.Source: §1 · pp. 1–2
- The framework converts simulator states into descriptive sentences and uses a language model as the RL agent with DSSAT and Gym-DSSAT simulations.The agent encodes the sentences into embeddings for state representation.Source: §1 · p. 2
- In Florida and Zaragoza maize simulations, the framework outperformed previous state-of-the-art approaches and empirical baselines across yield, resource use, and environmental impact.The baselines included expert-recommended production guidelines and surveyed farmer practices.Source: §1 · p. 2
2. Related Work
Prior work applied RL to crop management and connected agents to crop simulators, while newer work explored language models as decision-makers. This paper presents the first effort, to the authors’ knowledge, to use LMs to formulate crop-model management strategies in agriculture.
2.1. Crop Management with RL
- Crop-management RL research progressed from narrow state and action spaces to broader features and imitation learning, but some approaches did not significantly surpass baselines.Studies addressed wheat, maize irrigation, fertilization, and partially observed management.Source: §2.1 · pp. 2–3
2.2. Crop Models for RL
- Crop simulators avoid laborious, time-consuming, and expensive real-world farming experiments, while Gym-DSSAT enables detailed daily agent interactions with simulated conditions.Some crop-model environments oversimplify crop and environmental details.Source: §2.2 · p. 3
2.3. LM for Decision-making
- The authors identify their work as the first endeavor, to their knowledge, to use language models to formulate optimal management strategies for agricultural crop models.Earlier language-model decision agents had been explored in other domains, including user-interface interaction and robot control.Source: §2.3 · p. 3
3. Methods
The methods formulate daily nitrogen and irrigation decisions as an MDP, represent simulator states in language for a BERT-based RL agent, and train its action-value function with DQN. The agent interacts with simulated crop conditions through Gym-DSSAT.
3.1. Problem Formulation
- The MDP gives the agent a daily environmental state and nitrogen and irrigation actions, with a reward based on yield at harvest and weighted management costs and nitrate leaching.The policy seeks to maximize future discounted return; reward weights steer the balance among objectives.Source: §3.1 · p. 3
3.2. LM-based RL Agent
- The agent converts simulator variables into sentences and uses pretrained distilled BERT to encode them into embeddings, followed by fully connected layers that map to actions.The sentence format makes variable names and values more interpretable to language models.Source: §3.2 · pp. 3–4
3.3. Policy Training with LM
- DQN training uses distilled BERT to represent the action-value function, which estimates expected future discounted return for a state-action choice.The optimal policy selects the action with the highest Q value, and training minimizes a loss using replay-buffer tuples and a target network.Source: §3.3 · p. 4
3.4. Crop Simulations with Gym-DSSAT
- Gym-DSSAT lets the agent read weather, soil, and crop information and apply management practices in the simulated environment daily.It provides the simulation interface for agent-environment interaction.Source: §3.4 · p. 4
4. Experiments and Results
The experiments evaluate LM-based RL policies for maize nitrogen and irrigation management in simulated Florida and Zaragoza settings, using several reward functions to represent different objectives. The reported policies outperform prior methods and empirical baselines in most metrics and scenarios, while ablations examine joint management and alternative agent architectures.
4.1. Experimental Setup
- The Florida 1982 and Zaragoza 1995 simulations tested nitrogen and irrigation policies for demonstrating framework viability, not immediate real-world deployment.Policies were trained under full observation using DQN and compared with local guideline- or survey-based baselines.Source: §4.1 · p. 4
4.2. Implementation Details and Evaluation Metrics
- Training used five reward functions to span trade-offs among yield, nitrogen, irrigation, and environmental impact; RF5 assigns substantially higher weight to nitrate leaching.RF2 treats irrigation water as free, RF3 treats nitrogen fertilizer as free, and RF4 doubles the nitrogen price relative to RF1.Source: §4.1 · p. 4, §4.2 · p. 5
- State inputs were encoded with end-to-end-trained DistilBERT and passed through fully connected layers of 512 and 256 units before mapping to actions.Numerical inputs were normalized to [0, 300] and tokenized using integer portions to address instability from numerical tokenization.Source: §4.2 · p. 5
4.3. Results of Experiments
- 49% and 67% increases in RF1 profit, and almost 445% and 37% increases in RF5, are reported for Florida and Zaragoza, respectively, alongside outperformance of prior methods and empirical baselines in most metrics and scenarios.The comparisons are against both previous state-of-the-art methods and empirical baselines.Source: Table 2 · p. 6, §4.3 · p. 6, Table 3 · p. 7
- Reward-function choice changed learned strategies: RF2 favored more irrigation and less nitrogen, while RF3 favored more nitrogen and less irrigation.Under RF4, the policy reduced nitrogen and saved over 64% of water while still achieving substantial yield.Source: §4.3 · p. 6
4.4.1 Training Separately on Fertilization and Irrigation
- Jointly optimizing nitrogen fertilization and irrigation produced superior outcomes to optimizing either practice independently in the Florida RF1 ablation.In the independent-policy comparisons, the non-optimized practice followed an established baseline protocol.Source: §4.4.1 · p. 7, Table 4 · p. 7
4.4.2 Exploration of Framework
- A three-layer MLP performed best among traditional RL agents, while scaling to ResNet152 reduced performance; the LM showed improved performance and avoided the overfitting observed with larger neural networks.The authors interpret the ResNet152 decline as suggesting overfitting; the framework exploration used the Florida case with RF1.Source: §4.4.2 · p. 7
5. Path to Deployment
The section examines the limits of transferring crop-management policies from DSSAT simulations to real settings and evaluates performance under measurement noise. It reports that noisy-input performance remained above baseline, while real-world robustness evaluation is left for future work.
5. Path to Deployment
- Policies trained in DSSAT simulations may not transfer directly to real agriculture because of weather uncertainty and differences between crop models and actual systems.The paper identifies this sim-to-real gap as a common challenge for policies developed and refined in simulation.Source: §5 · pp. 7–8
5.1. Closing the Sim-To-Real Gap
- Real-world robustness evaluation is deferred to future work, with domain and dynamics randomization of model parameters and weather proposed during training.The authors say the current study establishes the framework and assesses its effectiveness; a forthcoming study will target robustness to real-world variability and uncertainty.Source: §5.1 · p. 8
5.2. Policy Evaluation with Measurement Noises
- 1248.8 average cumulative reward under accumulated measurement noise was 15.3% below noise-free performance yet remained considerably above baseline.The reported evaluation used Policy 1 under RF1.Source: Table 6 · p. 8, §5.2 · p. 8
6. Conclusion
The paper concludes that its LM-based RL agents outperform baseline and state-of-the-art methods, with performance maintained under measurement noise. It presents the framework as a proof of concept for LMs in crop management and calls for further exploration in practical agricultural settings.
6. Conclusion
- LM-based RL agents surpass baseline models and significantly outperform existing state-of-the-art methods, including under measurement noise in observable states.The authors describe this noise robustness as promising for real-world applications.Source: §6 · p. 8
- The authors attribute performance to adapting strategies across reward designs and reasoning about crop and environmental models like expert agronomists.Source: §6 · p. 8
- The paper is intended as a proof of concept to motivate research and practical exploration of advanced language models in agriculture.The authors hope such advances could contribute to smarter, more efficient, and sustainable farming.Source: §6 · pp. 8–9