Source-linked AI summary
Extreme Gradient Boosting for Yield Estimation compared with Deep Learning Approaches
Florian Huber, Artem Yushchenko, Benedikt Stratmann, Volker Steinhage
TL;DR
Pre-harvest yield prediction needs broad temporal and spatial data, while Deep Learning approaches can be data-hungry and difficult to explain. The paper converts remote-sensing imagery into feature-based inputs for XGBoost, which performs competitively with Deep Learning on United States soybean data and supports feature-importance analysis. The authors caution that the evidence is limited to one dataset and that Deep Learning may improve with more training data.
Problem
Yield prediction requires extended monitoring of phenological and climatic conditions, while Deep Learning methods can require plentiful data and remain difficult to explain.
Method
The study processes remote-sensing images into compact feature-based representations for XGBoost yield prediction and analyzes feature importances with Shapley Values.
Results
XGBoost performs on par with or outperforms state-of-the-art Deep Learning yield predictors on the USDA soybean dataset across multiple metrics and test cases.
Takeaways & Limitations
The results support XGBoost as a potentially interpretable approach for remote-sensing yield prediction, with important features aligned with expert knowledge.
Takeaways & Limitations
The experiments use a single United States soybean dataset, and Deep Learning performance may improve with more training data.
Abstract
from arXiv · showhide
Accurate prediction of crop yield before harvest is of great importance for crop logistics, market planning, and food distribution around the world. Yield prediction requires monitoring of phenological and climatic characteristics over extended time periods to model the complex relations involved in crop development. Remote sensing satellite images provided by various satellites circumnavigating the world are a cheap and reliable way to obtain data for yield prediction. The field of yield prediction is currently dominated by Deep Learning approaches. While the accuracies reached with those approaches are promising, the needed amounts of data and the ``black-box'' nature can restrict the application of Deep Learning methods. The limitations can be overcome by proposing a pipeline to process remote sensing images into feature-based representations that allow the employment of Extreme Gradient Boosting (XGBoost) for yield prediction. A comparative evaluation of soybean yield prediction within the United States shows promising prediction accuracies compared to state-of-the-art yield prediction systems based on Deep Learning. Feature importances expose the near-infrared spectrum of light as an important feature within our models. The reported results hint at the capabilities of XGBoost for yield prediction and encourage future experiments with XGBoost for yield prediction on other crops in regions all around the world.
1. Introduction
Yield prediction benefits logistics, food distribution, and producers, but current Deep Learning approaches can require plentiful data and offer limited explanation. This study introduces an interpretable XGBoost pipeline for remote-sensing yield prediction and reports competitive soybean results against Deep Learning.
- Motivation: Remote sensing provides inexpensive, widely available information about plant phenology and climatic conditions needed for yield prediction.Multispectral imagery captures visible and non-visible information throughout the year.
- Background: Deep Learning dominates remote-sensing yield prediction but works especially well when data are plentiful and extensive explanations are unnecessary.Its black-box nature limits interpretability.
- Study aim: XGBoost uses a compact feature-based representation of remote-sensing images to predict soybean yields and is compared with two state-of-the-art Deep Learning approaches.The evaluation uses annual soybean yields from the United States.
- Contributions: The comparative experiments report that XGBoost outperforms two Deep Learning approaches by an average of at least 25% in RMSE.This result is stated for soybean yield data in the United States.
- Contributions: Shapley Value feature importances are used to improve trust by showing that important model features align with expert knowledge of yield prediction.The paper identifies explainability as a contribution alongside predictive effectiveness.
2. Material
The study evaluates soybean yield prediction across 13 adjacent United States states using county yields and multi-source remote-sensing, weather, land-cover, and boundary data. Inputs span 2003–2021 and are spatially aligned to soybean farmland for pipeline processing.
- Study area and period: The evaluation covers United States soybean production from 2003 to 2021 across 13 adjacent states.The selected states contain most United States soybean farmland.
- USDA yield data: County-level USDA soybean yields provide the ground-truth labels for model training and validation.Yields are reported in bushels per acre, with 1 bu/ac approximately equivalent to 67.26 kg/ha.
- Input data: MODIS reflectance and temperature products provide 500 m, eight-day composites, while Daymet supplies gridded precipitation and vapor-pressure data.The reflectance product includes seven visible and infrared bands, and temperature data include day and night bands.
- Spatial processing: The Cropland Data Layer is upscaled from 30 m to 500 m and masks remote-sensing images to soybean farmland.TIGER state boundaries are used to crop the data to county borders.
3. Methods
The method converts irregular remote-sensing imagery into feature-based representations for XGBoost, whose tree ensemble is optimized iteratively with regularization and approximate split finding. The pipeline represents each band’s pixel distribution using quantiles while relying on permutation invariance, and tunes hyperparameters through sequential model-based optimization.
- 3. Methods: The pipeline transforms cropped and masked satellite images into histogram- or feature-based inputs suited to Deep Learning or XGBoost, respectively.The input imagery has variable county dimensions and contains 34 temporal episodes across 11 bands.
- 3. Methods: XGBoost builds an ensemble by iteratively adding regression trees that fit current residuals while optimizing a loss-plus-regularization objective.Regularization penalizes tree complexity through leaf counts and leaf weights to support generalization.
- 3. Methods: The approximate split-finding algorithm selects promising feature-value splits from feature distributions instead of evaluating every possible split, improving time efficiency for the experiments.The implementation stores sorted feature columns in blocks that can support parallel computation.
- 3. Methods: Hyperparameters are tuned with Tree-structured Parzen Estimation, which sequentially models performance from historical measurements.The approach is used as a sequential model-based optimization method for selecting hyperparameters.
- 3. Methods: Each satellite-band distribution is represented by its median and 20% and 80% quantiles, approximating mostly skewed normal distributions for XGBoost.These statistics summarize central tendency and the distribution’s lower and upper slopes, reducing dimensionality.
- 3. Methods: The dimensionality reduction assumes yield depends more on non-cropped pixel values than their locations, potentially omitting position-bound features such as soil properties or elevation.This is explicitly identified as an assumption underlying the feature-based representation.
4. Experiments and Discussion
Across end-of-year and in-year soybean yield experiments, XGBoost matched or exceeded Deep Learning approaches, while feature engineering compressed the remote-sensing data and enabled interpretable analysis. Results also indicate that performance depends on data availability, processing choices, and the transferability of the single U.S. soybean setting.
- 4.1. End-of-the-Year Prediction: 25% lower RMSE and 0.13 higher average R2 were obtained by XGBoost than Deep Learning for end-of-year prediction across five testing years.The end-of-year experiments used the complete span from day 49 to day 321.
- 4.4. Explaining Predictions with Feature Importances: Feature importance was highest in the later crop-development periods, with the second MODIS reflectance band most significant during final development and harvest.The results associate information closer to harvest with better end-of-year accuracy than in-year accuracy, where later-stage information is missing.
- 4.5. Discussion: XGBoost was at least on par with Deep Learning overall, although CNN improved with expanded training data and outperformed XGBoost in R2 only in 2021.The authors connect this pattern to Deep Learning’s benefit from extensive training data.
- 4.5. Discussion: The study’s conclusions are constrained by evaluation on one U.S. soybean dataset, omitted non-remote-sensing inputs, and untested transferability across crops and regions.The authors note that data quality and prediction accuracy may differ elsewhere, especially for non-yearly crops.
5. Conclusions
The study presents a feature-extraction pipeline that enables XGBoost yield prediction from remote sensing data. Its conclusions indicate performance on par with state-of-the-art Deep Learning approaches, with interpretable feature importance and potential broader agricultural applications.
- The proposed pipeline efficiently converts remote sensing images into feature vectors for Machine Learning.
- The approach was evaluated on USDA soybean data and is expected to generalize to other crops, prediction tasks, and regions.
- Decision-tree base learners make feature importance analysis natively available for examining conditions in different application areas.
- XGBoost is capable of yield prediction on remote sensing data on par with state-of-the-art Deep Learning approaches.
- Shapley Value feature importances can raise trust by showing that important prediction features align with expert knowledge.