Source-linked AI summary
Yield forecasting with machine learning and small data: what gains for grains?
Michele Meroni, François Waldner, Lorenzo Seguini, Hervé Kerdiles, Felix Rembold
TL;DR
In data-poor settings, scarce yield labels and difficult model testing complicate determining whether machine learning improves crop-yield forecasts over simpler models. The authors build an automated framework that selects features and models and evaluates monthly forecasts for Algerian barley and wheat using public satellite and climate data. The best machine-learning models always outperformed simple benchmarks in the Algerian forecasts.
Problem
In data-poor settings, scarce yield labels and difficult model testing complicate determining whether machine learning improves crop-yield forecasts over simpler models.
Source: §1 · p. 2Method
The authors build an automated framework that selects features and models and evaluates monthly forecasts for Algerian barley and wheat using public satellite and climate data.
Source: §1 · p. 2Results
The best machine-learning models always outperformed simple benchmarks in the Algerian forecasts.
Source: §6 · p. 15Takeaways & Limitations
Machine-learning superiority over simple benchmarks requires careful calibration and selection, particularly with small data.
Source: §6 · p. 15The 18-year archive yielded <410 data points, which the authors characterize as small data for machine-learning standards.
Source: §5.3 · p. 15
Abstract
from arXiv · showhide
Forecasting crop yields is important for food security, in particular to predict where crop production is likely to drop. Climate records and remotely-sensed data have become instrumental sources of data for crop yield forecasting systems. Similarly, machine learning methods are increasingly used to process big Earth observation data. However, access to data necessary to train such algorithms is often limited in food-insecure countries. Here, we evaluate the performance of machine learning algorithms and small data to forecast yield on a monthly basis between the start and the end of the growing season. To do so, we developed a robust and automated machine-learning pipeline which selects the best features and model for prediction. Taking Algeria as case study, we predicted national yields for barley, soft wheat and durum wheat with an accuracy of 0.16-0.2 t/ha (13-14 % of mean yield) within the season. The best machine-learning models always outperformed simple benchmark models. This was confirmed in low-yielding years, which is particularly relevant for early warning. Nonetheless, the differences in accuracy between machine learning and benchmark models were not always of practical significance. Besides, the benchmark models outperformed up to 60% of the machine learning models that were tested, which stresses the importance of proper model calibration and selection. For crop yield forecasting, like for many application domains, machine learning has delivered significant improvement in predictive power. Nonetheless, superiority over simple benchmarks is often fully achieved after extensive calibration, especially when dealing with small data.
1. Introduction
Crop yield forecasting supports food-security decisions, but data-poor settings make it difficult to establish whether machine learning improves on simpler approaches. The paper develops an operational forecasting framework and tests it on Algeria’s main cereals.
1. Introduction
- Forecasts are needed to anticipate production losses and support food-security policies, market stabilization, and emergency planning.The paper also identifies the selection of explanatory variables and processing gridded data at administrative-unit level as practical barriers.Source: §1 · pp. 1–2
- Operational yield forecasts commonly use empirical regressions linking historical yields to seasonal satellite and climate data, including vegetation indices such as NDVI.Linear regressions may fail to capture complex environmental interactions, motivating tests of machine-learning approaches.Source: §1 · p. 1
- Data-poor settings limit machine learning because yield labels are scarce, while avoiding information leakage can constrain model testing and validation.The authors argue that machine-learning advantages over simpler regression need empirical verification when sample size is critical to reliable prediction.Source: §1 · p. 2
- The framework automates feature and model selection, defines forecast timing from land-surface phenology, and compares machine-learning forecasts with simple benchmarks using public satellite and climate data.It is demonstrated in hindcasts of barley, soft wheat, and durum wheat yields in Algeria from 2002 to 2018.Source: §1 · p. 2
2. Study area and data
The study combines Algerian provincial yield statistics with public satellite and meteorological time series, then aggregates inputs over a phenology-defined growing season. Its data cover three cereal crops and 2002–2018.
2.1. Study area
- The study covers more than 20 Algerian provinces and the country’s main cereals, with yields generally ranging from 0.5 to 2.5 t/ha.The mapped study area identifies crop-area fractions and the provinces covered for each crop, alongside mean yield.Source: §2.1 · p. 2, Figure 1 · p. 3
2.2. Yield data
- Official provincial yield statistics span 2002–2018, providing 408 durum wheat, 391 barley, and 340 soft wheat province-year data points.The statistics are administrative-level averages of grain weight per unit of sown area.Source: §2.2 · p. 3
2.3. Remotely-sensed and climate data
- Province-level inputs are crop-area-averaged NDVI, precipitation, temperature, and global radiation time series from the ASAP early-warning system.Satellite and meteorological pixel values are aggregated to provinces using cropland-area weights.Source: §2.3 · p. 3
2.4. Time domain and temporal aggregation
- A double-logistic fit to NDVI defines the average season from November to June, after which inputs are aggregated monthly and standardized.Monthly features use average, maximum, minimum, or sum operators.Source: §2.4 · p. 4
3. Methods
The methods evaluate monthly yield forecasts with nested leave-one-year-out validation, comparing multiple machine-learning configurations against simple benchmarks. Feature sets, feature selection, and practical significance are also assessed.
3.1. Machine learning workflow
- Nested leave-one-year-out cross-validation holds out each year for testing and uses an inner loop to select hyperparameters without training on the test year.Forecasts use observations only through the month preceding each forecast, with eight forecast months per season.Source: §3 · p. 4, §3.1.1 · p. 5
- The workflow tests satellite, meteorological, and combined feature sets, reduced subsets, mRMR selection, and province-specific one-hot-encoded categorical variables.mRMR retains a tuned percentage of features, while the categorical variables represent province identity.Source: §3.1.2 · pp. 5–6
3.2. Benchmark models
- Five regression algorithms are tested, and two benchmarks are used: a province’s mean yield across other years and a linear peak-NDVI model.The algorithms are LASSO, Random Forest, MLP, linear and radial-basis-function SVR, and gradient boosting.Source: §3.1.2 · p. 6, §3.2 · p. 7
3.3. Statistical analysis
- Performance is evaluated with RMSEp, relative RMSEp, mean error, and R2p at provincial and national levels, including errors in low-yield years.The best configuration is selected using average province-level rRMSEp; Bayesian correlated t-tests assess practical differences using a 5% ROPE and 90% confidence level.Source: §3.3.1 · p. 8, §3.3.2 · p. 8
4. Results
The best machine-learning configurations beat the peak NDVI benchmark across forecast months, but model choice and practical significance vary. Administrative-unit features and feature selection often improve accuracy, while meteorological inputs contribute mainly early in the season.
4.1. Performances by forecasting time
- The best ML models were more accurate than peak NDVI at every forecast month, although benchmark models outperformed many tested ML configurations.Fig. 5 shows the null and peak NDVI benchmarks’ percentile ranks among tested configurations.Source: §4.1 · p. 9, Figure 4 · p. 10, Figure 5 · p. 10
- SVR was selected most often, followed by Lasso and MLP; RF and GBR were never best, and administrative-unit features were always preferred.Remote-sensing features were always selected; meteorological features appeared in a minority of configurations and only alongside remote sensing.Source: §4.1 · p. 9
- For May forecasts, spatial R2p was 0.47, 0.50, and 0.66, while temporal province-level R2p was 0.46, 0.42, and 0.34 for barley, soft wheat, and durum wheat, respectively.Table 5 covers monthly comparisons of the null, peak NDVI, and best ML models from April to July.Source: §4.1 · p. 10, Table 5 · p. 11
- At the national level, ML R2p was 0.81, 0.74, and 0.61 versus peak NDVI’s 0.52, 0.51, and 0.58 for barley, soft wheat, and durum wheat, respectively.In low-yield years, ML forecasts were twice as accurate as peak NDVI for barley and soft wheat, with similar performance for durum wheat.Source: §4.1 · p. 11
- Differences were not always practically significant: peak NDVI was significantly less accurate across the season for barley, but only during part of the season for soft and durum wheat.The null model performed similarly to the best ML method until March, then underperformed; many comparisons among ML algorithms were inconclusive at the 5% ROPE and 90% confidence threshold.Source: §4.1 · p. 11, Figure 6 · p. 12
4.2. Effects of feature sets and feature selection
- Administrative-unit one-hot features reduced median rRMSEp by 13 to 1%, with improvement generally declining at later forecast times.Meteorological features were mainly relevant early in the season; from April onward they did not sensibly reduce error unlike remote-sensing features.Source: §4.2 · p. 12, Figure 7 · p. 13
- Feature selection improved accuracy in 63.5% of cases, with the greatest and most variable benefits for the RS&Met feature set.Benefits increased through the season for Lasso, linear SVR, and MLP, decreased for RF and GBR, and remained constant for RBF SVR.Source: §4.2 · p. 12, Figure 7 · p. 13
5. Discussion
The best-selected machine-learning configurations beat benchmarks across crops and forecast times, but gains and useful features varied by crop, season, and model. The discussion also emphasizes the limits of the available data and the computational cost of tuning.
5.1. ML model forecasting performances
- Best-selected ML configurations outperformed benchmarks at every forecast time and for every crop, with generally smaller performance losses than peak NDVI in low-yield years.The low-yield comparison had an exception for durum wheat in May.Source: §5.1 · pp. 13–14
- 28-34% rRMSEp on province forecasts and 17-21% on national forecasts were observed in April; accuracy gains were minimal after early May.Minimum error came in June for barley and durum wheat and July for soft wheat. Forecast accuracy was comparable to other studies in the region, above Meroni et al.'s 0.35 t/ha for Tunisia, and below Balaghi et al.'s near-harvest province MAE of 0.21 t/ha versus this study's 0.31 t/ha.Source: §5.1 · pp. 13–14
- Durum wheat had lower errors but smaller ML gains than barley and soft wheat; after April, its best configurations used only NDVI, while the other crops often retained meteorological features.The authors suggest durum wheat's more northern cultivation, lower inter-annual variability, and greater NDVI suitability may help explain the difference.Source: §5.1 · p. 13
5.2. Effect of province information, feature sets and feature selection
- Province information improved ML performance, indicating that province differences were not captured by observed NDVI or climate drivers alone.Peak NDVI models implicitly include province information because they are tuned separately for each province.Source: §5.2 · p. 14
- Meteorological-only and NDVI-only models performed similarly early in the season, while NDVI-based performance improved as the season progressed; mRMR feature selection generally helped, with effects varying by algorithm and month.A reduced combined feature set appeared useful from May to July. mRMR helped LASSO, linear SVR, and MLP, while tree-based methods showed the opposite pattern; month-specific selection effects were difficult to interpret.Source: §5.2 · p. 14
- Knowledge-driven feature selection slightly outperformed fully data-driven selection, while combining both approaches achieved the largest improvements.Source: §5.2 · p. 14
5.3. Perspectives
- The workflow’s scope was constrained by four input variables, limited historical records, and a static generic cropland mask shared across crops.Eighteen years yielded fewer than 410 data points, and crop-type maps were harder to produce because past crop-specific training data were unavailable. The Algerian tuning run required 3.3 CPU-years (20 cluster days), though operational forecasting required negligible resources.Source: §5.3 · pp. 14–15
6. Conclusion
The study introduced an automated workflow for forecasting crop yields with small public satellite and climate datasets. In Algeria, the best ML configurations beat simple benchmarks, but no single model or feature set was consistently best, underscoring the need for extensive calibration.
6. Conclusion
- The workflow forecasts barley, soft wheat, and durum wheat yields monthly in Algeria using small, public satellite and climate time series, and applies rigorous testing at provincial and national levels.It also evaluates errors in low-yield years; no single model or feature-set combination consistently delivered the best forecasts, and benchmarks beat a large proportion of tested models.Source: §6 · p. 15