PaperlayerView exact revision on arXiv

Source-linked AI summary

Crop Yield Prediction for Punjab, Pakistan: A Tree-Ensemble and Leaf-Health Prototype, and What Random Validation Hides

Amina Asif, Qurat Ul ain Asif, Noor Bakhat Asif, Kasa-e-Fatima

arXiv:2610.07059v2cs.CV

TL;DR

Small agricultural tables can make yield forecasts appear accurate under careless splits. This paper audits a Punjab prediction prototype and evaluates it with temporal validation and external data checks. Per-crop trends outperform models not given the year, while the leaf classifier performs well on its image benchmarks but cannot establish a yield link.

  • Problem

    Small agricultural tables can produce misleading evaluations under careless splits, despite yield forecasts' relevance to farm-input and grain-import decisions.

    Source: §1 · pp. 1–2

  • Method

    The paper audits a Kaggle-derived yield table, evaluates models with year-based validation, checks findings against FAOSTAT and district wheat data, and builds a leaf-health prototype.

    Source: §1 · p. 2, §3.4 · p. 6, §4.6 · p. 9

  • Results

    Per-crop trends beat models without the year (leave-one-year-out RMSE 0.24 Ton/Ha on the Kaggle-derived table and 0.29 Ton/Ha on FAOSTAT); leaf recall was 96.4% on 336 unseen diseased leaves.

    Source: §10 · pp. 22–23

  • Takeaways & Limitations

    Yield-model comparisons should include a trend baseline and hold out crop-years or whole years to avoid rewarding duplicated observations.

    Source: §7 · p. 21

    Weather effects remain unresolved because rainfall is constant, soil values are crop-level averages and temperature is a national mean of unclear origin.

    Source: §6 · p. 20

Abstract

from arXiv · show

Yield forecasts help planners and farmers decide on inputs, storage and imports, but small agricultural tables can make reported accuracy fail on a new season. We built a crop-yield prototype for Punjab, Pakistan that combines Random Forest, XGBoost, support vector regression and a Ridge-stacked ensemble with a MobileNetV2 leaf-health classifier, and deployed it as a web application with SHAP explanations. A random 80/20 split of a merged Kaggle-derived table (414 rows) gives the ensemble an R2 of 0.991. An audit showed that the table contains only 46 independent observations: a join with nine temperature records per year copied every crop-year nine times. Holding out whole years takes XGBoost on the same rows from R2 = 0.994 to -0.20. On deduplicated data, and on a longer FAOSTAT table (1990-2024, 70 observations), a per-crop linear trend (leave-one-year-out RMSE 0.29 Ton/Ha) beats every model not given the year (0.84-1.06). Neither pesticide use nor national temperature change explains the trend residuals. An apparent pesticide gain in the Kaggle table disappears on FAOSTAT, where the pesticide series is mostly imputed and the two releases disagree. An independent district-level wheat panel (36 districts, 13 seasons) shows that most variation in Punjab is spatial and that a district mean with a common trend matches the learned models. The leaf classifier reaches 99.87% accuracy on held-out PlantVillage images and recalls 96.4% of 336 unseen diseased leaves after near-duplicates were removed. However, no leaf image is paired with a yield record, so the health score used in the yield models had to be constructed and adds nothing. We report these negative findings together with the prototype.

1 Introduction

The paper presents a Punjab yield-prediction prototype while revisiting its initially strong scores after finding join-induced replication in a small public table. It frames careful validation and transparent limits as important contributions.

1 Introduction

  • The 414-row table contains only 46 independent observations because a join copied each crop-year nine times, allowing duplicates into both sides of a random split.The authors report that evaluation by whole years changed the results and that a one-line linear trend was their strongest model.Source: §1 · p. 2
  • The audit found the national pesticide series to be an exact straight line from 2001 to 2013 and includes a reproducible deduplication step.Source: §1 · p. 2
  • The study compares five regressors under random, leave-one-year-out and forward-chaining validation, with bootstrap confidence intervals over years and tests of weather and input variables.Source: §1 · p. 2
  • An independent Punjab wheat panel covers 36 districts and 13 seasons to assess spatial variation and the potential contribution of season-wide shocks.Source: §1 · p. 2
  • A rebuilt FAOSTAT table extends the national replication to 1990–2024 and 70 observations, where the pesticide series that appeared helpful is largely imputed.Source: §1 · p. 2
  • The MobileNetV2 leaf-health classifier reached 99.87% accuracy on held-out PlantVillage images and 96.4% recall on 336 unseen diseased leaves; the prototype includes a web application with SHAP explanations.Source: §1 · p. 2
  • No paired leaf and yield data existed, so the constructed image-derived feature added nothing; the authors list data needed to test the idea properly.Source: §1 · p. 2

2 Related Work

Prior work uses climate and soil inputs with classical and deep learners for yield prediction, while image-disease research and validation risks motivate the paper's methods and checks.

2 Related Work

  • Yield models commonly use temperature, rainfall and soil type, and temperature effects can be strongly nonlinear.Source: §2 · p. 2
  • The study combines Random Forest, XGBoost [8] and support vector regression through stacked generalisation with a Ridge meta-learner.Source: §2 · p. 3
  • PlantVillage disease classifiers use deep networks [27], but background cues [14] and training-image variety can affect robustness; MobileNetV2 [32] was chosen for lightweight hosting.Source: §2 · p. 3
  • Standard cross-validation can mislead with temporal, spatial or hierarchical dependence; forecasting needs care, and simple benchmarks are often hard to beat.Source: §2 · p. 3
  • The paper adds a concrete example of unintended join-created leakage in a public table, a recognized source of over-optimistic results.Source: §2 · p. 3

3 Data

The paper combines national crop-yield tables with district wheat data and maize leaf images, auditing how the tables were assembled and extending the national series with FAOSTAT data. The section also describes a deduplication step for honest evaluation and the scope of each dataset.

3 Data

  • Four public datasets support national yield modelling, FAOSTAT and district-panel replications, unseen-image classifier testing, and exploratory wheat plots.The first two datasets feed the national yield models; the FAOSTAT table and district panel are used for replications.Source: Table 1 · p. 3, §3 · p. 3

3.1 Tabular data and the merged table

  • The merged Pakistan table has 414 rows for 1990–2013, with 2003 absent, and joins two crop types across yield, pesticide, soil, and climate data.Yield was converted from hg/ha to Ton/Ha, district-level soil and climate values were averaged by crop, and NPK_total was derived as N + P + K.Source: §3.1 · pp. 3–4
  • Crop-level soil variables and constant rainfall act only as crop indicators, while national pesticide and temperature values are shared across the two crops.From 2001 to 2013, the Kaggle-derived pesticide series falls by exactly 989.2 tonnes per year, then remains at 0.92 tonnes; the current FAOSTAT series differs and is mostly imputed.Source: §3.1 · p. 4

3.2 The nine-fold replication

  • Each crop-year is repeated nine times because the temperature file has nine readings per year, leaving 46 independent yield observations rather than 414.The within-year temperature standard deviation is 2.5 °C, five times the 0.5 °C spread of yearly means; the repeated rows were collapsed by crop-year using mean temperature for honest evaluation.Source: §3.2 · p. 4, Figure 1 · p. 5

3.3 District-level wheat panel

  • The independent wheat panel contains 464 observations from 36 Punjab districts across 13 seasons, with no weather variables.Province-level yields correlate at r = 0.99 with the national FAO wheat yields for the six overlapping harvest years.Source: §3.3 · p. 5

3.4 FAOSTAT table, 1990–2024

  • The rebuilt FAOSTAT table covers 1990–2024 with 70 crop-year observations, and its overlapping yields match the Kaggle-derived table.The current FAOSTAT pesticide series is mostly imputed and does not resemble the Kaggle-derived series; the table also includes national temperature change.Source: §3.4 · pp. 5–6, Figure 2 · p. 6

3.5 Leaf images

  • The PlantVillage image module merges three maize disease classes into one diseased label, making it a binary classifier for maize leaves only.The image set contains 2,690 diseased and 1,162 healthy images; wheat leaf classes are not included.Source: §3.5 · p. 6

4 Methodology

The methodology builds a crop-yield prototype from joined and audited agricultural tables, with separate feature sets, regression models, leaf-image classification, and validation protocols. It also tests the approach on district and FAOSTAT data and deploys an interface with model explanations.

4.1 Preprocessing pipeline

  • The pipeline uses the 414-row joined table for deployed models, the 46-row audited table for honest evaluation, and FAOSTAT and district panels for replication.The pipeline description also notes that the 12-year Punjab wheat statistics were used only for exploratory plots.Source: §4.1 · p. 6, Figure 3 · p. 7

4.2 Feature sets

  • Feature set A contains nine tabular inputs, B adds the classifier health score, and C adds year as a separate, forecast-known trend feature.An earlier feature set dropped constant rainfall, a temperature–rainfall product, crop code, and year; year was later isolated in C.Source: §4.2 · p. 7

4.3 Regression models

  • The regressors include crop-mean and crop-trend baselines, Ridge, Random Forest, XGBoost, SVR, and a Ridge stack trained on five-fold out-of-fold predictions.Tree settings were fixed rather than tuned, while SVR and Ridge used standardized inputs.Source: §4.3 · pp. 7–8

4.4 Leaf-health classifier and 4.6 Evaluation of the classifier on unseen images

  • A frozen MobileNetV2 backbone feeds a trained dense head; unseen-image testing removes near-duplicates and evaluates recall on the remaining diseased leaves.The head uses global pooling, 128 ReLU units, dropout, and a sigmoid output; the unseen set has 336 images, all diseased, so healthy-leaf false alarms could not be measured.Source: §4.4 · p. 8, §4.6 · p. 9

4.5 Integrating the leaf score into the yield models

  • Because yield records lack paired leaf images, training health_score is constructed from NPK_total and takes values 0.0007 for wheat and 0.9921 for maize.NPK_total is constant within each crop, so these values encode the crop-specific nutrient totals rather than observed leaf health.Source: §4.5 · pp. 8–9

4.7 Evaluation protocols

  • Evaluation uses repeated random five-fold splits, leave-one-year-out folds, and forward chaining, grouping both crops by year and bootstrapping year-level RMSE intervals.Leaf-classifier accuracy is reported with a Wilson interval.Source: §4.7 · pp. 9–10

4.8 District-panel evaluation and 4.9 Replication on the FAOSTAT table

  • District tests compare district means, a common trend, learned predictors, and an oracle season shock; FAOSTAT repeats trend/model comparisons with A′ and C′ features.The oracle uses held-out labels and is not a practical predictor; the FAOSTAT tests use leave-one-year-out and forward chaining.Source: §4.8 · p. 10, §4.9 · p. 10

4.10 Explainability and deployment

  • The deployed Gradio interface accepts climate and soil sliders and optional maize-leaf images, returns four model predictions, and shows a SHAP explanation for XGBoost.Without an uploaded image, the health-score input is 0.5.Source: §4.10 · p. 10, Figure 6 · p. 11

5 Results

The apparent high accuracy is largely a consequence of replicated crop-year rows, while year-aware evaluation favors a simple trend over models without year. Residual models recover some time-structured deviations in the Kaggle table, but the apparent pesticide signal is not a supported agronomic effect.

5.1 Results as originally obtained

  • 0.9906 R2 and 0.046 Ton/Ha mean absolute error made the Ridge-stacked ensemble the random-split leader on the 414-row table.The table caption flags these figures as inflated by replication.Source: Table 2 · p. 11, §5.1 · p. 11

5.2 The same rows under a different split

  • 0.994 to −0.204: XGBoost’s R2 fell when evaluation changed from random 5-fold to leave-one-year-out on the same 414 rows.Random Forest fell from 0.965 to −0.063; SVR changed less because it was already poor.Source: §5.2 · p. 11, Figure 7 · p. 12, Table 3 · p. 12

5.3 Honest evaluation on the crop-year table

  • 0.243 Ton/Ha LOYO RMSE for the per-crop trend beats 0.59–0.74 for feature-set-A models; forward-chaining RMSE is 0.317 versus 0.70–0.96.The trend intervals do not overlap those of the feature-set-A models.Source: Table 4 · p. 12, §5.3 · p. 13
  • 0.141 LOYO RMSE for year-aware Random Forest beats the trend’s 0.243, but forward chaining ties the trend at 0.317.The authors interpret the LOYO result as interpolation; trees cannot extrapolate the trend.Source: §5.3 · p. 13, Figure 8 · p. 13
  • 0.54 rather than 0.994: XGBoost’s random 5-fold R2 fell on the deduplicated crop-year table, isolating replication as the main source of inflation.The protocol and feature set were unchanged.Source: §5.3 · p. 13
  • 0.737 LOYO RMSE: the ensemble was worse than RF, XGBoost and SVR individually under the honest protocols.The authors suggest inner cross-validation may be unstable with 45 or fewer training rows, but did not test that explanation.Source: §5.3 · p. 13

5.4 Modelling deviations from the trend

  • 0.159 LOYO and 0.260 forward-chaining RMSE: the forest residual model improves on the trend’s 0.243 and 0.317, respectively.The model adds predicted residuals to a per-crop trend fitted on training years.Source: Table 5 · p. 14, §5.4 · p. 14
  • 0.085 Ton/Ha LOYO gain came from pesticide use in the Kaggle table, but a similar gain from calendar year and no temperature gain indicate timing information, not a pesticide effect.The pesticide series is an exact line in time for 13 years, and the gain disappears on FAOSTAT.Source: Table 6 · p. 14, §5.4 · pp. 14–15

5.5 Replication on the current FAOSTAT release

The longer FAOSTAT table and independent district panel support simple trend or district-mean baselines over models without year. The leaf score adds virtually nothing to yield predictions, while the classifier performs well on its image tasks subject to validation and source limitations.

5.5 Replication on the current FAOSTAT release

  • 0.292 Ton/Ha LOYO RMSE for the per-crop trend beats 0.84–1.06 for every FAOSTAT model without year.Under forward chaining, the trend scores 0.339 versus 0.93–1.15 for those models.Source: Table 7 · p. 15, §5.5 · p. 15
  • 0.282–0.296 Ton/Ha: FAOSTAT pesticide-and-temperature residual models do not improve on the trend’s 0.292 LOYO RMSE; only year helps.The authors attribute the earlier Kaggle pesticide gain to an interpolated series; FAOSTAT pesticide use is nearly constant, and the two releases disagree.Source: §5.5 · pp. 15–16, Table 8 · p. 16

5.6 Independent check on district-level wheat

  • 85.4% of district-panel yield variance is explained by district identity, versus 6.7% by season alone; the district mean with a common trend scores 0.231 LOYO RMSE.That matches Ridge at 0.234 and beats XGBoost at 0.258 and Random Forest at 0.340.Source: Figure 9 · p. 16, §5.6 · pp. 16–17, Table 9 · p. 17
  • Weather could not be tested directly because the file has no weather variables; its potential gain is capped at roughly a quarter of district-mean error, with a smaller practical gain.Source: §5.6 · p. 17

5.7 Effect of the leaf score

  • 0.001 Ton/Ha or less: adding the leaf score changes year-aware RMSE by at most this amount, and XGBoost and the ensemble predictions are unchanged to three decimals.Permutation importance also indicates that shuffling health_score costs nothing.Source: Table 10 · p. 17, §5.7 · p. 17, Figure 10 · p. 18
  • 0.043 Ton/Ha: the largest deployed-model prediction shift when the leaf score moves across its full range; XGBoost shifts by nothing.The ensemble shifts by 0.004 Ton/Ha.Source: Table 11 · p. 18, §5.7 · p. 18

5.8 Leaf-health classifier

  • 99.87% accuracy: the classifier made one error on 772 held-out PlantVillage images, but the same validation set selected the stopping epoch and reported accuracy.The authors treat this as an optimistic estimate for clean maize images; no Bahawalpur field photographs were tested.Source: §5.8 · pp. 18–19, Table 12 · p. 19
  • 96.4% recall: the classifier recognized 324 of 336 unseen diseased leaves after near-duplicates were removed.The unseen set contains no healthy leaves, so false-alarm rates are unknown; it also includes no Bahawalpur images.Source: Table 13 · p. 19, §5.8 · p. 19

6 Findings

The findings are bounded by the available data: they do not establish that weather or leaf images cannot help yield prediction, and they do not support interpreting pesticide importance as an agronomic effect.

6 Findings

  • The tables cannot establish that weather has no yield effect because rainfall is constant, soil values are crop averages, and temperature is a national mean of unclear origin.The authors describe a null weather result as expected under these measurements.Source: §6 · p. 20
  • The results show only that learned models without year did not beat a trend in these data, not that such models cannot do so generally.Source: §6 · p. 20
  • The study cannot test whether leaf images help predict yield because it lacks paired leaf images and yield records, and pesticide importance is not evidence of an agronomic effect.False-alarm rates on healthy field leaves also remain unknown because the unseen images are all diseased.Source: §6 · p. 20

7 Discussion

The discussion recommends checking the unit of observation, including a trend baseline, and limiting claims about image features to what paired data can test.

7 Discussion

  • A many-to-one join created nine copies per crop-year, so counting distinct targets and splitting by crop-year or year would have caught the leakage.The resulting rectangular table looked clean and had no missing values.Source: §7 · pp. 20–21
  • A per-crop trend beats the nine-feature models here, and the apparent pesticide gain disappears on the longer table where that series is imputed.The independent district panel also found a district mean with a common trend matched or beat learned models.Source: §7 · p. 21
  • Without leaf images paired to yields from the same fields and seasons, the image feature is only a stand-in and adds no new information.The present interface is better described as a leaf-health tool alongside a yield estimator, not as a combined model.Source: §7 · p. 21

8 Hyperparameters and Reproducibility

Model settings were fixed across validation protocols, and the scripts used to produce the tables and figures are available in the repository.

8 Hyperparameters and Reproducibility

  • All random seeds were fixed at 42, hyperparameters were chosen once without tuning, and scripts for each table and figure are in the repository.Source: §8 · p. 21
  • Table 15 lists every setting used for the models.The same settings were used under every validation protocol.Source: Table 15 · p. 21, §8 · p. 21

9 Limitations

The study’s limited sample size and aggregated data constrain model comparisons and their applicability to regional or decision-making contexts.

9 Limitations

  • 46 independent Kaggle-derived observations and 70 FAOSTAT observations leave wide confidence intervals, making learned-model rankings unreliable.Source: §9 · p. 22
  • National or crop-level yields, pesticides, and temperature, alongside crop-level soil and humidity means, cannot represent district variation.Source: §9 · p. 22
  • Annual national temperature change cannot describe provincial growing-season conditions, so its null result is weak evidence about weather generally.Source: §9 · p. 22
  • The Kaggle table ends in 2013; the 70-observation FAOSTAT table is national, while the wheat-only district panel ends in 2019–20 and lacks weather variables.Generalization to unseen districts was not tested.Source: §9 · p. 22
  • The Kaggle pesticide series is an exact straight line, while FAOSTAT’s is mostly imputed; neither measures annual use reliably.The nine yearly temperature readings are undocumented, and their averaged summary may not be appropriate.Source: §9 · p. 22
  • The constructed health_score adds nothing to yield prediction, and the classifier’s evaluation leaves healthy-field false alarms unknown.The unseen-image test included diseased leaves only, and no Bahawalpur photographs were tested; the PlantVillage validation set also selected the stopping epoch.Source: §9 · p. 22
  • Hyperparameters were not tuned, only one seed was reported for the classifier and deployed models, and the system is not validated for real decisions.Tuning an SVR on 46 rows would itself require nested validation.Source: §9 · p. 22

10 Conclusion and Future Work

The prototype’s initially high yield scores reflected repeated crop-year rows rather than real predictive skill; year-based evaluation favored simple trends without extra inputs. The paper also reports a leaf-classifier benchmark and outlines data collection and validation priorities.

10 Conclusion and Future Work

  • 0.24 Ton/Ha leave-one-year-out on the Kaggle-derived table and 0.29 Ton/Ha on the longer FAOSTAT table were the per-crop linear trend RMSEs; models without year did not beat the trend.Models given the year matched or beat it.Source: §10 · p. 22
  • The prototype combined tree ensembles, SVR, stacking, and a MobileNetV2 leaf-health classifier, but its high yield scores came from nine-fold replication in the merged table.The reported random-split ensemble score was R2 = 0.991.Source: §10 · p. 22
  • The leaf classifier achieved 99.87% accuracy and recalled 96.4% of 336 unseen diseased leaves, but could not inform yield models without paired images and yield records.Source: §10 · p. 23
  • Future work should extend district-season panels with crops and local weather or vegetation data, and test yield value using leaves matched to yields.Source: §10 · p. 23
  • The authors recommend headline leave-one-year-out and forward-chaining results alongside a trend baseline, plus testing the classifier on healthy and diseased Bahawalpur leaves and on wheat.Source: §10 · p. 23

Data and Code Availability

The authors provide code, notebooks, trained models, result tables, and supporting data products through a repository and demo.

Data and Code Availability

  • The repository includes the deduplicated crop-year table, extended FAOSTAT table, district-panel cleaning code, and a notebook to rebuild FAOSTAT and run the unseen-image test.The district yield file is not redistributed because its portal lists no licence; users should check Kaggle source-data licences before redistribution.Source: Data and Code Availability · p. 23

Related papers

Loading 2610.07059v2…