An Input-Frugal Deep Learning Framework for Weather-Driven National Crop-Yield Forecasting: A Case Study of Brazilian Soybean

1 Introduction

• Traditional statistical and process-based approaches can require long consistent series or detailed local soil, management, and cultivar inputs that are difficult to scale.These constraints create a trade-off between physical realism and the practical availability of consistent inputs.Source: §1 · p. 2
• Three operational gaps are complex inputs, confounded architecture comparisons, and approaches not designed to incorporate operational weather forecasts for in-season updating.The paper frames the need as reproducibility, fair encoder comparisons, and forecast-ready deployment.Source: §1 · p. 3
• The proposed framework uses globally accessible meteorological data and a simplified agro-environmental label to reduce reliance on complex local information.The label proxies hard-to-observe heterogeneity such as soil type, management, and cultivar choices.Source: §1 · p. 3
• Spatial instance expansion links multiple grid-node weather sequences within each municipality to the same observed municipal yield, increasing training samples and diversity.The paper presents this strategy as reducing overfitting susceptibility while retaining a municipality-scale yield reference.Source: §1 · p. 3
• The study benchmarks temporal encoders under identical inputs and fusion, evaluates earlier in-season predictions, and uses SHAP diagnostics to examine the roles of year, weather, and context.The framework is presented as adaptable to other crops and geographies with minimal bespoke data collection.Source: §1 · pp. 4–5

2.1.1 Problem formulation

• The task is supervised regression from subnational administrative units, with model predictions assessed primarily after harvested-area-weighted aggregation to national yield.The reporting scale is intended to align with operational monitoring and market impacts.Source: §2.1.1 · p. 5
• Two static covariates complement the weather sequence: an agro-environmental label for persistent location differences and a crop-year identifier for long-run yield change.The year input lets models learn trends jointly with interannual weather–yield variation rather than detrending and restoring the trend separately.Source: §2.1.1 · p. 5

2.1.2 Temporal alignment and aggregation

• A user-defined crop calendar sets the within-season window and time-step granularity, which must remain consistent across seasons and locations.Daily, weekly, monthly, or other resolutions are possible; finer-resolution data can be aggregated as a configurable preprocessing step.Source: §2.1.2 · p. 6
• Earlier in-season forecasts retain only the first k time steps, enabling lead-time comparisons under the same temporal alignment and modelling pipeline.The truncated-input variants preserve the fixed modelling setup while varying how much of the season is observed.Source: §2.1.2 · p. 6

2.1.3 Spatial instance expansion (SIE)

• Spatial instance expansion turns each unit–season into node–season instances with locally varying weather but the same observed unit-level yield and static context.This increases effective sample size and introduces within-unit variability from gridded fields without synthetic yield labels.Source: §2.1.3 · p. 6
• A context-wise cap balances expanded training instances across context classes by subsampling to the smallest class within each training fold.The cap is applied only to training data and leaves the held-out test year unchanged.Source: §2.1.3 · p. 7
• SIE keeps all instances from a unit–season in the same LOYO split, uses only observed yield targets, and aggregates predictions back to unit–year yield.Its target is invariant across grid nodes, and context-wise capping limits domination by classes with denser grid coverage.Source: §2.1.3 · p. 7

2.1.4 Input representation

• Each instance combines a T × P within-season weather matrix, context label, crop-year identifier, and observed unit-level yield target.The mapping predicts unit yield from node-level weather and unit/season context; T may reflect daily, weekly, or monthly sequences with a consistent calendar.Source: §2.1.4 · pp. 7–8

2.2.1 Common fusion scheme and evaluated architectures

• All architectures share weather, agro-environmental, and crop-year inputs with a common fusion scheme; only the weather-sequence encoder changes.The static embeddings are concatenated with a fixed-length weather embedding and passed to the regression head.Source: §2.2.1 · p. 8, Figure 1 · p. 9
• The MLP baseline flattens weather inputs and omits explicit time-step order, so sequence encoders can be compared for their use of temporal structure under the same inputs.All models use the same input representation, fusion strategy, and training protocol.Source: §2.2.1 · p. 9, §2.2.3 · p. 11

2.2.2 Transformer encoder (reference implementation)

• The Transformer serves as the reference model because self-attention models long-range dependencies and can scale to longer weather sequences.It anchors the detailed description, ablations, in-season variants, and explainability analyses.Source: §2.2.2 · p. 9
• The Transformer adds learned positional offsets to weather vectors, processes them with one self-attention and feed-forward block, then max-pools over time for fusion with static embeddings.The encoded sequence retains T time steps, and global max pooling produces a fixed-length weather representation.Source: Figure 2 · p. 10, §2.2.2 · pp. 10–11
• Transformer hyperparameters are selected during initial development and then held fixed across all cross-validation folds.Full configuration details are provided in the Supplementary Material for replication and adaptation.Source: §2.2.2 · p. 11

2.2.3 Other encoders within the framework

• The CNN captures local temporal patterns, the LSTM encodes recurrent dependencies, and the CNN–LSTM combines convolutional and recurrent sequence modelling.Mamba uses input-gated state-space updates with linear-time sequence complexity, making it attractive for higher-frequency inputs.Source: §2.2.3 · pp. 11–12

2.3.1 Study domain and unit of analysis

• The study spans 10 major soybean-producing states and 1,360 municipalities, representing more than 90% of national production in the reference season.The case study is designed to test yield–weather relationships across diverse Brazilian agro-meteorological conditions.Source: §2.3.1 · p. 12, §2.3 · p. 12

2.3.2 Data sources

• Municipal soybean yields and planted and harvested areas come from PAM/IBGE, with harvested areas later used to weight broader-scale predictions.Source: Soybean yield and area (PAM/IBGE) · p. 13
• Weather inputs use 10 agronomically relevant AgERA5 variables, monthly aggregated over September–April and aligned to each soybean growing cycle.The selected variables include temperature, precipitation, humidity, and radiation; snow-related indicators are excluded as negligible in the study regions.Source: Meteorological predictors (AgERA5) · pp. 13–14
• Municipalities receive one of 17 official edaphoclimatic-region labels, a compact categorical context for relatively stable agro-environmental differences.The label is used to condition weather–yield relationships without assembling detailed soil, management, or cultivar datasets.Source: Agro-environmental context label… · pp. 14–15

2.3.3 Case-study input construction

• Core experiments use six monthly steps from September–February, pairing each municipality–season yield and context with weather sequences from grid nodes inside its boundary.Each node-level sequence is paired with the same municipal yield target and context covariates for that municipality–season.Source: Temporal alignment and sequence… · p. 15, Node-level representation and spatial… · p. 15

2.4 Training, validation, and hyperparameter tuning

• Leave-one-year-out cross-validation holds out each of the 20 crop seasons in turn, fitting transformations only on non-held-out years to prevent test-year leakage.Source: §2.4 · pp. 15–16
• Within each fold, an 80/20 municipality–year group-aware split of training data provides validation data only for early stopping and model selection.The held-out year is not used for training or early-stopping decisions.Source: §2.4 · p. 16
• Architecture-specific hyperparameters are selected on a separate development split, then frozen for LOYO evaluation rather than retuned on test years.The Transformer search uses a priori ranges and selects a final configuration by validation performance and inspection of training and validation loss curves.Source: §2.4 · pp. 16–17, Table 2 · p. 17

2.5 Prediction aggregation and baselines

• Node-level predictions are averaged within each municipality, then municipal yields are aggregated to state and national levels using harvested-area weights.Aggregation is deterministic and identical across architectures.Source: §2.5 · pp. 17–18
• Headline results use harvested-area-weighted national yields, while state aggregates provide intermediate spatial diagnostics.Municipality-level errors are used mainly as internal diagnostics.Source: §2.5 · p. 18
• The deep-learning encoders are benchmarked against both an operational heuristic and a linear statistical model.Source: §2.5 · p. 18
• The farmer baseline predicts national yield from the previous five observed national yields, while ridge regression uses weather, edaphoclimatic labels, and crop year.Ridge predictions use the same harvested-area-weighted aggregation scheme as the deep-learning models.Source: §2.5 · pp. 18–19

2.6 Evaluation metrics and statistical analysis, and contextual benchmarking

• Held-out predictions are aggregated to municipalities and then state and national levels; national RMSE, MAE, bias, rRMSE, and R^2 summarize performance.R^2 quantifies the fraction of interannual variance in national yield explained by predictions.Source: §2.6 · pp. 19–20
• National encoder rankings are also compared using a two-sided Wilcoxon signed-rank test on year-paired absolute errors across 20 LOYO test years.Source: §2.6 · p. 20
• State-scale errors are evaluated to assess whether national performance masks systematic state-level misfit.The contextual benchmark is a targeted set of Brazil-focused soybean studies intended to provide performance context, not a systematic literature review.Source: §2.6 · pp. 20–21
• RMSE and rRMSE confidence intervals use 10,000 year-level BCa bootstrap resamples and remain conditional on the chosen modelling procedure.A moving-block bootstrap sensitivity analysis gave conclusions consistent with the year-wise intervals.Source: §2.6 · p. 21

2.7 Ablation study design

• The ablations test the edaphoclimatic label and SIE independently, keeping the Transformer architecture and other inputs or training settings fixed.The label is zeroed for its ablation; SIE is disabled by using one deterministic grid node per municipality–season.Source: §2.7 · pp. 21–22
• SIE is assessed with 25%, 50%, 75%, and 100% of available training years, while the LOYO test year and other settings remain fixed.End-of-season ablations also use the full available training years in each LOYO fold.Source: §2.7 · p. 22
• Edaphoclimatic-label effects are tested on paired municipality–year absolute errors, while SIE is assessed on national-yield errors, including a one-sided paired Wilcoxon test across LOYO years.The label comparison uses harvested-area-weighted least squares, year-clustered standard errors, and a two-sided test at α = 0.05.Source: §2.7 · pp. 22–23

2.8 In-season forecasting setup

• Six issuance variants use weather from the first one through six months of the season, corresponding to September through September–February.The Transformer architecture, optimizer, and hyperparameters match the end-of-season model; only input length changes.Source: §2.8 · p. 23
• Each issuance model is retrained from scratch and evaluated on out-of-sample years using the same leave-one-year-out protocol as end-of-season forecasts.Static covariates remain unchanged across issuance variants.Source: §2.8 · p. 23
• Municipal predictions are averaged across spatial samples, then aggregated nationally using planted-area weights because harvested area and production are not yet observed at issuance.Performance is summarized by national RMSE and Skill relative to the five-year rolling-mean farmer baseline, which is available pre-season and fixed within the year.Source: §2.8 · p. 23

2.9 Model explainability via SHAP

• SHAP decomposes each Transformer prediction into a baseline plus feature contributions; positive and negative values indicate increases or decreases relative to that baseline.The analysis uses the model-agnostic Kernel SHAP estimator.Source: §2.9 · p. 24
• The feature vector keeps weather variables separated by month and includes the edaphoclimatic label and crop-year identifier as explicit features.The reference Transformer is trained on the first 18 seasons and tested on the last two, allowing multiple crop-year values in the test set.Source: §2.9 · p. 24
• SHAP backgrounds use all training instances without yields, and a 700-representative k-means background produced nearly indistinguishable global patterns and rankings.The full training-input distribution is used for the reported analysis.Source: §2.9 · pp. 24–25
• SHAP is diagnostic, not causal, and covers a stratified sample from two held-out seasons rather than all 20 LOYO models.A fully LOYO-aligned analysis was outside scope because it would require an order of magnitude more computation, on the order of hundreds of GPU-hours.Source: §2.9 · p. 25
• Global importance is mean absolute SHAP magnitude, aggregated across months for each weather variable and compared with crop year and edaphoclimatic context.Weather-feature importance is also aggregated to an overall weather measure for comparison with the static inputs.Source: §2.9 · p. 25

3.1 End-of-season national performance

• 149 kg ha⁻¹ RMSE, 5.3% rRMSE, and R² = 0.784 made the Transformer best nationally, with 47.6% lower RMSE than the farmer baseline and 33% lower than ridge.The Transformer and LSTM were nearly tied (149 versus 150 kg ha⁻¹ RMSE); a year-level test found no detectable systematic advantage (p = 0.985).Source: §3.1 · pp. 25–27, Table 3 · p. 26
• All deep-learning variants outperformed ridge, and sequence encoders outperformed MLP; the authors attribute gains to nonlinear interactions and temporal structure in monthly weather.The Transformer was selected for downstream analyses because it trained about three times faster in the authors’ implementation.Source: §3.1 · pp. 26–27
• Across municipalities, Transformer predictions had median Pearson correlation r = 0.629 (IQR 0.465–0.734), with 96.7% positive; state-level RMSE ranged 145–535 kg ha⁻¹, showing heterogeneous residual errors.State trajectories broadly tracked interannual fluctuations without systematic temporal drift, while state-specific departures remained.Source: §3.1 · p. 27, Figure 5 · p. 30, Figure 7 · p. 32

3.2 Contributions of edaphoclimatic label and spatial instance expansion

• Removing the edaphoclimatic label raised national RMSE from 149 to 162 kg ha⁻¹ (+8.7%) and municipality-level absolute error by 20.5 kg ha⁻¹ (p = 0.0158).The label represents persistent soil–climate context and improved accuracy at both national and municipal scales.Source: §3.2 · pp. 33–34, Table 5 · p. 34
• Disabling SIE raised national RMSE from 149 to 165 kg ha⁻¹ (+10.7%); reduced-data tests favored SIE directionally at 50% training data, but not significantly.The authors support SIE as a robustness device because it improves full-data accuracy and shows no evidence of harm under sub-sampling.Source: Table 6 · p. 34, §3.2 · pp. 34–35, Table 7 · p. 35

3.3 In-season performance and lead time

• National RMSE fell from 251 kg ha⁻¹ in September to 146 kg ha⁻¹ in February, while Skillfarm rose from 12.6% to 49.0%; forecasts beat the baseline at every cutoff.The largest month-on-month gain occurred from December to January: RMSE fell from 220 to 173 kg ha⁻¹ and Skillfarm rose from 23.3% to 39.6%.Source: §3.3 · pp. 35–36, Figure 9 · p. 36

3.4 Positioning relative to previous yield-forecasting studies

• The simpler weather-plus-edaphoclimatic inputs reached the same performance neighborhood as richer Brazil pipelines, but cross-study comparisons are contextual rather than strictly like-for-like.The cited studies differ in inputs, spatial unit, or validation strategy, and most lack a replicable farmer-baseline comparison.Source: §3.4 · pp. 36–37, Table 8 · p. 38

3.5 SHAP-based interpretation of the reference Transformer

• Crop year accounted for roughly three-quarters of grouped SHAP importance (760 kg ha⁻¹), versus 197 kg ha⁻¹ for weather and 94 kg ha⁻¹ for edaphoclimatic region.The attributions associate crop year with long-run variation and weather and regional context with shorter-run deviations; they are model-based associations, not causal effects.Source: §3.5 · pp. 38–39, Figure 10 · p. 39
• Higher crop-year values receive stronger positive SHAP contributions, capturing long-run yield gains, while weather and regional context adjust predictions around that trajectory.The model incorporates crop year directly rather than separating trend handling into detrending and post-hoc retrending steps.Source: §3.5 · p. 40, Figure 11 · p. 40, §3.6 · p. 44
• Cloud fraction is the most influential weather predictor; moisture and thermal-demand variables cluster, while solar flux and wind add partly independent information and solid precipitation has negligible importance.Because meteorological predictors are coupled, the SHAP clusters represent overlapping process bundles rather than proof of causal or independent effects.Source: §3.5 · pp. 40–41, Figure 12 · p. 42
• The edaphoclimatic label adjusts predictions around the national baseline, and Region 101 shows the widest variation in its contribution across municipality–year observations.These SHAP adjustments need not follow regions’ observed mean-yield rankings; Region 101’s wide spread is consistent with interannual and spatial contrasts and nonlinear interactions.Source: §3.5 · p. 43, Figure 13 · p. 43

3.6 Strengths, limitations, and implications for transferability

• The framework’s low-burden inputs support transfer to new crops or regions, but local yield and meteorology data would typically be needed for retraining or fine-tuning.Its weather-centric design can ingest operational ensemble forecasts, and the paper reports end-of-season national performance on par with recent Brazil benchmarks using data-rich pipelines.Source: §3.6 · p. 44
• Weather-only inputs miss non-weather shocks and canopy or management signals, while static regional context cannot anticipate structural changes in production regimes.The paper notes that accuracy may require monitoring and periodic retraining or recalibration; region- and year-specific sowing expectations could improve phenological alignment while preserving operational feasibility.Source: §3.6 · pp. 44–45
• Frugal inputs, national-scale accuracy, and in-season skill could support timely, spatially resolved yield information for food-security and stable-market goals, especially where satellite or survey systems are delayed or inconsistent.The interpretable contributions from year, weather, and region can help practitioners and policymakers diagnose model behavior when applying the framework to new settings.Source: §3.6 · p. 45

• The framework uses routine weather as its only time-varying signal plus a simple agro-environmental label, and adopts the top-ranked Transformer as its reference implementation.The conclusion identifies transfer to other crops and regions as a goal.Source: §4 · pp. 45–46
• The Transformer achieved national RMSE 149 kg ha^-1 (rRMSE ≈ 5.3%) and reduced error 47.6% versus a five-year moving-average baseline.In-season accuracy improved as weather information accumulated, reaching ~49–50% lower error by February.Source: §4 · p. 46
• Sequence encoders outperformed non-sequential baselines, with the Transformer and LSTM in the national top tier; context labels and spatial instance expansion provided additive benefits.Sub-national diagnostics showed residual heterogeneity but no evidence of systematic temporal drift.Source: §4 · p. 46
• SHAP diagnostics associate most of the long-run yield trajectory with crop year, while weather and edaphoclimatic context refine interannual departures.Moisture/cloud and thermal-demand variables dominate weather contributions; the trend-aware approach requires updating when production regimes shift.Source: §4 · pp. 46–47
• Routine weather inputs support operational-forecast integration, while limitations include non-weather shocks, simplified planting calendars, and the need for periodic recalibration as regimes evolve.Future work will apply the template to other crops and geographies and develop probabilistic, forecast-driven outputs.Source: §4 · p. 47