PaperlayerView exact revision on arXiv

Source-linked AI summary

Batched NN 4D-Var: a performant hybrid model framework for the Integrated Forecasting System

Alban Farchi, Marcin Chrust, Massimo Bonavita, Marc Bocquet, Wei Pan, Patrick Laloyaux

arXiv:2610.32582v1stat.MLcs.LG

TL;DR

Offline corrections do not fully account for their interactions with the physical forecast model, motivating online learning. The paper proposes batched NN 4D-Var, which improves out-of-sample generalisation compared with sequential online learning.

  • Problem

    Offline training does not fully account for interactions between neural-network corrections and the physical forecast model, motivating online methods that incorporate them.

    Source: §3.2.2 · p. 9

  • Method

    Batched NN 4D-Var processes assimilation windows in random order and averages parameter increments across mini-batches.

    Source: §1 · p. 2, §5 · p. 18

  • Results

    Batched training removes the generalisation failure observed with sequential online learning and yields hybrid models that progressively improve during training.

    Source: §5 · p. 18

  • Takeaways & Limitations

    The authors present targeted and batched NN 4D-Var as a practical path toward generalisable model-error correction while retaining the physical model's dynamical characteristics.

    Source: §5 · p. 19

    The 48 h experimental setup relies on observations extending beyond forecast initialisation, so it is not directly operationally compatible.

    Source: §3.3.2 · p. 11

Abstract

from arXiv · show

Hybrid forecasting systems combining machine learning and numerical weather prediction offer a promising route toward improved forecast skill while preserving physical consistency. In this study, we investigate online learning of state-dependent model-error corrections within the Integrated Forecasting System using 4D-Var. Building on previous work, we represent model error using a neural network embedded within the forecast model and trained from observations through the data assimilation cycle. First, we introduce a targeted variant of neural-network 4D-Var in which only the neural-network parameters are estimated, while the atmospheric state is provided by a precomputed analysis. We then propose a batched extension that processes assimilation windows in random order and averages parameter increments over multiple windows, drawing inspiration from mini-batch stochastic optimization. Experiments are conducted with the Integrated Forecasting System of the European Centre for Medium-Range Weather Forecasts at an intermediate resolution. Sequential online learning substantially improves forecasts within the training period, with error reductions exceeding 10%in some cases. However, these improvements fail to generalize to unseen data. In contrast, the proposed batched formulation produces hybrid models that consistently improve forecast skill in out-of-sample experiments. Larger batch sizes accelerate convergence and improve robustness by reducing the noise of parameter updates. Diagnostics further show that the online-trained corrections preserve realistic forecast variability and do not improve skill through artificial smoothing. These results demonstrate that the formulation of the online learning problem is critical for generalisation and establish batched online training as a promising framework for hybrid machine-learning/data-assimilation systems.

1 Introduction

Machine-learning weather prediction has achieved strong forecast skill, but deterministic systems can smooth forecasts and may struggle to maintain dynamical balances. The paper motivates hybrid model-error correction and proposes batched NN 4D-Var to improve generalisation.

1 Introduction

  • Deterministic ML weather forecasts can progressively smooth with lead time and may not maintain all relevant dynamical balances, especially in cycled data assimilation.Generative methods address smoothing symptoms, but their integration into existing data-assimilation infrastructure remains an open question.Source: §1 · p. 2
  • Neural-network model-error correction offers a hybrid route that augments physical models, but online NN 4D-Var has shown poor generalisation to unseen data.The approach trains a neural network within weak-constraint 4D-Var to represent model error.Source: §1 · p. 2
  • The proposed batched NN 4D-Var processes random batches of assimilation windows in parallel to improve robustness and generalisation.The paper applies the method to the ECMWF Integrated Forecasting System.Source: §1 · pp. 2–3

2 From offline to batch training

This section develops the hybrid model and training framework, moving from offline analysis-increment training to online parameter estimation and a batched variant. The methodological changes target model-error learning while keeping the physical forecast model in the training loop.

2.1 Hybrid model formulation

  • The hybrid model adds a parameterized correction after each numerical integration step and composes these corrected steps into a forecast.The online training methods require a parameterized forecast model; offline training instead depends on an additive correction.Source: §2.1 · p. 3

2.2 Offline training from analysis increments

  • Offline training uses analysis increments as proxies for forecast errors, rescales them to per-step corrections assuming linear error growth, and can misrepresent instantaneous model error when growth is nonlinear.It uses precomputed analyses and forecasts without explicitly requiring the physical forecast model.Source: §2.2 · p. 4

2.3 Online training with NN 4D-Var

  • NN 4D-Var jointly estimates the initial state and neural-network parameters by assimilating observations through a parameterized weak-constraint 4D-Var framework.The state analysis advances to the next window with the hybrid model, while parameter values are carried forward unchanged.Source: §2.3 · p. 4

2.4 Tailoring NN 4D-Var to focus on model parameters

  • Targeted NN 4D-Var freezes a precomputed initial-state analysis and estimates model parameters only, avoiding state adjustments that could compensate for model error.Joint estimation can alias state errors into parameter increments when parameter background covariances are poorly calibrated.Source: §2.4 · p. 5

2.5 Batching NN 4D-Var to enhance generalisation

  • Because fixed precomputed analyses make windows independent, batched NN 4D-Var processes randomly selected windows and averages their parameter increments, reducing update noise.Processing multiple windows in parallel also exposes the model to Ne times more observational information for a fixed number of optimisation steps.Source: §2.5 · p. 6, Figure 2 · p. 7

2.6 Nomenclature: offline versus online learning

  • The paper calls the targeted and batched methods online under the machine-learning convention because their parameter updates involve the physical forecast model, despite using precomputed analyses and random window order.Under the geoscientific convention, those same features are associated with offline learning.Source: §2.6 · pp. 6–7

3.1 Technical experimental details

The experiments use ECMWF’s IFS with a neural-network model-error correction and temporally separated training and evaluation data. Online corrections are masked in the tropics, and forecast launches are spaced to limit computational cost.

3.1 Technical experimental details

  • Experiments use OOPS with IFS cycle 49r1 and incremental 4D-Var with three outer loops; the nonlinear model runs at TCo399 with inner loops at TL95, TL159, and TL255.Source: §3.1 · p. 7
  • The correction is a column-based neural network for temperature, log surface pressure, vorticity, and divergence, using state, date, time, latitude, and longitude as predictors.It has four tanh hidden layers and 1,214,876 trainable parameters, and is masked between latitudes −30° and 30° in online experiments.Source: §3.1 · pp. 7–8
  • Offline training uses 2017–2022 data, online training uses 2024 observations, and 2025 forecasts are evaluated every 48 hours across 183 initialisations.The 48-hour spacing is a pragmatic choice to reduce computational cost.Source: §3.1 · p. 8

3.2 Offline training experiments

Offline training uses analysis increments to initialize a neural-network correction embedded in the IFS. On unseen 2025 data, it improves forecast accuracy modestly, with gains that fade at longer lead times.

3.2.1 Training the hybrid model

  • The offline neural network is trained on analysis states and increments from a 2017–2022 strong-constraint 4D-Var reanalysis, using 2017–2021 for training and 2021–2022 for validation.Inputs and targets are truncated to T15 and interpolated onto a 16 × 32 Gaussian grid; training uses prediction mode and MSE loss.Source: §3.2.1 · p. 8

3.2.2 Forecasts from unseen data

  • The neural-network output is rescaled into an additive per-time-step bias correction and updated every three hours, increasing forecast computational cost by less than 3%.This update frequency balances forecast accuracy and computational cost.Source: §3.2.2 · p. 8
  • For unseen-data evaluation, the hybrid model is tested over 2025, outside the offline and online training periods, against control forecasts initialized from operational analyses.Negative normalized RMSE differences indicate improvement over control.Source: §3.2.2 · pp. 8–9
  • Error reductions reach approximately 4% for both T850, which is directly corrected, and Z500, which is not explicitly corrected.The improvement peaks after one to two days and gradually vanishes at longer lead times; the authors attribute cross-variable gains to interactions with model dynamics.Source: §3.2.2 · p. 9
  • The modest forecast gains contrast with offline evaluation, where the same network explains about 25% of temperature analysis-increment variance by relative MSE.The authors conclude that offline training alone does not fully account for interactions between the correction and the physical forecast model.Source: §3.2.2 · p. 9

3.3 Sequential online training

Sequential online training updates neural-network parameters through successive assimilation windows, with forecast skill assessed both during training and on unseen data. Although the 48 h targeted configuration improves forecasts of the day, sequentially trained models deteriorate on out-of-sample forecasts.

3.3.1 Training experiments

  • The three sequential experiments compare joint estimation of state and parameters, targeted parameter-only estimation with a 12 h window, and targeted estimation with a 48 h window.The targeted formulation uses a precomputed operational analysis as the initial state; experiments run sequentially for ten months over 2024.Source: §3.3.1 · p. 10

3.3.2 Forecasts of the day

  • The original and targeted 12 h experiments have mixed forecast-of-the-day effects and are generally less accurate than the offline-trained hybrid.They sometimes outperform the reduced-resolution standard experiment, while the standard experiment is systematically less accurate than control forecasts initialized from operational analyses.Source: §3.3.2 · pp. 10–11
  • The targeted NN 4D-Var configuration with a 48 h window is the only tested setup that visibly improves on the offline-trained hybrid, with Z500 error reductions exceeding 10% up to day 3.The improvement is consistent across variables and regions.Source: §3.3.2 · p. 11
  • Forecast-of-the-day results use a changing sequence of updated models, and the 48 h setup uses observations extending beyond the forecast initial time, limiting its operational compatibility.The study therefore evaluates generalization on unseen data in the following section.Source: §3.3.2 · p. 11

3.3.3 Forecasts from unseen data

  • Across all three sequential experiments, hybrid-model forecasts deteriorate over training time on unseen 2025 data, including the 48 h targeted configuration that had improved forecasts of the day.The deterioration is reported across lead times throughout the 10-day forecast.Source: §3.3.3 · p. 12, Figure 5 · p. 12

3.4 Non-sequential online training

The non-sequential experiments process assimilation windows in random order, and the batched model improves out-of-sample forecast skill. Larger batches yield faster, more robust convergence, while diagnostics show that the improvement is not due to artificial smoothing.

3.4.1 Training experiments

  • The experiments use 183 non-overlapping 48 h windows and random, without-replacement sampling; the two batch sizes are Ne = 1 and Ne = 8.Each configuration runs 183 optimization steps; this corresponds to one epoch at Ne = 1 and eight epochs at Ne = 8.Source: §3.4.1 · p. 13

3.4.2 Forecasts from unseen data

  • With Ne = 8, forecast skill improves overall as training progresses, while Ne = 1 shows a noisier signal, particularly for temperature.Averaging increments across eight windows reduces update noise, and processing more windows per step cycles through the training data faster.Source: §3.4.2 · p. 13
  • After an initial 50–100-day transition, parameter-increment norms stabilize and the decline in forecast RMSE slows.The transition is interpreted as adaptation from predicting analysis increments to representing instantaneous model error; the authors suggest reducing σp later or using a longer training dataset.Source: §3.4.2 · p. 13
  • The hybrid and control forecasts have visually indistinguishable 500 hPa wind kinetic-energy distributions, indicating that improvements are not achieved through artificial smoothing.This diagnostic uses forecasts after 244 days of Ne = 8 training.Source: §3.4.2 · pp. 13, 15, Figure 7 · p. 14
  • The scorecard shows broad improvement across variables and levels, especially in the stratosphere, while the largest degradations remain concentrated in the tropics.It compares RMSE differences against operational analysis and independent observations over forecast days 1–10.Source: Figure 8 · p. 15, §3.4.2 · p. 15

4 Discussion

The discussion connects targeted and batched NN 4D-Var to DA/ML coordinate descent and, under a small-covariance approximation, to stochastic gradient descent. It identifies possible optimizer improvements while cautioning that replacing the variational solve with a single gradient evaluation may be unstable.

4.1 Coordinate descent interpretation

  • Targeted and batched NN 4D-Var estimate parameters separately using a precomputed analysis, linking the methods to the DA/ML iterative framework.The broader coordinate-descent approach separates trajectory estimation by data assimilation from parameter estimation by machine learning; joint optimization becomes computationally intractable for sufficiently long datasets.Source: §4.1 · p. 16

4.2 Simplifying the targeted NN 4D-Var method

  • When parameter-error variance is small relative to observation-error variances, targeted NN 4D-Var is formally equivalent to stochastic gradient descent, with σp^2 acting as its learning rate.This is a first-order approximation, and the 4D-Var case adds temporal evolution and nonlinearities.Source: §4.2 · p. 17
  • Replacing the variational minimization with one parameter-gradient evaluation could reduce computational cost, but repeated corrections may amplify gradients and produce unstable or exploding updates.The authors say further investigation is needed to establish stability and the conditions for viability.Source: §4.2 · p. 17

4.3 Similarities with stochastic gradient descent

  • The framework could incorporate momentum to mitigate sampling-related oscillations and adaptive optimizers such as Adam to potentially improve convergence speed and robustness.Momentum would add memory to parameter updates; the paper presents these as prospective strategies, not tested results.Source: §4.3 · pp. 17–18
  • Because σp^2 acts as an effective learning rate, reducing σp later in training could provide additional control over convergence and potentially improve generalization.The discussion proposes a learning-rate scheduler as a future improvement informed by the slower RMSE decline in the later training stages.Source: §4.2 · p. 17, §4.3 · p. 18

5 Conclusions

The study examines neural-network model-error corrections trained online within an NWP model. Batched NN 4D-Var generalizes in out-of-sample forecasts, while tropical use remains a practical boundary.

5 Conclusions

  • Offline-trained corrections yield consistent but modest out-of-sample improvements that peak at short lead times and then diminish.The learned correction does not account for interactions with the physical model.Source: §5 · p. 18
  • Targeted NN 4D-Var uses precomputed analyses and re-assimilates observations to estimate parameters only, reducing aliasing risk; sequential training still failed to generalize.This also alleviates the need to calibrate state-parameter background-error balances.Source: §5 · p. 18
  • Batched training removes sequential generalization failure and yields progressively improving out-of-sample forecasts; hybrid forecasts were dynamically indistinguishable from control.The method processes windows in random order and averages parameter increments across mini-batches.Source: §5 · p. 18
  • Potential extensions include adding momentum, adaptive update rules, and learning-rate scheduling; direct first-guess-gradient updates could reduce computational cost but require stability investigation.The proposed simplification would replace the inner variational minimisation.Source: §5 · p. 18
  • The correction had to be disabled in the tropics after degrading skill; proposed remedies include humidity inputs, more frequent updates, and horizontally extended corrections.Horizontal extensions were not technically available in the current implementation.Source: §5 · pp. 18–19
  • The authors expect model upgrades to require periodic fine-tuning rather than retraining from scratch, keeping maintenance costs limited.Previously trained corrections could be reused as a starting point as model error evolves gradually with upgrades.Source: §5 · p. 19
Loading 2610.32582v1…