Source-linked AI summary
Using machine learning to correct model error in data assimilation and forecast applications
Alban Farchi, Patrick Laloyaux, Massimo Bonavita, Marc Bocquet
TL;DR
Sparse and noisy observations make standard machine-learning reconstruction of system dynamics difficult. The paper combines data assimilation with deep learning to learn corrections to an existing knowledge-based model, producing hybrid surrogates that improve forecasts and data-assimilation analyses in a quasi-geostrophic model.
Problem
Standard machine-learning methods cannot rigorously handle sparse and noisy observations when learning dynamical models, motivating their combination with data assimilation.
Method
The method alternates strong-constraint 4D-Var data assimilation with deep-learning steps that learn a correction to an existing knowledge-based model.
Results
Hybrid surrogate models learn a substantial part of the model error, improve short- to mid-range forecasts, and yield significantly better analyses than the original model.
Takeaways & Limitations
Learning model error on top of an existing model is numerically feasible and improves both forecasting and data assimilation in the tested quasi-geostrophic setting.
Takeaways & Limitations
The correction method is offline, and the corrected data-assimilation formulation implicitly assumes that model error grows linearly in time.
Abstract
from arXiv · showhide
The idea of using machine learning (ML) methods to reconstruct the dynamics of a system is the topic of recent studies in the geosciences, in which the key output is a surrogate model meant to emulate the dynamical model. In order to treat sparse and noisy observations in a rigorous way, ML can be combined to data assimilation (DA). This yields a class of iterative methods in which, at each iteration a DA step assimilates the observations, and alternates with a ML step to learn the underlying dynamics of the DA analysis. In this article, we propose to use this method to correct the error of an existent, knowledge-based model. In practice, the resulting surrogate model is an hybrid model between the original (knowledge-based) model and the ML model. We demonstrate numerically the feasibility of the method using a two-layer, two-dimensional quasi-geostrophic channel model. Model error is introduced by the means of perturbed parameters. The DA step is performed using the strong-constraint 4D-Var algorithm, while the ML step is performed using deep learning tools. The ML models are able to learn a substantial part of the model error and the resulting hybrid surrogate models produce better short- to mid-range forecasts. Furthermore, using the hybrid surrogate models for DA yields a significantly better analysis than using the original model.
1 Introduction
The paper addresses model error in numerical weather prediction, where sparse and noisy observations limit standard machine-learning approaches. It investigates combining data assimilation with machine learning to learn corrections to an existing knowledge-based model.
- The study targets errors in numerical models despite reasonably well-known physical laws, including errors from misrepresented or unresolved processes.
- Standard machine-learning methods generally require accurate system states and cannot directly handle sparse observations, motivating their combination with data assimilation.
- Alternating data-assimilation and machine-learning steps estimates both the system state and surrogate-model dynamics with increasing accuracy.
- The proposed approach corrects an existing knowledge-based model rather than constructing a surrogate model from scratch, reducing concerns about initialization and computational cost.
- Experiments use a two-layer, two-dimensional quasi-geostrophic channel model with 1600 variables, enabling controlled tests of the proposed method.
2 Methodological aspects
The method alternates data assimilation and machine learning to estimate states and model parameters, then adds a learned correction to an existing model. It supports either resolvent correction for simpler implementation or tendency correction for matching the original prediction horizon.
- 2.1 A Bayesian framework for machine learning and data assimilation: The data-assimilation objective accommodates sparse and noisy observations by jointly considering the initial state, trajectory, model, observation, and background-error terms.
- 2.2 Optimisation strategy: The optimization alternates data-assimilation estimation of the trajectory with machine-learning estimation of surrogate-model parameters.This coordinate-descent strategy treats the trajectory and parameters as different optimization variables.
- 2.3.1 Correcting the resolvent: The hybrid model adds a machine-learning correction to the original model resolvent, with the correction output remaining in the system state space.The additive formulation can represent arbitrary model error, while multiplicative or compositional alternatives may suit other error structures.
- 2.3.1 Correcting the resolvent: Resolvent correction requires only minor modifications to existing numerical methods but predicts only at a multiple of the integration interval.
- 2.3.2 Correcting the tendencies: Tendency correction integrates the original and machine-learning tendencies, potentially matching the original model horizon but requiring substantial changes to variational and gradient calculations.
3 The quasi-geostrophic model
The study uses a two-layer, two-dimensional quasi-geostrophic channel model whose reference dynamics are perturbed through parameter changes to create model error. The model exhibits wave-like, chaotic behaviour, while the perturbed setup degrades forecast skill relative to the reference dynamics.
- 3.1 Brief model description: The experiments use a two-layer, two-dimensional quasi-geostrophic channel model with 40 × 20 grid points and a Gaussian-hill orography.The stream function is the control variable for forecasts and data assimilation.
- 3.2 Dynamical behaviour: The reference model evolves as a westward-moving wave with a mean period of about 16 d and model variability of 4.95.Its climatology is characterised by time-averaged and time-standard-deviation patterns related to the orography.
- 3.2 Dynamical behaviour: Changing parameters or initial conditions can produce different doubling times and climatologies, suggesting several quasi-stationary regimes around the model attractor.The numerical experiments nevertheless checked that generated trajectories yielded the same reference climatology.
- 3.3 The perturbed QG model: The perturbed setup changes both layer depths, doubles the integration time step, and moves the Gaussian orographic hill to the domain centre.These parameter perturbations represent errors associated with physical parametrisation, spatial resolution, or time integration.
- 3.3 The perturbed QG model: Figure 5 evaluates the perturbed model’s forecast skill against lead time using 100 initial conditions and compares it with model variability.Figure 6 visualises forecast error from the perturbed setup after 1 h, 1 d, and 2 d integrations.
4 The original data assimilation step
The original data-assimilation step uses strong-constraint 4D-Var to assimilate sparse, noisy observations with the perturbed model. Across 18 trajectories, the cycled analyses run without numerical failure and achieve a time-averaged analysis RMSE of 0.24.
- 4.1 The observation database: The observation database samples the stream function every 2 h at 50 random locations with Gaussian observation noise R = 0.1I.The setup therefore represents partial observations with noise rather than a fully observed system.
- 4.2 The data assimilation method: Each 1 d assimilation window solves a strong-constraint 4D-Var problem using the original model resolvent, observation covariance R, and background covariance B.The unknown is the state at the start of each window.
- 4.2 The data assimilation method: The background covariance is parameterised as B = b2C, with horizontal correlation length, vertical correlation, and standard deviation tuned to minimise time-averaged analysis RMSE.The optimal parameterisation is reported in Table 2.
- 4.3 Data assimilation results: The cycled strong-constraint 4D-Var analyses achieve a time-averaged RMSE of 0.24, about 5% of model variability, across 18 trajectories.Each trajectory completes 1032 consecutive cycles without numerical failure, after dropping 8 spin-up cycles.
5 The machine learning step
The machine-learning step learns model-error corrections from analysis trajectories, using neural networks combined with the original model to improve short- and mid-range forecasts. Experiments examine database construction, architecture, training quality, and the effects of sampling period and trajectory length.
- 5.1 Database preparation: The hybrid surrogate combines the original model resolvent with a neural-network correction trained from effective analysis databases that approximate the unknown true model-error database.The effective database is accurate when analyses approximate the true state and analysis increments approximate model error.
- 5.2 Machine learning models architecture: Convolutional and dense neural networks are used for 1600-dimensional states, with convolutional layers providing local operations motivated by spatially decreasing correlations.The D-model and CD-model templates use two-dimensional convolutional layers, padding for periodic boundaries, and reshape layers around dense components.
- 5.3 Training example: 17.43% normalised test MSE is obtained on 16 test trajectories for the analysis-increment prediction task, but 68.50% when evaluating the same network against the true model-error database.The network reproduces model-error patterns but has incorrect scaling, showing that analysis increments only approximate model error.
- 5.4 Improving the quality of the training database: Longer model integrations capture more developed model error, while the effective database excludes analysis increments and uses trajectories initialized from analyses.The sampling period τ is the integration length used to capture model error over several days.
- 5.5 Comparison between the neural networks: ReLU networks outperform linear networks, while convolutional and dense models perform equivalently overall, making convolutional models attractive for high-dimensional systems.More nodes or filters generally improve forecasts, whereas deeper networks do not improve them in this weakly nonlinear regression problem.
- 5.6 Training with dense, noiseless observations: With dense, noiseless observations, forecast skill improves with longer training trajectories and larger sampling periods, and the gap between ideal and effective databases narrows as τ increases.Training the full dynamics without the original model requires larger databases and ultimately produces less accurate forecasts than correcting the existing model.
6 The corrected data assimilation step
The corrected DA step replaces the original model resolvent with a hybrid surrogate model in strong-constraint 4D-Var, improving analyses while revealing sensitivity to sampling-period assumptions.
- 6.1 Data assimilation setup: The corrected DA uses the same observations and 4D-Var cost function, but substitutes the hybrid model resolvent Mo + Mml for the original resolvent Mo.The implementation uses the forcing formulation of weak-constraint 4D-Var.
- 6.1 Data assimilation setup: The forcing formulation assumes model error grows linearly in time, making the sampling period a key source of analysis error as growth becomes nonlinear.The correction is scaled by δt/τ under this assumption.
- 6.2 Data assimilation results: About 25% lower analysis RMSE is achieved in the best hybrid-surrogate cases than with the original model.The database size has little influence, whereas larger sampling periods worsen analysis because model-error growth becomes nonlinear.
- 6.2 Data assimilation results: Online weak-constraint 4D-Var obtains an analysis RMSE of about 0.07 in the same DA problem, leaving room to improve the offline correction.The proposed method is evaluated as an offline model-error estimation approach.
- 6.3 Sensitivity to observation density: The method succeeds across observation densities, retaining reductions of the same order even with 10 observations every 2 h.The authors attribute this robustness to the 4D-Var algorithm.
7 Conclusions
The study combines DA and deep learning to correct a knowledge-based model with sparse, noisy observations, producing a hybrid surrogate that improves forecasts and analyses in a quasi-geostrophic test model.
- 7 Conclusions: The proposed iterative method alternates DA state estimation with ML dynamics learning, correcting an existing model rather than constructing full dynamics from scratch.This avoids cold-start divergence and is expected to be an easier learning problem.
- 7 Conclusions: A small neural network learns a significant part of model-error variance, and the hybrid surrogate improves forecasts over the original model up to 16 days.Nonlinear networks outperform linear ones, while CNN layers can replace DNN layers without substantially changing performance.
- 7 Conclusions: The experiments cover a two-layer, two-dimensional quasi-geostrophic model with perturbed parameters representing model error.The original DA uses strong-constraint 4D-Var, and analysis increments train the ML models.
- 7 Conclusions: The offline method remains limited by the experimental setting: 600 observations per DA window may suffice for state estimation but may not suffice to train larger ML models online.The authors identify online correction as an important direction for future work.