Machine Learning for Stochastic Parameterization: Generative Adversarial Networks in the Lorenz '96 Model

David John Gagne; Hannah M. Christensen; Aneesh C. Subramanian; Adam H. Monahan

Source-linked AI summary

Machine Learning for Stochastic Parameterization: Generative Adversarial Networks in the Lorenz '96 Model

David John Gagne, Hannah M. Christensen, Aneesh C. Subramanian, Adam H. Monahan

arXiv:1909.04711v1Sep 10, 2019physics.ao-phcs.LGnlin.CDstat.ML

TL;DR

Unresolved sub-grid processes create model uncertainty, while existing data-driven stochastic parameterizations impose structural assumptions. This paper develops and tests a conditional GAN stochastic parameterization in Lorenz ’96, finding that some configurations outperform a bespoke benchmark and reproduce system statistics across weather and climate timescales, despite important training and deployment limitations.

Problem
Existing data-driven stochastic parameterizations require structural assumptions about conditional distributions, motivating a more flexible approach to representing unresolved-process uncertainty.
Method
The study develops a conditional GAN stochastic parameterization for Lorenz ’96 and evaluates different input-noise configurations in weather and climate model runs.
Results
Some GAN configurations outperform the bespoke polynomial benchmark, reproduce spatio-temporal correlations and regimes, and show that weather- and climate-skill relationships depend on the evaluated metric.
Takeaways & Limitations
GANs can serve as explicit stochastic parameterizations of uncertain sub-grid processes directly from data, with autocorrelated noise important for skillful weather forecasts but less important for climatological distributions.
Takeaways & Limitations
GAN training and offline evaluation do not reliably predict forecast-model performance, and sequential application to correlated fields may introduce nonlinear feedback effects.

Abstract

from arXiv · show

Stochastic parameterizations account for uncertainty in the representation of unresolved sub-grid processes by sampling from the distribution of possible sub-grid forcings. Some existing stochastic parameterizations utilize data-driven approaches to characterize uncertainty, but these approaches require significant structural assumptions that can limit their scalability. Machine learning models, including neural networks, are able to represent a wide range of distributions and build optimized mappings between a large number of inputs and sub-grid forcings. Recent research on machine learning parameterizations has focused only on deterministic parameterizations. In this study, we develop a stochastic parameterization using the generative adversarial network (GAN) machine learning framework. The GAN stochastic parameterization is trained and evaluated on output from the Lorenz '96 model, which is a common baseline model for evaluating both parameterization and data assimilation techniques. We evaluate different ways of characterizing the input noise for the model and perform model runs with the GAN parameterization at weather and climate timescales. Some of the GAN configurations perform better than a baseline bespoke parameterization at both timescales, and the networks closely reproduce the spatio-temporal correlations and regimes of the Lorenz '96 system. We also find that in general those models which produce skillful forecasts are also associated with the best climate simulations.

1 Introduction

The paper motivates stochastic parameterization as a way to represent unresolved-process uncertainty and evaluates GANs as a data-driven alternative to structurally constrained approaches. Lorenz ’96 provides a proof-of-concept testbed for assessing GAN parameterization across weather and climate timescales.

Unresolved sub-grid processes contribute substantially to weather and climate model uncertainty, requiring representation of both model and initial-condition uncertainty in reliable ensembles.
Stochastic parameterization represents the distribution of possible sub-grid tendencies conditioned on the resolved state, allowing ensemble members to experience different realizations.
Existing data-driven stochastic approaches require structural assumptions about the physical process or conditional distribution, limiting tractability with conventional methods.
Generative models can sample from distributions of arbitrary complexity, while GANs use competing generator and discriminator networks to learn representative samples.
The study evaluates whether GANs can learn stochastic sub-grid parameterizations and uncertainty quantification using the Lorenz ’96 system at weather and climate timescales.

2 Methods

The methods construct a conditional GAN parameterization for Lorenz ’96 sub-grid tendencies and compare it with a bespoke stochastic polynomial benchmark. GAN variants differ in predictors, noise magnitude, correlation, and output-layer noise.

2.1 Lorenz ’96 Model Configuration: Lorenz ’96 represents an extratropical-atmosphere toy model with coupled large-scale X and small-scale Y variables whose interaction supports parameterization testing.
2.1 Lorenz ’96 Model Configuration: The forecast model truncates the small Y scales and uses a parameterized tendency to approximate their impact on the resolved X variables.
2.2 GAN Parameterizations: A conditional dense GAN predicts the current sub-grid tendency from the previous state, with variants using X and U or X alone as predictors.
2.2 GAN Parameterizations: The generator combines previous-state inputs with a latent random vector, while the discriminator distinguishes training-data tendencies from generated tendencies.
2.2 GAN Parameterizations: GAN configurations vary in predictor set, noise magnitude, noise correlation, and optional output-layer noise, with results summarized in Table 1.
2.3 Polynomial Regression Parameterization: The benchmark fits a cubic polynomial to truth-run (X, U) data and adds temporally correlated noise because X alone does not uniquely determine U.

3 Results

The GAN parameterizations are evaluated offline and in weather- and climate-scale simulations using distributional, forecast, correlation, regime, and spectral diagnostics. Several configurations match or outperform the bespoke polynomial parameterization, with strong correspondence between weather and climate skill.

Metrics: Offline Hellinger-distance evaluations generally favor GANs using both X and U inputs over GANs using X alone.Most GANs show decreasing Hellinger distance during early training followed by stable oscillations, and larger input-noise standard deviations reduce fluctuation.
Weather Evaluation: Lower RMSE and well-matched spread indicate reliable ensemble forecasts; X-tny-r, X-tny-w, and X-tny-w* slightly outperform the polynomial regression.White-noise GANs are generally underdispersive, while most red-noise GANs are somewhat overdispersive; X-med-r has a spread/error ratio closest to 1.
Climate Evaluation: Several GANs reproduce the Lorenz ’96 climatological PDF as well as or better than the benchmark bespoke polynomial parameterization.XU-sml-w and XU-sml-r produce overly broad, symmetric PDFs, whereas the other evaluated models reproduce the non-Gaussian truth distribution more skillfully.
Climate Evaluation: Models that skillfully capture the X PDF also represent the system’s spatio-temporal correlations, with X-sml-w* improving over the stochastic polynomial approach.The paper links strong marginal-distribution skill with strong correlation statistics across the evaluated parameterized models.
Climate Evaluation: Most GAN parameterizations reproduce the Lorenz ’96 regime structures, although GANs transition between regimes slightly more often than the truth run.The polynomial parameterization is slightly more likely to remain in the same regime, while XU-sml-w and XU-sml-r are exceptions to generally good regime reproduction.
Wavelet Analysis: GANs are competitive with or improve upon the polynomial parameterization across wavenumber climate metrics, while spectral performance varies by period.All small-noise GANs except XU-sml-r and XU-sml-w closely follow the truth power curve, but no GAN consistently performs best at every wavelength.

4 GAN Health Risks

GAN parameterizations face unstable training and mismatches between offline diagnostics and forecast-model performance. Applying GANs to correlated fields can also introduce nonlinear feedbacks, making successful integration difficult to predict.

Offline verification statistics and GAN losses did not correlate with performance in weather and climate integrations.The generator and discriminator may converge to an equilibrium that is unstable or suboptimal.
Applying GANs trained on independent, identically distributed data to sequentially processed spatially and temporally correlated fields can introduce nonlinear feedback effects.
Larger additive noise can help prevent runaway feedbacks from model biases but increases weather prediction errors.

5 Discussion

The study finds that GAN stochastic parameterizations can match or exceed a bespoke polynomial scheme across weather and climate evaluations, while performance depends on noise and model configuration. Weather forecast skill generally tracks climate performance, but the Lorenz ’96 setting limits direct generalization to more complex systems.

5 Discussion: Several GANs achieve weather and climate skill competitive with the bespoke polynomial parameterization, and some improve upon it.The evaluation included distributional, spatio-temporal correlation, and regime metrics.
5 Discussion: Weather models with lower RMSE generally also perform well on climate metrics, whereas forecast spread-error reliability is a poor predictor of climate performance.The comparison links initialized forecast accuracy more closely to climate performance than forecast reliability.
5 Discussion: Wavelet analysis reveals that some GANs add energy at long periods or across all periods, exposing scale-dependent differences missed by time-domain metrics.
5 Discussion: Noise standard deviation affects both GAN training and simulations: excessive noise limits distribution learning, while insufficient noise can break GAN equilibrium.
5 Discussion: Adding hidden-layer neurons did not improve performance in the relatively simple Lorenz ’96 system, while activation and regularization choices affected training or results.
5 Discussion: Lorenz ’96 has fewer dimensions and a simpler target distribution than Earth system models, so GAN overfitting and benefits from extra capacity may differ in higher-dimensional systems.
5 Discussion: Higher-complexity Earth system output can provide spatial training coverage but is temporally limited by available computational resources.

6 Conclusions

The study develops and evaluates stochastic GAN parameterizations for Lorenz ’96, identifying models that outperform a bespoke benchmark and reproduce key system statistics. It also highlights sensitivity to noise design, temporal dependence, and model complexity.

6 Conclusions: The GAN framework approximates the joint distribution of resolved states and sub-grid tendencies, with a subset outperforming the benchmark bespoke cubic polynomial parameterization.The same subset produces the most accurate weather forecasts and climate simulations across probability distributions, spatial and temporal correlations, and regime dynamics.
6 Conclusions: Models with the most accurate weather forecasts also produce the most accurate climate simulations, although reliable ensemble spread does not necessarily accompany the lowest forecast error.The models that perform best in offline mode are not necessarily those that perform best in weather and climate forecasts.
6 Conclusions: GAN performance required iterative development and was highly sensitive to noise magnitude and other hyperparameters.The authors suggest newer GAN losses and regularizers as possible ways to reduce failure modes.
6 Conclusions: The study demonstrates that GANs are a promising machine learning approach for stochastic parameterization in complex general circulation models.Future work will test machine-learned stochastic parameterizations in higher-complexity Earth system models and compare alternative generative frameworks.
6 Conclusions: Including the previous timestep can build temporal dependence into the parameterization but may accumulate forecast error, motivating consideration of local-in-time parameterizations.The conclusion identifies temporal-history inputs as a possible source of forecast degradation.
6 Conclusions: Autocorrelated noise is important for skillful weather forecasts but appears less important for reproducing the climatological distribution.This distinction indicates that noise characteristics affect forecast and climate performance differently.
6 Conclusions: Spatial correlations may matter in higher-complexity Earth system models, but their importance could not be assessed with the simple Lorenz ’96 system.The study therefore motivates evaluation beyond this proof-of-concept model.
6 Conclusions: The GAN framework may eventually learn noise characteristics automatically, reducing the need to manually tune stochasticity.This possibility is presented as an unanswered methodological question rather than an established result.