PaperlayerView exact revision on arXiv

Source-linked AI summary

ACE2: Accurately learning subseasonal to decadal atmospheric variability and forced responses

Oliver Watt-Meyer, Brian Henn, Jeremy McGibbon, Spencer K. Clark, Anna Kwa, W. Andre Perkins, Elynn Wu, Lucas Harris, Christopher S. Bretherton

arXiv:2411.11268v1physics.ao-phcs.LG

TL;DR

Machine-learning climate models have not been formulated to assess atmospheric responses to changing external forcing, and generalization to unseen climate conditions remains a challenge. ACE2 is an autoregressive emulator evaluated on historical atmospheric variability and forced responses. It reproduces several observed variability patterns and long-term trends, although its forcing sensitivities and strong-climate-change scope remain limited.

  • Problem

    Existing long-term atmospheric emulators have not demonstrated accurate responses to changing external forcing across the past 80 years, and generalization to unseen climate conditions remains a challenge.

    Source: §1 · p. 1

  • Method

    ACE2 is an autoregressive atmospheric emulator trained on ERA5 reanalysis or GFDL’s SHiELD historical simulation, with user-specified SST and CO2 forcing.

    Source: §1 · p. 2

  • Results

    ACE2 reproduces atmospheric responses to El Niño, the Madden Julian Oscillation, tropical cyclone distributions, and global temperature trends over the historical period.

    Source: §1 · p. 2, §3 · p. 13

  • Takeaways & Limitations

    The study demonstrates the feasibility of emulating atmospheric variability and forced responses from days to decades while exactly conserving dry air mass and moisture.

    Source: §3 · p. 13

    Because training data cover only the past 80 years, ACE2 is not expected to properly simulate strong climate change such as doubled CO2, and its separate SST and CO2 sensitivities are not entirely realistic.

    Source: §2.4 · p. 13, §3 · p. 14

Abstract

from arXiv · show

Existing machine learning models of weather variability are not formulated to enable assessment of their response to varying external boundary conditions such as sea surface temperature and greenhouse gases. Here we present ACE2 (Ai2 Climate Emulator version 2) and its application to reproducing atmospheric variability over the past 80 years on timescales from days to decades. ACE2 is a 450M-parameter autoregressive machine learning emulator, operating with 6-hour temporal resolution, 1{\deg} horizontal resolution and eight vertical layers. It exactly conserves global dry air mass and moisture and can be stepped forward stably for arbitrarily many steps with a throughput of about 1500 simulated years per wall clock day. ACE2 generates emergent phenomena such as tropical cyclones, the Madden Julian Oscillation, and sudden stratospheric warmings. Furthermore, it accurately reproduces the atmospheric response to El Ni\~no variability and global trends of temperature over the past 80 years. However, its sensitivities to separately changing sea surface temperature and carbon dioxide are not entirely realistic.

1 Introduction

Machine-learning climate emulators offer faster, differentiable simulations, but whether they can generalize to unseen climate conditions remains uncertain. ACE2 is evaluated on historical atmospheric variability and trends, including its response to changing external forcing.

1 Introduction

  • Machine-learning climate models can accelerate simulations through efficient GPU execution and longer time steps, potentially supporting large ensembles, rare-event study, and forced-response analysis.Their differentiability also makes them useful for data assimilation applications.Source: §1 · p. 1
  • Whether machine learning will produce more accurate climate models remains uncertain, while climate prediction commonly requires forecasts of previously unseen conditions such as warming under doubled CO2.Out-of-sample generalization is identified as a fundamental challenge that may require physics-based priors or training on physics-based model output.Source: §1 · pp. 1–2
  • ACE2 is evaluated on 1940–2020 atmospheric variability and trends, with skillful training demonstrated on ERA5 reanalysis and GFDL’s SHiELD historical simulation.SHiELD can also simulate perturbed climates for training an emulator intended to simulate long-term climate change.Source: §1 · p. 2
  • ACE2 adds CO2 forcing, emulates observed atmospheric trends over 80 years, and exactly conserves dry air mass and atmospheric moisture.It operates at 1° horizontal resolution with eight vertical layers and can integrate forward in 6-hour steps under user-specified SST.Source: §1 · p. 2
  • ACE2’s evaluation covers atmospheric responses to El Niño, long-term trends and interannual variability of global mean temperature and total water path, and weather forecast skill.The ERA5-trained model is also evaluated on tropical cyclones and the Madden Julian Oscillation, phenomena less well represented in the coarser training models used previously.Source: §1 · p. 2
  • No previously reported long-term stable atmospheric emulator demonstrates accurate responses to changing external forcing over the past 80 years.The cited NeuralGCM simulations had reasonable trends and low climate biases, but about one third became unstable before 30 years.Source: §1 · p. 2

2 Results

The results assess ACE2’s reproduction of climate trends and atmospheric variability, including its simulation stability and limitations in forecast skill, regional biases, and stratospheric variability. They also examine how temperature trends vary across CO2 configurations and report on performance at coarser resolution.

2.2.1 Climate skill

  • 1.05mm versus 1.09 mm: ACE2-ERA5’s 2020 total-water-path error was similar to NeuralGCM’s, and ACE2 models outperformed ACE-climSST across test-period variables.Over the 10-year test period, ACE2-SHiELD errors were typically 1.1–1.5 times SHiELD reference variability. In 81-year runs, ACE2 tracked reference long-term trends, while interannual variability was slightly muted; ACE2-ERA5’s mean 2-meter temperature R2 was 0.93 versus 0.97 for SHiELD reference variability.Source: §2.1 · pp. 2–3, Figure 1 · p. 3, §2.2.1 · p. 4

2.2.2 Atmospheric response to ENSO variability

  • 0.46 and 0.48 mm/day/K: ACE2-ERA5 and ACE2-SHiELD ENSO precipitation regression-map RMSEs were below ACE-climSST’s 0.64 mm/day/K and near SHiELD variability at 0.54 mm/day/K.ACE2 reproduced the canonical precipitation response and details of its ERA5 and SHiELD reference maps; the learned response was qualitatively similar in 81-year rollouts.Source: §2.2.2 · pp. 5–6, Figure 4 · p. 5

2.2.3 Tropical cyclone climatology

  • ACE2-ERA5 overpredicted global tropical cyclone frequency by about 28% versus ERA5, but closely matched ERA5 basin-by-basin frequency and cyclone strength.The reported frequency depends on detection settings applied at 1° resolution; the climatological-SST test produced a similar frequency and distribution to historical-SST forcing.Source: §2.2.3 · pp. 6–7, Figure 5 · p. 7

2.2.4 Tropical precipitation variability

  • ACE2-ERA5 captured MJO eastward propagation in ERA5 with consistent phase speed and longitudinal extent, whereas SHiELD lacked coherent propagation.ACE2-ERA5 also represented ERA5’s tropical precipitation variability more closely than SHiELD, including eastward-propagating Kelvin waves, though its fields were smoother.Source: §2.2.4 · pp. 7–8, Figure 7 · p. 8, Figure 6 · p. 8

2.2.5 Polar stratospheric variability

  • ACE2-ERA5 reproduced the seasonal asymmetry of polar stratospheric vortex strength and variability, including Northern Hemisphere sudden stratospheric warmings, but its QBO-like winds had an incorrect period.The ten-year comparison supports qualitative rather than quantitative assessment of vortex variability.Source: §2.2.5 · p. 9, Figure 8 · p. 9

2.2.6 Weather skill

  • ACE2-ERA5 medium-range forecasts lagged the ERA5-version IFS by about half a day at five-day lead for 850-temperature forecasts and lagged GraphCast by another day.Forecast skill was assessed using global RMSE over 10-day forecasts initialized throughout 2020.Source: §2.2.6 · p. 10, Figure 9 · p. 10

2.2.7 Millennial timescale stability

  • ACE2-ERA5 showed no instability or long-term drift in a 1000-year run forced by repeating climatological conditions, and its seasonal total-water-path cycle remained consistent.Time-mean climate was nearly identical across different 100-year periods, improving on the unrealistic seasonal-cycle fluctuations reported for ACE-climSST.Source: §2.2.7 · p. 10, Figure 10 · p. 11

2.3 Learning at coarser horizontal resolution

  • 4° ACE2 had slightly higher time-mean temperature and precipitation biases than 1° ACE2, but both were less biased than C24 SHiELD; ACE2 was about 25x more energy efficient than C24.The 4° and 1° emulators had similar ENSO precipitation-regression skill and reproduced long-term trends and interannual variability similarly; the 1° model represented the overall temperature trend better.Source: §2.3 · pp. 11–12, Figure 11 · p. 11, Figure 12 · p. 12
  • At 4° resolution, performance may benefit from a larger embedding dimension, possibly because coarser resolution has increased subgrid activity.The 4° model was trained with the same hyperparameters as the 1° model for consistency.Source: §2.3 · p. 12

2.4 Using CO2 as an input feature

  • With historical SST and CO2 forcing, ACE2 approximately matches reference near-surface warming and stratospheric cooling; fixing CO2 removes stratospheric cooling and much of the surface-warming trend.The fixed-CO2 experiment retains increasing SSTs and uses the 1940 CO2 concentration of 307 ppm.Source: §2.4 · p. 13, Figure 14 · p. 13
  • Training with SST but without CO2 somewhat underestimates reference warming and stratospheric-cooling trends, while stratospheric temperature has excess interannual variability.Using CO2 forcing appears to improve some CO2-induced trends but introduces non-physical relationships; additional SHiELD simulations with fixed CO2 could test whether training-data augmentation improves physical sensitivities.Source: §2.4 · p. 13

3 Discussion

ACE2 demonstrates the feasibility of emulating atmospheric variability and forced responses from days to decades while conserving dry air mass and moisture. Its scope is limited by the historical training data and imperfect physical sensitivities, and future work includes broader CO2 training and additional climate components.

3 Discussion

  • ACE2 generates atmospheric variability and forced responses from days to decades, while exactly conserving dry air mass and moisture.Its autoregressive six-hour stepping simulates century-long trends, and explicitly resolved weather may make its behavior more interpretable.Source: §3 · p. 13
  • Training on the past 80 years limits expected skill under strong climate change, such as a doubling of CO2, and ACE2 does not accurately represent the response to increasing SST with CO2 fixed.The ERA5 and SHiELD datasets also have shortcomings, and ACE2 does not exactly conserve global atmospheric energy.Source: §3 · p. 14
  • Future work will train ACE2 on SHiELD simulations spanning wider CO2 concentrations and add ocean and sea-ice components.The paper identifies these additional climate components as a basic requirement for a useful climate-model emulator.Source: §3 · p. 14

4 Methods

ACE2 is evaluated on ERA5 and SHiELD data, with training and checkpoint selection designed for climate statistics and hard physical constraints. Its inference is substantially faster and less energy-intensive than SHiELD at both compared resolutions.

4.1 Versioning nomenclature

  • ACE2 introduces increased parameter count, an updated loss, hard mass and moisture constraints, and a new checkpoint strategy; ACE-climSST denotes the earlier ACE trained with repeating climatological SSTs.Results distinguish ACE2-SHiELD from ACE2-ERA5 according to their training datasets.Source: §4.1 · p. 14

4.2 Datasets

  • ACE2 is trained on ERA5 reanalysis or SHiELD simulations, with sea surface temperature, carbon dioxide, solar radiation, and surface properties supplied as forcing inputs.The datasets span 1940–2022 for ERA5 and 1940–2021 for SHiELD; SHiELD targets use observed SSTs and greenhouse gases.Source: §4.2 · pp. 14–15
  • ACE2 combines model-level temperature, water, and wind fields into eight layers and adds near-surface prognostic variables relevant to human impacts and ocean coupling.The added variables include 2-meter temperature and humidity and 10-meter winds.Source: §4.2 · p. 14
  • SHiELD output is conservatively regridded, smoothed by a spherical harmonic transform round trip, and vertically coarsened from 79 layers to ACE’s eight layers.Reference data are aligned to ACE2’s 1° grid and 6-hour intervals, with fluxes averaged over each interval.Source: §4.2 · pp. 14–15

4.3 Training

  • ACE2 uses an SFNO with a 384 embedding dimension and a physical corrector; training minimizes two-step 6-hour forecast loss, while checkpoints are selected by time-mean climate biases.The corrector enforces physical constraints, including positive moisture, precipitation, and radiative fluxes, and conservation of dry air mass and moisture.Source: §4.3 · pp. 16–17

4.4 Evaluation metrics

  • ACE2 evaluates climate skill using global RMSE of time-mean fields, R2 for global- and annual-mean series, and area-weighted RMS differences between ENSO response maps.The time-series R2 reflects long-term means and trends as well as interannual variability.Source: §4.4 · pp. 18–19

4.5 Computational cost

  • ACE2 is about 100 times faster and 700 times less energy-intensive than C96 SHiELD, and about 50 times faster and 25 times less energy-intensive than C24 SHiELD.ACE2 and C96 have the same horizontal resolution; C24 has four times lower horizontal resolution.Source: §4.5 · p. 19, Table 3 · p. 19

A.1 Spatial variability of temperature trends

ACE2-SHiELD reproduces most spatial patterns of 1940–2020 surface-air-temperature trends in the SHiELD reference, with regional exceptions.

A.1 Spatial variability of temperature trends

  • ACE2-SHiELD closely follows SHiELD’s 1940–2020 2-meter temperature trends over open ocean, land, and polar regions, with excessive Himalayan warming and slightly weak warming over Siberia and North America.Open-ocean temperature trends closely follow the prescribed SST trends.Source: §A.1 · p. 24

A.2 OLR response to ENSO

ACE2 reproduces the atmospheric OLR response to ENSO with lower mean pattern error than the earlier ACE-climSST model. The ACE2 error is slightly below the reported variability of the SHiELD reference response.

A.2 OLR response to ENSO

  • 2.6 W/m2/K is the mean error in ACE2’s OLR response to Niño3.4 for both ACE2-ERA5 and ACE2-SHiELD, below SHiELD response variability of 3.0 W/m2/K.ACE-climSST has a muted tropical-Pacific OLR response and a larger mean pattern error of 3.7 W/m2/K.Source: §A.2 · p. 24

A.3 Tropical cyclone statistics and dependence on sea surface temperature dataset

ACE2-ERA5 produces similar tropical-cyclone counts and geographic distributions when forced by observed historical SSTs or climatological SSTs.

A.3 Tropical cyclone statistics and dependence on sea surface temperature dataset

  • Using climatological rather than actual 2001–2010 sea surface temperatures, simulations produced very similar annual tropical-cyclone counts and geographic distributions, suggesting true historical SSTs did not strongly influence cyclone generation in these tests.Source: §A.3 · p. 25

A.4 Seed variability and checkpoint selection

Across the training ensembles, validation loss declined, but climate accuracy did not improve monotonically and could deteriorate with further training. The two datasets also differed in validation loss and in how climate accuracy changed during training.

A.4 Seed variability and checkpoint selection

  • ACE2-ERA5's final validation loss was about two times larger than ACE2-SHiELD's, although losses steadily decreased across both four-model ensembles.The authors suggest ERA5 is harder to learn than SHiELD, a 100 km atmosphere-only numerical model.Source: §A.4 · p. 25, Figure 19 · p. 26
  • All trained models were indefinitely stable, but climate accuracy varied substantially across models and training checkpoints.Training optimized predictions over 12 hours, not the much longer autoregressive trajectories used in climate simulations.Source: §A.4 · pp. 25–26
  • Climate accuracy can worsen despite declining validation loss, deteriorating after about 50,000 iterations in some runs before sometimes improving later.Climate accuracy is time-mean pattern RMSE averaged over output variables.Source: §A.4 · p. 26, Figure 19 · p. 26

A.5 Ablation of hard physical constraints

The constraint ablations show that dry-air and moisture conservation close their respective global budgets, with distinct and qualified effects on climate biases. They were evaluated only for SHiELD-trained models, using fewer random seeds for ablated cases.

A.5 Ablation of hard physical constraints

  • The ablations used only SHiELD-trained models and two random seeds, versus four seeds for the primary models; positivity-constraint sensitivity was not shown.The authors report that the positivity constraint had its intended effect without worsening any skill metric.Source: §A.5 · pp. 26–27
  • Without dry-air conservation, global dry-air surface pressure deviated by up to 20 Pa; without moisture conservation, single-step total-water-path budget violations exceeded 0.2 mm/day.Dry-air conservation produced constant global dry-air mass, while the moisture constraint closed the global total-water-path budget.Source: §A.5 · p. 27, Figure 21 · p. 27
  • Dry-air conservation robustly reduced surface-pressure bias, while total-water-path bias changed inconsistently and remained small across constraint settings.Precipitation and evaporation biases decreased with the moisture constraint in the shown simulations, but not in all trained models.Source: §A.5 · p. 27, Figure 22 · p. 28
  • The time-mean spatial pattern errors for surface pressure, total water path, precipitation, and evaporation were not significantly changed by the global-mean constraints.The authors expected this because the constraints act on global means.Source: §A.5 · p. 27, Figure 22 · p. 28

A.5.1 Imposing constraints in ERA5, a non-conservative dataset

Although ERA5 has nonzero global budget residuals, ACE2-ERA5 exactly obeys the dry-air and moisture budgets. The ERA5 pressure residuals were larger earlier in the record, while its time-mean moisture residual was small.

A.5.1 Imposing constraints in ERA5, a non-conservative dataset

  • ACE2-ERA5 exactly obeyed the global dry-air and moisture budgets despite ERA5 residuals reaching 400 Pa for dry-air pressure and about 0.5 mm/day for moisture.ERA5 dry-air pressure fluctuations were around 10 Pa after 1979, and its time-mean moisture residual was below 0.005 mm/day.Source: §A.5.1 · p. 28
  • The largest ERA5 dry-air pressure variations occurred earlier in the record, when observing systems were more limited.After 1979, fluctuations were much smaller, on the order of 10 Pa.Source: §A.5.1 · p. 28
  • ACE2-ERA5's time-mean precipitation and evaporation biases were -0.05 mm/day and -0.04 mm/day, respectively, small relative to approximately 3 mm/day.These biases were slightly larger in magnitude than those for ACE2-SHiELD.Source: §A.5.1 · p. 28

B Dataset details

ACE2 uses eight vertical layers described by hybrid sigma-pressure coordinates, with dataset-specific vertical coordinates for ERA5 and SHiELD. The dataset tables specify the variables and vertical-level mapping used.

B Dataset details

  • ACE2's hybrid sigma-pressure coordinates approximately match SPEEDY's vertical coordinate, but use slightly different coordinates for SHiELD and ERA5 to avoid interpolating native layers.The pressure at a layer interface is determined from surface pressure and the hybrid sigma-pressure coordinates.Source: B Dataset details · pp. 28, 30
  • Table 4 lists ACE2 input and output variables, with height-dependent variables indexed by k from 0 at the top of the atmosphere to 7 toward the surface.Its time categories distinguish snapshots, six-hour means, and time-invariant quantities; TOA means Top Of Atmosphere.Source: B Dataset details · p. 28, Table 4 · p. 29
  • Table 5 maps ACE2's vertical interfaces, from k = 0 at the model top to k = 8 at the surface, to levels in the original datasets.The source datasets have 79 SHiELD layers and 137 ERA5 layers.Source: Table 5 · p. 30

C Training hyperparameters

The training setup lists SFNO and optimization hyperparameters, with model parameters averaged across training steps using an exponential moving average. The architecture changes only in its first and last spherical harmonic transforms to accommodate Gaussian-grid data.

C Training hyperparameters

  • The first and last spherical harmonic transforms use Gauss-Legendre quadrature to accommodate Gaussian-grid data, unlike the equiangular grid in Bonev et al. (2023).These are the only SFNO architecture modifications made in this work.Source: C Training hyperparameters · p. 30
  • Table 6 lists the SFNO hyperparameters; unlisted configuration options use the linked code's defaults.The parameter names correspond to the Spherical-FourierNeuralOperatorNet class.Source: Table 6 · p. 31
  • Table 7 lists the optimization hyperparameters used in the study.Source: Table 7 · p. 31
  • Model parameters were averaged across training steps using an exponential moving average (EMA).Source: C Training hyperparameters · p. 31

Related papers

Loading 2411.11268v1…