FuXi: A cascade machine learning forecasting system for 15-day global weather forecast

Lei Chen; Xiaohui Zhong; Feng Zhang; Yuan Cheng; Yinghui Xu; Yuan Qi; Hao Li

Source-linked AI summary

FuXi: A cascade machine learning forecasting system for 15-day global weather forecast

Lei Chen, Xiaohui Zhong, Feng Zhang, Yuan Cheng, Yinghui Xu, Yuan Qi, Hao Li

arXiv:2306.12873v3Jun 22, 2023physics.ao-phcs.AIcs.LG

TL;DR

Long-term ML weather forecasting remains challenged by accumulating iterative errors, and a single model may not perform optimally across lead times. FuXi uses cascaded models to produce 15-day global forecasts and achieves performance comparable to ECMWF EM, with ACC >0.6 skillful lead times of 10.5 days for Z500 and 14.5 days for T2M.

Problem
Accumulating errors challenge long-term ML weather forecasts, while a single model is insufficient for optimal performance across different lead times.
Method
FuXi cascades separately fine-tuned ML models for 0–5, 5–10, and 10–15-day windows to generate 15-day forecasts.
Results
FuXi achieves performance comparable to ECMWF EM and extends ACC >0.6 skillful lead times to 10.5 days for Z500 and 14.5 days for T2M.
Takeaways & Limitations
FuXi is the first ML-based weather forecasting system reported to perform comparably to ECMWF EM in 15-day forecasts.
Takeaways & Limitations
FuXi ensemble perturbations use flow-independent Perlin noise that decays during model integration, potentially limiting ensemble spread at longer lead times.

Abstract

from arXiv · show

Over the past few years, due to the rapid development of machine learning (ML) models for weather forecasting, state-of-the-art ML models have shown superior performance compared to the European Centre for Medium-Range Weather Forecasts (ECMWF)'s high-resolution forecast (HRES) in 10-day forecasts at a spatial resolution of 0.25 degree. However, the challenge remains to perform comparably to the ECMWF ensemble mean (EM) in 15-day forecasts. Previous studies have demonstrated the importance of mitigating the accumulation of forecast errors for effective long-term forecasts. Despite numerous efforts to reduce accumulation errors, including autoregressive multi-time step loss, using a single model is found to be insufficient to achieve optimal performance in both short and long lead times. Therefore, we present FuXi, a cascaded ML weather forecasting system that provides 15-day global forecasts with a temporal resolution of 6 hours and a spatial resolution of 0.25 degree. FuXi is developed using 39 years of the ECMWF ERA5 reanalysis dataset. The performance evaluation, based on latitude-weighted root mean square error (RMSE) and anomaly correlation coefficient (ACC), demonstrates that FuXi has comparable forecast performance to ECMWF EM in 15-day forecasts, making FuXi the first ML-based weather forecasting system to accomplish this achievement.

1 Introduction

ML weather forecasting has advanced to high-resolution 10-day forecasts, but cumulative errors make longer lead times challenging. FuXi addresses this gap with a cascade architecture that achieves performance comparable to ECMWF EM and extends skillful forecasts to 10.5 and 14.5 days for Z500 and T2M.

Motivation: NWP forecasting requires high-performance computing, with some simulations taking several hours on thousands of nodes.
Motivation: ML weather forecasting offers faster speeds and potential accuracy gains over traditional NWP models.
Prior work: 0.25° FourCastNet forecasts extended high-resolution global prediction to 7 days, but remained less accurate than ECMWF HRES.
Contribution: FuXi proposes a cascade ML architecture to reduce accumulation errors and achieves comparable performance to ECMWF EM.The system uses 39 years of 6-hourly ECMWF ERA5 reanalysis data at 0.25° spatial resolution.
Contribution: 10.5 and 14.5 days are the skillful forecast lead times with ACC >0.6 for Z500 and T2M, respectively.

2 Dataset

FuXi is trained and evaluated using a 39-year, 6-hourly, 0.25° ERA5 subset containing 70 atmospheric variables. Data are split into training, validation, and 2018 out-of-sample testing sets, with ECMWF reference datasets used for forecast assessment.

Dataset source: ERA5 provides hourly surface and upper-air reanalysis data at approximately 31 km resolution across 137 model levels from 1940 onward.The dataset assimilates global observations using ECMWF’s IFS model.
Dataset configuration: The study uses 39 years of ERA5 at 0.25° resolution on a 721 × 1440 grid with 6-hour temporal spacing.Predictions cover five upper-air variables at 13 pressure levels and five surface variables.
Prediction targets: 70 variables are predicted and evaluated, including geopotential, temperature, horizontal wind components, mean sea-level pressure, and total precipitation.The variables include upper-air and surface atmospheric quantities.
Data splits: The training set contains 540203 samples from 1979–2015, validation uses 2920 samples from 2016–2017, and testing uses 1460 samples from 2018.The 2018 test set is out-of-sample relative to the training and validation periods.
Evaluation references: Model performance is evaluated against ERA5 and ECMWF forecasts using HRES-fc0 and ENS-fc0 reference datasets.HRES-fc0 and ENS-fc0 contain the first time step of each HRES and ensemble control forecast, respectively, for assessing ECMWF HRES and EM.

3 Methodology · 3.1 Generating 15-day forecasts using FuXi · 3.2 FuXi model architecture

FuXi generates autoregressive 15-day forecasts through a cascade of specialized 5-day models. Its base architecture combines cube embedding, a U-Transformer, and a fully connected layer to process high-resolution atmospheric inputs.

3.1 Generating 15-day forecasts using FuXi: FuXi autoregressively uses weather parameters from two preceding 6-hour time steps to forecast the upcoming time step.Its outputs are fed back as inputs to generate forecasts at different lead times.
3.1 Generating 15-day forecasts using FuXi: 60 iterative runs are required to generate a 15-day forecast with a single FuXi model.Iterative forecasting can accumulate errors and produce unrealistic long-term predictions because pure data-driven models lack physical constraints.
3.1 Generating 15-day forecasts using FuXi: FuXi uses three fine-tuned models—FuXi-Short, FuXi Medium, and FuXi-Long—to specialize in successive 5-day forecast windows.The cascade is designed to optimize performance across short and long lead times because a single model cannot perform optimally at every lead time.
3.2 FuXi model architecture: The base FuXi architecture consists of cube embedding, a U-Transformer, and a fully connected layer.Its input combines upper-air and surface variables in a 2 × 70 × 721 × 1440 data cube representing two preceding time steps and 70 variables.
3.2 FuXi model architecture: Cube embedding reduces the input to C × 180 × 360 through joint space-time processing, with C set to 1536, before the U-Transformer and fully connected layer generate predictions.The output is initially reshaped to 70 × 720 × 1440 and then restored to the original format.
3.2.1 Cube embedding: The space-time cube embedding applies a 3D convolution with a 2×4×4 kernel and stride to reduce spatial and temporal dimensions and accelerate training.Layer normalization follows embedding to improve training stability, producing a C × 180 × 360 data cube.
3.2.2 U-Transformer: The U-Transformer uses 48 repeated Swin Transformer V2 blocks and includes U-Net-style downsampling and upsampling.Its Down Block reduces the data dimension to C × 90 × 180, lowering computational and memory requirements for self-attention.

3.3 FuXi model training

FuXi training uses supervised single-step pre-training followed by autoregressive curriculum fine-tuning. Three cascaded models are optimized for progressively longer lead times and combined to produce 15-day forecasts while reducing accumulation errors.

Training procedure: Training comprises pre-training and fine-tuning, following an approach similar to GraphCast.Pre-training optimizes FuXi to predict a single time step using the training dataset.
Pre-training: The pre-training objective is a latitude-weighted L1 loss averaged across grid points and variables.Latitude weights decrease as latitude increases.
Fine-tuning: Fine-tuning uses autoregressive curriculum training, increasing steps from 2 to 12 for 6-hourly forecasts spanning 0–5 days.FuXi-Short is optimized for 0–5-day forecasts, while the same procedure is applied to longer-lead models.
Cascaded forecasting: FuXi-Short, FuXi-Medium, and FuXi-Long are cascaded to generate complete 15-day forecasts and reduce accumulation errors at longer lead times.FuXi-Long is optimized for forecasts of 10–15 days.
Implementation: Fine-tuning each cascaded model takes approximately two days on a cluster of 8 Nvidia A100 GPUs using a constant learning rate of 1×10−7.The model is developed using PyTorch.

3.4 FuXi ensemble forecast

Because weather forecasting is uncertain due to atmospheric chaos, ensemble forecasting is necessary, especially at longer lead times. FuXi generated a 50-member ensemble forecast at substantially lower computational cost than conventional NWP models, following ECMWF’s ensemble approach.

Weather forecasting is uncertain because of the chaotic nature of the weather system.
Ensemble forecasting is necessary to address this uncertainty, particularly at longer lead times.
50-member ensemble forecasts were generated using the FuXi model at significantly lower computational costs than conventional NWP models.
The FuXi ensemble runs followed the approach used by ECMWF, which involves perturbing both initial conditions and other forecast inputs.The supplied passage truncates after “initial condit,” so the latter part of the approach is only partially specified.

3.5 Evaluation method

Forecast performance is evaluated with latitude-weighted RMSE and ACC, supplemented by normalized differences for distinguishing closely performing models. ECMWF forecasts and ensemble quality are assessed with dedicated verification procedures and metrics.

Deterministic forecast metrics: Forecast performance is evaluated using latitude-weighted root mean square error (RMSE) and ACC.These metrics follow the evaluation approach in.
Model comparison: Normalized RMSE difference is (RMSEA −RMSEB)/RMSEB, while normalized ACC difference is (ACCA −ACCB)/(1 −ACCB).The climatological mean M is calculated from ERA5 reanalysis data between 1993 and 2016; negative RMSE and positive ACC normalized differences favor model A over baseline B.
ECMWF verification: ECMWF HRES and EM are evaluated using ECMWF’s verification method, with HRES-fc0 and ENS-fc0 analyses serving as their respective ground truths.The ground truth differs between the deterministic HRES and ensemble-mean EM evaluations.
Ensemble forecast metrics: Ensemble forecast quality is assessed using the continuous ranked probability score (CRPS) and spread-skill ratio (SSR).CRPS is calculated with the xskillscore Python package; ensemble members are assumed Gaussian, using the ensemble mean and variance.
Ensemble forecast metrics: A reliable ensemble has an SSR of one, whereas lower values indicate underdispersion and higher values indicate overdispersion.SSR measures consistency between ensemble spread and the EM’s RMSE.

4 Results

Using 2018 testing data, FuXi produced 6-hourly forecasts initialized at 00:00 and 12:00 UTC for 15 days. FuXi outperformed ECMWF HRES and GraphCast at longer lead times and showed comparable 15-day performance to ECMWF EM.

Experimental setup: 2018 data supported 6-hourly FuXi forecasts initialized daily at 00:00 and 12:00 UTC across a 15-day horizon.The evaluation used two daily initialization times and 15-day forecasts.
Deterministic forecast metrics: FuXi and GraphCast significantly outperformed ECMWF HRES, with comparable performance through 7 days and FuXi superior thereafter across all evaluated variables and lead times.FuXi achieved the lowest RMSE and highest ACC beyond 7 days among the compared forecasts.
Deterministic forecast metrics: FuXi outperformed ECMWF EM during 0–9-day forecasts but was slightly poorer beyond 9 days, while remaining comparable over 15 days.FuXi had higher ACC and lower RMSE than ECMWF EM on 67.92% and 53.75% of 240 testing combinations, respectively.
Spatial forecast performance: FuXi outperformed ECMWF HRES at most grid points, while ECMWF EM showed comparable spatial performance across most areas.FuXi’s spatial RMSE patterns were evaluated for Z500 and T2M at 5-, 10-, and 15-day lead times.
Ensemble forecast metrics: The ensemble comparison evaluated CRPS, latitude-weighted Spread, and SSR for FuXi and ECMWF ensembles over the same 15-day testing forecasts.The comparison used Z500, T850, MSL, and T2M.

5 Conclusion and Future Work

FuXi uses cascaded, fine-tuned ML models to achieve performance comparable to ECMWF EM in 15-day forecasts. Future work targets improved ensemble spread, sub-seasonal forecasting, and end-to-end data assimilation.

Conclusion: Single-model ML forecasting is insufficient for optimal performance across the 0-5-, 5-10-, and 10-15-day windows.Recent ML systems outperform ECMWF HRES in 10-day forecasts, but reducing accumulation error remains challenging.
Conclusion: FuXi cascades fine-tuned models for different forecast windows to produce 15-day forecasts comparable to ECMWF EM at 6-hour and 0.25◦ resolution.The system is described as the first ML-based weather forecasting system to achieve this 15-day comparison.
Future Work: Flow-independent Perlin-noise perturbations may decay during integration, motivating flow-dependent initial-condition perturbations to maintain adequate medium-range ensemble spread.FuXi currently incorporates random Perlin noise into initial conditions for ensemble forecasts.
Future Work: Future work will examine the cascade architecture for sub-seasonal forecasting by fine-tuning models for 14-28-day lead times.Sub-seasonal forecasting remains difficult, and ensemble forecasts are necessary rather than deterministic methods.
Future Work: Future work aims to develop data-driven data assimilation that uses observations to generate initial conditions for truly end-to-end ML weather forecasting.Current ML methods still rely on analysis data generated by conventional NWP models for initial conditions.

Code Availability Statement

The study uses Swin Transformer V2 as FuXi’s backbone, while the FuXi model, training and inference code, and sample input data are publicly accessible through a Google Drive folder.

Code Availability Statement: Swin Transformer V2 provides the backbone architecture, with its code available on GitHub.The cited repository is https://github.com/microsoft/Swin-Transformer.
Code Availability Statement: The FuXi model, training and inference source code, and sample input data are available through a linked Google Drive folder.Access is provided to individuals with the supplied link.

Appendix · A Effectiveness of the cascade model architecture

This appendix evaluates whether the cascade ML architecture reduces accumulation errors by comparing 15-day forecasts from FuXi with those from a single non-cascade base model. The comparison finds comparable Z500 and T2M performance through 7 days and reports globally averaged latitude-weighted ACC.

A Effectiveness of the cascade model architecture: The appendix examines the cascade ML architecture as a method for reducing accumulation errors in weather forecasting.
A Effectiveness of the cascade model architecture: The experiment uses a single FuXi base model, called FuXi-short, without the cascade, to generate 15-day forecasts.
A Effectiveness of the cascade model architecture: The single base FuXi model is evaluated against the original FuXi model.
A Effectiveness of the cascade model architecture: 0–7 days: FuXi-short and FuXi show comparable performance for Z500 and T2M forecasts.
A Effectiveness of the cascade model architecture: The comparison covers lead times ranging from 0 to 7 days.
A Effectiveness of the cascade model architecture: Figure A.1 compares the globally averaged latitude-weighted ACC of FuXi and FuXi without cascade.
A Effectiveness of the cascade model architecture: Figure A.1 represents FuXi with red lines and FuXi without cascade with gray lines.

B Skillful forecast lead time comparison

FuXi improves skillful forecast lead times for all eight evaluated variables, with the largest reported gain for T2M, reaching 14.5 days versus 10 days for ECMWF HRES and Graphcast.

B Skillful forecast lead time comparison: FuXi improves the skillful lead time of all eight variables shown.The comparison includes four surface variables and four upper-air variables at 500 hPa.
B Skillful forecast lead time comparison: 14.5 days: FuXi extends T2M skillful lead time from 10 days for ECMWF HRES and Graphcast.T2M shows the most significant improvement among the variables discussed.
B Skillful forecast lead time comparison: The comparison evaluates ACC >0.6 skillful lead times for eight variables across ECMWF HRES, Graphcast, and FuXi.The variables comprise MSL, T2M, U10, V 10, Z500, T500, U500, and V 500.