Source-linked AI summary
FengWu: Pushing the Skillful Global Medium-range Weather Forecast beyond 10 Days Lead
Kang Chen, Tao Han, Junchao Gong, Lei Bai, Fenghua Ling, Jing-Jia Luo, Xi Chen, Leiming Ma, Tianning Zhang, Rui Su, Yuanzheng Ci, Bin Li, Xiaokang Yang, Wanli Ouyang
TL;DR
Global medium-range weather forecasting remains limited by accuracy, long-lead error accumulation, and computational constraints. FengWu addresses the problem with a multi-modal, multi-task AI architecture, uncertainty-based task weighting, and a replay buffer. In 2018 ERA5 hindcasts, it outperformed GraphCast on 80% of 880 predictands and extended skillful z500 forecasts to 10.75 days.
Problem
Global medium-range weather forecasting is limited by forecast accuracy, extendibility, long-lead error accumulation, and computational and memory costs.
Method
FengWu uses modality-specific encoders and decoders, cross-modal Transformer fusion, uncertainty-based multi-task weighting, and a replay buffer for autoregressive long-lead prediction.
Results
80% of 880 reported predictands had higher accuracy than GraphCast, and skillful z500 forecasting reached 10.75 days with ACC > 0.6.
Takeaways & Limitations
FengWu extends skillful global medium-range AI weather forecasting to 10.75 days and achieves top performance among released data-driven systems.
Takeaways & Limitations
Comparisons with physics-based systems are affected by differing initial fields: FengWu and GraphCast use ERA5 states, while IFS-HRES uses its own initial analysis.
Abstract
from arXiv · showhide
We present FengWu, an advanced data-driven global medium-range weather forecast system based on Artificial Intelligence (AI). Different from existing data-driven weather forecast methods, FengWu solves the medium-range forecast problem from a multi-modal and multi-task perspective. Specifically, a deep learning architecture equipped with model-specific encoder-decoders and cross-modal fusion Transformer is elaborately designed, which is learned under the supervision of an uncertainty loss to balance the optimization of different predictors in a region-adaptive manner. Besides this, a replay buffer mechanism is introduced to improve medium-range forecast performance. With 39-year data training based on the ERA5 reanalysis, FengWu is able to accurately reproduce the atmospheric dynamics and predict the future land and atmosphere states at 37 vertical levels on a 0.25° latitude-longitude resolution. Hindcasts of 6-hourly weather in 2018 based on ERA5 demonstrate that FengWu performs better than GraphCast in predicting 80\% of the 880 reported predictands, e.g., reducing the root mean square error (RMSE) of 10-day lead global z500 prediction from 733 to 651 $m^{2}/s^2$. In addition, the inference cost of each iteration is merely 600ms on NVIDIA Tesla A100 hardware. The results suggest that FengWu can significantly improve the forecast skill and extend the skillful global medium-range weather forecast out to 10.75 days lead (with ACC of z500 > 0.6) for the first time.
5 The Institute of Atmospheric Physics, Chinese Academy of Sciences 6 Shanghai Meteorological Bureau
The section identifies the paper's main subject areas: medium-range weather prediction, deep learning, multi-modal multi-task learning, and Transformer architectures.
- Medium-range weather prediction is a central subject of the paper.
- Deep learning is identified as a core technical area.
- Multi-modal multi-task learning and Transformers are highlighted as architectural themes.
1 Introduction
Global medium-range weather forecasting remains important but is limited by accuracy, extendibility, and computational challenges. FengWu addresses these challenges with a multi-modal, multi-task AI design and replay buffer, achieving stronger forecast skill and a 10.75-day skillful lead time.
- Global medium-range forecasting predicts atmospheric conditions up to fourteen days ahead and supports operational and regional weather prediction.
- Existing systems remain constrained by initial and boundary uncertainties, nonlinear physical processes, and heavy computation costs.
- 10.75 days is FengWu's skillful z500 forecast lead time under ACC > 0.6, while t2m reaches 11.5 days.
- FengWu treats atmospheric variables at different pressure levels as separate modalities processed by modality-specific encoders, a cross-modal Transformer, and decoders.
- The uncertainty loss automatically balances variable-regression tasks according to homoscedastic uncertainty without expensive manual weight searches.
- The replay buffer reuses predictions from previous optimization iterations to mimic intermediate autoregressive errors for long-lead training.
- 80% of 880 reported predictands have higher accuracy than GraphCast in 2018 six-hourly ERA5 hindcasts.
2 Preliminary
FengWu uses ERA5 to model 189 atmospheric and surface predictands on a high-resolution global grid, generating six-hourly forecasts autoregressively toward 14 days.
- 2.1 Dataset: ERA5 provides FengWu with 0.25° latitude-longitude data across 37 vertical pressure levels for weather forecasting.The dataset covers variables including temperature, humidity, precipitation, wind, and pressure.
- 2.1 Dataset: FengWu simulates 189 predictands comprising five atmospheric variables across 37 pressure levels plus four surface variables.The atmospheric variables are geopotential, relative humidity, zonal and meridional wind, and air temperature.
- 2.1 Dataset: Training uses 1979–2015 data, validation uses 2016–2017, and testing uses 2018 with six-hourly samples.This validation strategy follows GraphCast’s evaluation setup.
- 2.2 Problem Formulation: FengWu represents weather states as tensors with C = 189, W = 721, and H = 1440 at 0.25° resolution.The system targets 14-day forecasts at six-hour intervals, producing 56 future states.
- 2.2 Problem Formulation: Because directly learning the full 14-day mapping is computationally difficult, FengWu predicts one next state and rolls it forward autoregressively.Each predicted state becomes input for the subsequent forecast step.
3 Method
FengWu combines modality-specific encoders and decoders with cross-modal Transformer fusion, uncertainty-based task weighting, and replay-buffer training for long-lead forecasts.
- 3 Method: FengWu has three main components: a Transformer-based modal-customized network, uncertainty loss for multi-task optimization, and a replay buffer.These components address representation learning, task weighting, and long-lead autoregressive training.
- 3.1 Network Architecture: Separate modality encoders extract weather-variable features, a cross-modal Transformer fuses them, and modality decoders separately predict future variables.The architecture treats weather variables as distinct modalities and uses joint representations for prediction.
- 3.2 Uncertainty Loss for Multi-task Optimization: Uncertainty loss automatically learns weights for weather-forecasting tasks by modeling each forecast probabilistically.FengWu predicts the mean and variance of a Gaussian distribution rather than only a point estimate.
- 3.2 Uncertainty Loss for Multi-task Optimization: Maximum likelihood estimation weights variables, pressure levels, and locations through homoscedastic uncertainty instead of manual tuning.The approach trades off tasks and locations within the loss objective.
- 3.3 Replay Buffer for Long-lead Predictions: Single-step training causes long-lead error accumulation during autoregressive inference, motivating a replay buffer for intermediate predictions.The buffer mixes original data with stored predictions during progressive long-lead training.
- 3.3 Replay Buffer for Long-lead Predictions: FengWu and GraphCast are compared using latitude-weighted RMSE and ACC across six-hour lead intervals over ten days.Lower RMSE and higher ACC indicate better forecast skill.
4 Results
FengWu is evaluated against GraphCast using latitude-weighted RMSE and ACC across 880 targets over 10-day forecasts. It shows stronger long-lead performance, while replay-buffer removal substantially degrades forecast skill as lead time increases.
- Evaluation Strategies: Latitude-weighted RMSE evaluates prediction error across spatial grids, with latitude weighting applied to account for geographic area differences.The evaluation uses channels, latitude-longitude grid indices, and valid test time slots.
- Evaluation Strategies: Latitude-weighted ACC compares predicted and observed anomalies while weighting latitudes to account for varying areas on a spherical Earth.The climatological mean is computed from daily ERA5 data averaged over 1993–2016; hourly climatology can produce higher ACC values.
- Quantitative Skill Evaluation: 80% of 880 targets show FengWu with lower RMSE and higher ACC than GraphCast over a 10-day forecast.At 1–5 days, FengWu is generally comparable to GraphCast except for t2m; improvements become more pronounced as lead time increases.
- Qualitative Prediction Evaluation: FengWu forecasts of z500 and t850 remain close to ERA5 at day 3, while absolute error increases and spreads to adjacent areas by later lead times.The visualizations cover lead days 3, 5, and 10.
- Effects of the Replay Buffer: Removing the replay buffer causes forecast performance to decrease significantly with increasing lead time.The ablation supports the replay buffer as a crucial component for improving long-lead prediction accuracy under autoregressive inference.
- Computation Cost: FengWu generates 10 days of six-hourly forecasts in less than 30 seconds on an NVIDIA Tesla-A100 GPU, with an estimated energy use roughly 2000 times below one IFS member.The FengWu estimate is roughly 12kJ versus about 26.6MJ for a single IFS member.
5 Conclusions and Discussions
FengWu combines multi-modal multi-task learning with a replay buffer and achieves strong global medium-range forecast performance. Comparisons with physics-based systems require caution because their initial fields differ in quality.
- 10.75 days is the extended skillful forecast lead time achieved by FengWu, with ACC of z500 > 60%.
- 80% is the share of reported prediction targets on which FengWu has higher accuracy than GraphCast.
- ERA5-based and physics-based forecasts use initial fields of different quality, complicating fairness in their comparisons.ERA5 benefits from a five-day analysis delay that allows more observations to be blended, whereas IFS-HRES uses timely initial analyses.
- 282 minutes is required by IFS for a 15-day, 51-member ensemble forecast on 1530 Cray XC40 nodes.A single 10-day IFS member roughly takes 98,400 node seconds.
- Removing the replay buffer causes forecast performance to decrease significantly as lead time increases.The experiment identifies the replay buffer as crucial for improving long-lead prediction accuracy.