Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecast

Kaifeng Bi; Lingxi Xie; Hengheng Zhang; Xin Chen; Xiaotao Gu; Qi Tian

Source-linked AI summary

Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecast

Kaifeng Bi, Lingxi Xie, Hengheng Zhang, Xin Chen, Xiaotao Gu, Qi Tian

arXiv:2211.02556v1Nov 3, 2022physics.ao-phcs.AIcs.CVcs.LG

TL;DR

Global medium-range weather forecasting must balance accuracy, resolution, and computational speed, while AI methods had not previously matched operational NWP accuracy at comparable resolution. Pangu-Weather trains deep networks on ERA5 data using a 3D Earth-specific transformer and hierarchical temporal aggregation, and reports higher accuracy than operational NWP across factors and forecast times, with fast inference. The system also supports extreme-weather and ensemble forecasting, although quantitative MSLP comparison was unavailable and cyclone intensity remains underestimated.

Problem
AI-based weather methods were faster and had reached 0.25°×0.25° resolution, but their medium-range forecast accuracy remained below operational IFS.
Method
Pangu-Weather trains deep neural networks on ERA5 data using a 3D Earth-specific transformer and hierarchical temporal aggregation.
Results
Pangu-Weather surpasses operational IFS in accuracy across all weather factors and forecast times from one hour to one week, with 1,400 ms inference on a single GPU.
Takeaways & Limitations
Fast inference enables Pangu-Weather to support downstream scenarios including tropical-cyclone tracking and large-member ensemble forecasting.
Takeaways & Limitations
Quantitative comparison for mean sea-level pressure was unavailable, and Pangu-Weather heavily underestimates tropical-cyclone intensity.

Abstract

from arXiv · show

In this paper, we present Pangu-Weather, a deep learning based system for fast and accurate global weather forecast. For this purpose, we establish a data-driven environment by downloading $43$ years of hourly global weather data from the 5th generation of ECMWF reanalysis (ERA5) data and train a few deep neural networks with about $256$ million parameters in total. The spatial resolution of forecast is $0.25^\circ\times0.25^\circ$, comparable to the ECMWF Integrated Forecast Systems (IFS). More importantly, for the first time, an AI-based method outperforms state-of-the-art numerical weather prediction (NWP) methods in terms of accuracy (latitude-weighted RMSE and ACC) of all factors (e.g., geopotential, specific humidity, wind speed, temperature, etc.) and in all time ranges (from one hour to one week). There are two key strategies to improve the prediction accuracy: (i) designing a 3D Earth Specific Transformer (3DEST) architecture that formulates the height (pressure level) information into cubic data, and (ii) applying a hierarchical temporal aggregation algorithm to alleviate cumulative forecast errors. In deterministic forecast, Pangu-Weather shows great advantages for short to medium-range forecast (i.e., forecast time ranges from one hour to one week). Pangu-Weather supports a wide range of downstream forecast scenarios, including extreme weather forecast (e.g., tropical cyclone tracking) and large-member ensemble forecast in real-time. Pangu-Weather not only ends the debate on whether AI-based methods can surpass conventional NWP methods, but also reveals novel directions for improving deep learning weather forecast systems.

1 INTRODUCTION

Pangu-Weather addresses the speed–accuracy tradeoff in global weather forecasting with a deep learning system designed for high-resolution deterministic and downstream forecasts. It combines 3D atmospheric modeling and hierarchical temporal aggregation, and reports gains over operational IFS across weather factors and forecast times.

Motivation: Conventional NWP offers accurate forecasts but incurs high computational costs, motivating data-driven alternatives that can trade off model complexity, resolution, and accuracy.FourCastNet reached 0.25°×0.25° resolution and much faster inference, but its forecast accuracy remained below operational IFS.
System: Pangu-Weather uses 43 years of ERA5 global weather data, with separate training, validation, and testing periods.The selected inputs include 13 pressure levels with five variables each and four surface variables.
Results: Pangu-Weather reports 296.7 RMSE for a single-member 5-day Z500 forecast, versus 333.7 for operational IFS and 462.5 for FourCastNet.Its inference cost is 1,400 ms on a single GPU, and the paper also reports high-resolution maps and tropical-cyclone tracking.
Results: Pangu-Weather surpasses operational IFS across all weather factors and forecast times from one hour to one week at 0.25°×0.25° resolution.The paper presents this as the first such result for an AI-based global weather forecast system.
Technical contributions: Its two key accuracy strategies are a 3D Earth-specific transformer that integrates height information and hierarchical temporal aggregation that alleviates cumulative forecast errors.The temporal strategy trains models for increasing lead times, reducing the iterations needed for medium-range forecasts.
Applications: The system supports extreme-weather and large-member ensemble forecasting while preserving timeliness through fast inference.The paper identifies these as downstream scenarios enabled by transferring deterministic forecasting capability.

2 PRELIMINARIES AND INSIGHTS

The paper frames weather forecasting as learning or approximating a high-dimensional spatiotemporal function from reanalysis data. It identifies pressure-level structure and iterative error accumulation as central challenges for medium-range AI forecasting, motivating a higher-dimensional, fewer-iteration design.

2.1 Problem Setting and Notations: ERA5 weather data vary across latitude, longitude, pressure level, time, and multiple weather factors, creating a large high-dimensional forecasting problem.The paper denotes the full weather state by A and formulates forecasting as applying a function to historical data.
2.2 NWP Methods: Conventional NWP approximates the forecasting function through numerical simulation of physical processes represented by partial differential equations on discrete grids.Finer grids improve spatial resolution but increase simulation costs.
2.2 NWP Methods: NWP methods historically led in prediction accuracy, but their computational overhead limits timeliness and the number of forecasts that can be produced.The paper attributes the burden to super-linearly increasing computation and difficult parallelization as observation data grow.
2.3 AI-based Methods: AI-based methods learn the forecasting function from abundant training data without requiring the underlying physical procedure or formulae.Deep neural networks are used as the learned function in this data-driven formulation.
2.3 AI-based Methods: Before Pangu-Weather, AI methods had achieved faster inference but generally remained inferior to operational IFS in medium-range accuracy at comparable resolution.FourCastNet reached 0.25°×0.25° resolution, yet its RMSE and ACC remained below operational IFS.
2.4 Insights: Existing AI approaches faced two main issues: 2D representations poorly handled pressure-level relationships, and repeated calls accumulated medium-range forecast errors.FourCastNet required 28 iterations for a 7-day forecast from a 6-hour base model.
2.4 Insights: The resulting design insight is to increase data dimensionality while reducing the number of forecast iterations, subject to the computational cost of large weather data.Pangu-Weather follows this tradeoff by using latitude, longitude, and height as spatial dimensions.

3 METHODOLOGY

Pangu-Weather combines deep neural networks with a 3D Earth-specific transformer and hierarchical temporal aggregation for fast global weather forecasting. Its architecture represents atmospheric height explicitly and uses Earth-specific attention design choices to address geometry and accumulated errors.

Overview: Pangu-Weather is an AI-based global weather forecast system designed for fast and accurate prediction.
Temporal Aggregation: Hierarchical temporal aggregation trains models with different lead times to alleviate cumulative errors from repeated short-lead forecasts.
Data Preparation and the Pre-training Task: The pre-training task maps historical weather observations at time t to predicted atmospheric states at t+∆t.
3D Earth-Specific Transformer: The model organizes upper-air pressure-level information into 3D weather data and processes it with a 3D Earth-specific transformer.
3D Earth-Specific Transformer: 3DEST uses windowed and shifted-window attention within an encoder-decoder architecture to reduce computational costs while processing 3D weather states.
3D Earth-Specific Transformer: Earth-specific positional bias assigns bias from absolute Earth coordinates to reflect uneven map spacing and weather relationships with latitude and pressure-level height.
Design Choices: The design uses fewer encoder and decoder layers because of computational overhead, while more time points could improve accuracy but exceed the available budget.

4 RESULTS

Pangu-Weather is evaluated for global deterministic forecasting and tropical-cyclone tracking, with comparisons against leading numerical and AI-based systems. The results section therefore covers both broad weather prediction and an extreme-weather scenario.

Evaluation Setup: The evaluation uses held-out ERA5 data for global deterministic forecasting and IBTrACS data for tropical-cyclone tracking.
Baselines: Pangu-Weather is compared with operational IFS and FourCastNet, while tropical-cyclone tracking also includes ECMWF-HRES.

4.1 Deterministic Forecast

Pangu-Weather provides 0.25°×0.25° deterministic forecasts at hourly spacing, with accuracy advantages over operational IFS across variables and forecast times. Its forecasts remain close to ERA5 and support fast inference, while iterative forecasting can accumulate errors.

0.25°×0.25° resolution and 1-hour spacing enable hour-by-hour deterministic forecasts comparable in resolution to ECMWF ENS and finer temporal spacing than FourCastNet.Forecasts use unperturbed ERA5 initial states.
Upper-air Atmospheric Variables: Pangu-Weather consistently outperforms operational IFS across forecast times and variables, with advantages becoming larger as forecast time increases.The reported forecast-time gain exceeds 12 hours for all variables and 24 hours for specific humidity.
Upper-air Atmospheric Variables: 134.5 and 296.7 m2/s2 are Pangu-Weather’s 3-day and 5-day Z500 RMSEs, versus 152.8 and 333.7 for operational IFS.For T850, Pangu-Weather reports 1.14 and 1.79 K versus 1.37 and 2.06 K for operational IFS; relative RMSE reductions exceed 10% in the reported scenarios.
Surface Weather Variables: 1.05 and 1.53 K are Pangu-Weather’s 3-day and 5-day T2M RMSEs, versus 1.34 and 1.75 for operational IFS and 1.39 and 2.00 for FourCastNet.For U10, the corresponding Pangu-Weather RMSEs are 1.61 and 2.53 m/s.
Diagnostic Studies: 72-hour forecasts can require 3 model calls, whereas a 71-hour forecast may require 8, causing lower accuracy through cumulative forecast errors.The paper identifies advanced temporal aggregation as a future direction and notes that 72-hour accuracy can be improved by shifting the forecast time point back.
1,400ms is Pangu-Weather’s single-GPU inference cost, more than 10000× faster than operational IFS and comparable to FourCastNet.System-level comparison reports 1,400ms on a Tesla-V100 for Pangu-Weather versus 280ms on a Tesla-A100 for FourCastNet.

4.2 Results on Extreme Weather Events

Pangu-Weather is evaluated on extreme-weather prediction through quantile-based intensity analysis and tropical-cyclone eye tracking. It generally underestimates extreme intensity, but achieves substantially more accurate cyclone tracks than ECMWF-HRES.

Evaluation setup: Extreme-weather evaluation combines relative quantile error analysis with tropical-cyclone eye tracking against operational IFS.The tracking evaluation uses IBTrACS as ground truth and ECMWF-HRES as the comparison system.
Extreme intensity: All three compared systems tend to underestimate extremes, and AI-based methods generally show increasing underestimation as forecast time grows.Operational IFS maintains the smallest absolute RQE values in the reported comparisons.
Tropical-cyclone tracking: 120.29km and 195.65km are Pangu-Weather’s 3-day and 5-day mean tropical-cyclone position errors, versus 162.28km and 272.10km for ECMWF-HRES.The comparison averages cyclone-eye position errors over the TC2018 dataset.
Case studies: Pangu-Weather correctly tracked Kong-rey’s path, while ECMWF-HRES forecast an incorrect landfall in China and corrected direction more than 48 hours later.The Pangu-Weather forecast nearly coincided with ground truth and changed little over time.
Case studies: For Hurricane Michael and Typhoon Ma-on, Pangu-Weather predicted landfall timing or location more accurately than ECMWF-HRES.Michael’s predicted landing delay was 3 hours versus 18 hours, while Ma-on’s forecast was approximately correct when ECMWF-HRES predicted the wrong Chinese landfall.
Interpretation and limitation: Cyclone tracking benefits from deterministic forecasts and intermediate variables, especially mean sea-level pressure, but tropical-cyclone intensity remains heavily underestimated.The authors attribute the intensity limitation partly to ERA5’s resolution and anticipate higher-resolution training data for future refinement.

4.3 Ensemble Forecast

Pangu-Weather enables large-member ensemble forecasting by making repeated inference computationally inexpensive. The 100-member ensemble is less accurate at short range but becomes significantly better than deterministic forecasting beyond five days.

Motivation: Ensemble forecasting probes forecast uncertainty by perturbing initial states or model parameters and observing changes in forecast results.These approaches require multiple inference runs.
Motivation: Pangu-Weather’s inference speed enables large-member ensembles at relatively low computational cost.The paper reports inference more than 10,000× faster than operational IFS.
Method: The study perturbs initial weather states with random Perlin noise and averages the resulting model outputs into the ensemble forecast.The perturbation amplitude is controlled by η = 0.2, with one unperturbed member included.
Results: A 100-member ensemble is slightly worse than deterministic forecasting at short range but significantly better when forecast time exceeds five days.This pattern indicates greater ensemble value as single-model accuracy declines, while added noise can hurt when deterministic forecasts are already accurate.
Results: Ensemble forecasting provides greater benefits for non-smooth variables such as Q500 and U10.The reported comparison uses latitude-weighted RMSE for long-range forecasts.

5 CONCLUSIONS AND FUTURE REMARKS

Pangu-Weather combines 3DEST with hierarchical temporal aggregation to deliver fast, accurate weather forecasts. The authors conclude that its efficiency supports meteorological integration while substantial computational and modeling headroom remains.

Conclusions: Pangu-Weather uses a 3D Earth-specific transformer and hierarchical temporal aggregation as its two main technical contributions.The system is trained on 39 years of global weather data and surpasses conventional NWP in both accuracy and speed.
Conclusions: Efficient inference creates opportunities for meteorologists to integrate domain knowledge into AI-based weather forecasting.The paper presents this as a direction for further applications rather than a completed integration.
Future remarks: The training procedure has not reached full convergence, leaving room to add observation factors, model the time dimension with 4D networks, and use deeper or wider networks.These improvements require GPUs with greater memory and computational capacity.