Forecasting the Future with Yesterday's Climate: Temperature Bias in AI Weather and Climate Models

• Historical training data may not represent future conditions, creating a challenge for AI models used in a changing climate.Source: §1 · p. 3
• Prior work found generally colder biases when AI weather models were evaluated in modern and future climates, especially for record-breaking events.Source: §1 · p. 3
• The study evaluates FourCastNet, Pangu, and ACE2 on temperature periods outside their training sets and warmer than their training climatologies.Source: §1 · p. 3
• The analysis asks whether predictions reflect the evaluation-period climate or remain influenced by the models’ training-data climate.Source: §1 · p. 3

2.1 FourCastNet and Pangu

• FourCastNet and Pangu provide 2m temperature forecasts at 2-day and 9-day leads for December 2020–February 2025.Their training data are centered around 1997–1998, making the evaluation period roughly 25 years more modern.Source: §2.1 · pp. 3–4

2.2 ACE2

• ACE2 generates a five-member ensemble of daily surface temperatures and is evaluated over 1996–2010, a period roughly 25–30 years more modern than its average training year.ACE2 includes explicit CO2 forcing and uses an SFNO architecture adapted for long-term climate simulation.Source: §2.2 · p. 4

2.3 Temperature Comparison

• The analysis defines boreal winter land temperatures as DJF values over land excluding Antarctica and Greenland.Source: §2.3 · p. 4
• Model temperatures are compared with ERA5 at model-compatible resolutions, using gridpoint mean differences, cosine-latitude-weighted global means, and two-sided Z-tests.Effective sample size accounts for lag-1 autocorrelation, with significance assessed at α = 0.1.Source: §2.3 · p. 4

3.1 Weather Models

• FourCastNet and Pangu show cold bias across lead times, with FourCastNet colder than Pangu in both 2-day and 9-day forecasts.The reported global mean differences are −0.35 K and −0.45 K for FourCastNet versus −0.26 K and −0.07 K for Pangu at 2-day and 9-day leads.Source: §3.1 · p. 5, Figure 1 · p. 5
• All three models produce cold-biased boreal winter temperatures resembling climates from 15–20 years earlier, with Eastern U.S. lags reaching roughly 30 years.FourCastNet and Pangu predictions for 2020–2025 are colder than ERA5, while ACE2 has a global mean bias of −0.35 K.Source: §3.1 · p. 5, §3.3 · p. 8
• A 9-day persistence forecast matches the 2020–2025 mean climate more closely than either AI weather model.Source: §3.1 · pp. 5–6, Figure 2 · p. 6

3.2 Extreme Modern Temperatures

• The hottest 10% of FourCastNet and Pangu forecasts are colder than ERA5 by 0.91 K and 0.34 K, whereas their cold tails show minimal or positive bias.This asymmetry corresponds to substantially more training data at or below the modern cold tail than at or above the modern hot tail.Source: §3.2 · pp. 7–8, Figure 3 · p. 7

3.3 Climate Models

• ACE2’s cold bias is strongest in the cold tail, unlike the weather models’ hot-tail bias.ACE2 is particularly cold over North America, Europe, and Russia, and its global mean bias represents 35% of the average RMSE against ERA5.Source: §3.3 · p. 8, Figure 4 · p. 8
• ACE2’s bias is largest where winter warming, cold-tail warming, and climate-change trends have been most pronounced, while summer bias is lower.The spatial, seasonal, and distributional pattern links ACE2’s bias to asymmetric warming in its training climate.Source: §3.3 · p. 9, Figure 5 · p. 9

4 Discussion and Conclusions

• Both AI weather and climate models produce modern boreal-winter temperatures resembling climates from 15–20 years earlier, consistent with anchoring toward older training-data climates.Their training data are centered approximately 25–30 years before the prediction periods.Source: §4 · pp. 9–10
• FourCastNet and Pangu are coldest relative to ERA5 in their hottest temperature predictions, consistent with limited training exposure to modern extreme heat events.ACE2 instead has stronger cold bias in the coldest predictions, associated with warming patterns in its training set.Source: §4 · p. 10
• ACE2’s spatial, seasonal, and distributional bias patterns align with regions of rapid historical warming, while its CO2 forcing does not fully eliminate training-set artifacts.The paper reports a stronger winter than summer bias and alignment with rapidly warming regions.Source: §4 · p. 10
• These present-day biases demonstrate limitations in extrapolating beyond training domains and motivate climate-robust AI models for present and future prediction.Proposed mitigation strategies include future-extending climate-model data and climate-invariant inputs.Source: §4 · p. 10

Appendix A Missing Data

• FourCastNet lacks three initialization dates and Pangu lacks four, representing 0.67% and 0.89% of the ERA5 data used, respectively.The passage lists the excluded dates and the corresponding affected forecast leads.Source: Appendix A · p. 10

Open Research Section

• ERA5, FourCastNet, Pangu, and ACE2 resources are identified through their respective publications, DOI records, and data repositories.ACE code and initialization files are available through the cited GitHub repository, with a permanent Zenodo DOI planned upon publication.Source: Open Research Section · p. 11

Geophysical Research Letters

• At 1-day lead, FourCastNet and Pangu already show global cold bias, with bias generally increasing for FourCastNet but decreasing for Pangu as lead time grows.Longer leads can still contain substantial offsetting regional bias.Source: Figure S1 · p. 16
• Figure S2 compares the percentages of FourCastNet and Pangu training data at least as extreme as the 10th and 90th percentiles of 2020–2025 ERA5 temperatures.Global means are shown for each panel and correspond to the summary in Figure 3f.Source: Figure S2 · p. 17
• Across additional FourCastNet percentiles, the hottest predictions exhibit stronger cold bias than the coldest predictions, consistent with less training data for hot extremes.Stippling marks grid points where selected biases are significantly non-zero.Source: Figure S3 · p. 18
• Pangu shows the same qualitative percentile-dependent pattern as FourCastNet, with stronger cold bias in hotter predicted percentiles and less training data for hot extremes.The figure uses 5th, 95th, 20th, and 80th percentile analyses analogous to the FourCastNet supplement.Source: Figure S4 · p. 19
• The supplementary comparison finds little global-mean warming difference between the 10th and 90th percentiles across the split weather-model training periods.This contrasts with the stronger percentile-dependent change identified for ACE2’s training set.Source: Figure S5 · p. 20