Source-linked AI summary
TC-Next: Zero-Shot Multimodal Cyclone Forecasting
Zhe Wang, Sijie Chen, Yiming Luo, Daehyun Kim, Chien-Yi Chang
TL;DR
Foundation weather models reproduce tropical-cyclone tracks but struggle with intensity, while prior approaches do not provide zero-shot transfer across diverse forecast models using satellite observations. TC-Next combines weather-model forecasts with satellite imagery and, trained on GraphCast, transfers across forecast sources while outperforming rule-based tracking baselines.
Problem
Foundation models can reproduce cyclone tracks, but tracker-derived intensity is no better than persistence at short leads, and existing approaches lack zero-shot transfer across diverse forecast models while leveraging satellite observations.
Source: §1 · pp. 1–2Method
Results
Across forecast sources, TC-Next outperforms TempestExtremes; on GraphCast, track error is 15–44% lower and intensity error is 3.2–6.1 times lower.
Source: §3 · p. 4Takeaways & Limitations
Because it relies on generic atmospheric variables, the same trained TC-Next model can be applied across forecast sources without retraining.
Source: §1 · p. 2, §4 · p. 4For the WN-C comparison, IFS HRES analysis frames replace ERA5 history, and the TC Vitals block replaces IBTrACS anchors with real-time estimates.
Source: §2 · p. 2
Abstract
from arXiv · showhide
We present TropicalCycloneNext (TC-Next), a multimodal deep learning model that forecasts tropical cyclone track and intensity at $6$-$24$ h leads by leveraging a foundation model's forecast fields of atmospheric kinematic and thermodynamic fields and GridSat infrared satellite imagery. Trained only on GraphCast forecasts over the Western Pacific (WP), yet reliant only on generic atmospheric variables, TC-Next on GraphCast lowers track error by $15$-$44\%$ and intensity error by a factor of $3$-$6$ relative to a conventional, rule-based tracker, TempestExtremes; applied without retraining to the forecast fields of Pangu-Weather and IFS HRES, it stays ahead of TempestExtremes on both. Applied zero-shot to the generic weather fields of WeatherNext Cyclones on the 2025 WP season, TC-Next attains lower intensity error at every lead time, and lower or comparable track error, compared to that model's specialized direct tracker in a deterministic comparison. Our ablation studies show that our multimodal model is able to utilize the additional modality to improve performance in tracking errors at every lead time and in intensity prediction at longer lead times.
1 Introduction
Foundation weather models forecast atmospheric states well but struggle to provide accurate cyclone intensity through conventional trackers, and existing approaches lack zero-shot transfer across diverse forecast models while using satellite observations. TC-Next combines forecast fields with satellite imagery and reports transfer performance across forecast sources.
Motivation
- Foundation models reproduce cyclone tracks well, but tracker-derived intensity is no better than persistence at short leads, and ERA5 intensity error is 12–15 m/s against best-track data.The passage attributes ERA5’s intensity error to resolution constraints and assimilation processes.Source: §1 · p. 1
Contributions
- Trained solely on GraphCast, TC-Next lowers track error by 15–49% and intensity error by a factor of 2–6 versus TempestExtremes across multiple models, and outperforms WN-C’s direct tracker.The reported WN-C comparison is deterministic.Source: §1 · p. 2
Approach
- TC-Next combines foundation-model forecast fields with high-resolution GridSat imagery of the storm’s recent history to predict track and intensity.The atmospheric fields provide large-scale steering flow and thermodynamic conditions; imagery provides storm convective structures.Source: §1 · p. 2
Multimodal efficacy
- Satellite infrared ablations lower track error at every lead and improve intensity error from +18 h onward, with the reported benefit peaking at +24 h.The benefit increases with lead time.Source: §1 · p. 2
Cross-model portability
- TC-Next requires only generic atmospheric variables from forecast sources, supporting use of the same trained tracker across sources.The passage notes that SST is persisted from the analysis.Source: §1 · p. 2
2 Methodology
TC-Next forecasts cyclone track, intensity, and extent from historical storm observations, atmospheric fields, satellite imagery, and future weather-model forecasts. Its evaluation uses chronologically partitioned Western Pacific data and defines zero-shot transfer as changing the forecast source without retraining, subject to stated comparison-specific input substitutions.
Problem formulation
- The model forecasts track and intensity at 6, 12, 18, and 24 h from observed history, predicting center, extent, and intensity updates at each lead.The motion update is bounded by δ = 4° per axis; the extent update adjusts an auxiliary storm box.Source: §2 · p. 2
Data
- Training pairs ERA5 storm history and GraphCast forecasts with GridSat-B1 infrared imagery and IBTrACS storm state, using a Western Pacific-centered atmospheric domain and storm-centered imagery windows.Storm intensity is the 1-minute maximum sustained wind (USA_WIND).Source: §2 · p. 2
Zero-shot evaluation
- Zero-shot transfer replaces GraphCast with an unseen forecast source at inference without retraining or changing other inputs, with specified input substitutions for the WN-C comparison.That comparison uses IFS HRES analysis frames instead of ERA5 history; its TC Vitals block also replaces IBTrACS anchors with real-time estimates.Source: §2 · p. 2
Data and evaluation
- Chronological partitions use 1990–2017 for training, 2018 for validation, and 2019–2020 for testing; a separate 2025 test set evaluates against WN-C.The 2025 test set contains 867 samples from 29 storms.Source: §2 · p. 3
Model
- TC-Next encodes atmospheric fields and satellite imagery separately, fuses geographically aligned storm-centered features, then uses recurrent encoder and decoder LSTMs to predict updates.A region-proposal network predicts storm-centered boxes, and the decoder rolls forward autoregressively over forecast frames.Source: §2 · p. 3
3 Experiments and Results
TC-Next is evaluated against WeatherNext Cyclones’ direct tracker and TempestExtremes, and with its satellite infrared branch ablated. It transfers across forecast sources and its infrared branch improves track errors at every lead and intensity errors at longer leads.
Evaluation settings
- The evaluation compares TC-Next with WeatherNext Cyclones’ direct tracker on the 2025 WP season, with TempestExtremes on pooled WP 2019–2020 seasons, and with an IR-ablated variant.TC-Next is trained solely on GraphCast fields; Pangu-Weather, IFS HRES, and WeatherNext Cyclones are evaluated without retraining.Source: §3 · pp. 3–4
WeatherNext Cyclones comparison
- On the 2025 WP season, TC-Next reads generic weather channels and outperforms WeatherNext Cyclones’ direct tracker’s single-member predictions.Against the IBTrACS anchor convention, track error is lower at every lead and early-lead intensity error is roughly halved; with TC Vitals anchors, track performance is on par and intensity error is 3.94–7.29 kt versus 7.95–9.49 kt.Source: §3 · pp. 3–4
TC-Next vs. TempestExtremes
- TC-Next outperforms TempestExtremes across forecast sources and lead times; on GraphCast, track error is 15–44% lower and intensity error is 3.2–6.1 times lower.Without retraining, track error is 18–45% lower on Pangu-Weather and 22–49% lower on IFS HRES; TC-Next intensity error stays below 11 kt on every source and lead, while TempestExtremes’ never falls below 15 kt.Source: §3 · p. 4, Table 2 · p. 4
IR ablation
- The satellite infrared branch lowers track error at every lead and intensity error from +18 h onward, with the benefit increasing with lead time.The fed variant’s track-error advantage grows from 0.7 km at +6 h to 2.2 km at +24 h; the IR-ablated model is separately trained with the IR input zeroed in training and evaluation.Source: §3 · p. 4, Table 3 · p. 4
4 Conclusion
TC-Next predicts tropical cyclone track and intensity from foundation-model forecast fields and high-resolution satellite imagery. Its generic atmospheric inputs support application across forecast sources without retraining, and the authors identify broader basin coverage as ongoing work.
Conclusion
- TC-Next predicts tropical cyclone track and intensity using foundation-model forecast fields and high-resolution satellite imagery of the storm’s history.Because the forecast input uses only generic atmospheric variables, the model can be applied across forecast sources without retraining; ongoing work focuses on expanding the dataset and benchmark to additional basins.Source: §4 · p. 4
A Data Details
The dataset combines ERA5 history, GraphCast forecasts, GridSat-B1 infrared imagery, and IBTrACS storm state. It uses a fixed Western Pacific-centered geographic view, with held-out 2019–2020 test seasons and selected atmospheric channels at three pressure levels.
Data sources and preprocessing
- The dataset pairs ERA5 history frames with GraphCast forecasts on the same 0.25° grid and variables, without regridding, and adds GridSat-B1 infrared-window imagery at 00, 06, 12, and 18 UTC.GridSat-B1’s native 0.07° imagery is bilinearly resampled; missing pixels and frames are zero-filled and accompanied by a per-pixel validity mask supplied as a second input channel.Source: A Data Details · p. 8
Storm-state targets
- Storm positions use fractional row-column coordinates on a zero-padded grid, while intensity targets use 1-minute maximum sustained wind.The extent target is a square centered on the best-track position, sized from the mean of four reported 34 kt wind radii and clipped to 2°–10°; absent radii default to 4°.Source: A Data Details · p. 8
Storage
- The atmospheric cache stores five frames of 280×420×12 fields per initialization time, while infrared frames are stored separately at 1120×1680 resolution.The cache totals 20,621 files and 271 GiB for 1990–2020; infrared imagery totals 34.7 GiB for 1990–2020.Source: A Data Details · p. 8
Dataset coverage
- The fixed geographic view spans 5°S–65°N and 95°E–160°W; the pooled test set contains 1,815 samples from 57 storms across 2019 and 2020.The view is not a basin mask and includes 1,108 training samples, mostly East Pacific, from storms IBTrACS assigns to adjacent basins.Source: Table 4 · p. 8
Atmospheric channels
- The atmospheric inputs use 850, 500, and 200 hPa levels; sea-surface temperature comes from ERA5 at initialization because GraphCast does not forecast it.The selected levels represent low-level circulation and moisture, mid-tropospheric flow and core ascent, and upper-level outflow; 850 and 200 hPa together yield deep-layer vertical wind shear.Source: Table 5 · p. 9
B Model and Training Details
TC-Next combines atmospheric forecasts and fine-scale satellite imagery through a recurrent encoder-decoder. Its training jointly supervises storm-box localization and intensity prediction.
Model and Training Details
- The macro CNN encodes atmospheric frames at 2° resolution, while a finer-stride micro CNN maps storm-centered GridSat imagery to a 0.25° feature grid.RoIAlign samples both feature maps around the proposed storm center.Source: B Model and Training Details · p. 9
- Low-rank fusion combines the environment and infrared streams while retaining unimodal terms alongside their element-wise products.Per-stream normalization is used because, without it, the larger environment signal can starve the infrared branch's gradient.Source: B Model and Training Details · p. 9
- The decoder uses forecast-environment crops at each lead and the previous predicted center, while historical infrared imagery enters through the encoder state.Separate box and intensity heads predict center, extent, and intensity updates; no future infrared imagery is used.Source: B Model and Training Details · p. 9
- Training scores forecast boxes and historical proposals with a box loss, and scores maximum sustained wind with an intensity loss.The extent target is the best-track gale-force wind radius R34, using a fixed 4° where unreported; the intensity weight reaches 2.5 after one epoch.Source: B Model and Training Details · pp. 9–10
C Experimental Setup Details
The experiments compare TC-Next with TempestExtremes and the WN-C direct tracker across forecast sources and evaluation settings. Scoring is restricted to matched samples, with additional differences in sample availability and intensity measurement.
Experimental Setup Details
- TempestExtremes requires a closed low and warm core, merges nearby candidates, and returns forecasts for 54–70% of samples; comparisons use only samples both methods produce.Its intensity uses minimum sea-level pressure and maximum 10 m wind within 2.0°, unlike the best-track 1-minute sustained wind.Source: C Experimental Setup Details · p. 10
- TC-Next is trained only on GraphCast fields, then evaluated without retraining on Pangu-Weather, IFS HRES, and WN-C; Pangu-Weather’s missing ω500 channel is zero-filled.For WN-C comparisons, HRES analyses replace ERA5 history; the TC Vitals configuration also replaces IBTrACS anchors with real-time estimates.Source: C Experimental Setup Details · p. 10
- Table 2 pools 2019–2020 matched samples at 00:00 and 12:00 UTC; intensity scoring further excludes samples without reported USA_WIND.For Table 1, samples where WN-C predicts dissipation are dropped for both models; WN-C ensemble members are scored separately before averaging errors.Source: C Experimental Setup Details · pp. 10–11
- Tables 6 and 7 count matched samples by lead, while Table 8 counts all held-out samples; Table 8 and Table 7 are not directly comparable.Intensity counts are lower because they also require reported USA_WIND.Source: Table 8 · p. 11, Table 7 · p. 11, Table 6 · p. 11