Tropical Cyclone Attributes in Sparse Autoencoder Features of an AI Weather Model

• Cyclone location, intensity, and pressure structure each relate to multiple features, rather than mapping cleanly to one component.Location and pressure structure share some of those features.Source: Plain Language Summary · p. 2
• The overall organization reappears across independently trained decompositions and GraphCast processing layers, despite variation in the exact components.Source: Plain Language Summary · p. 2
• Deeper-layer position features match the cyclone’s state six hours later more closely for both location and pressure structure; earlier-layer features match the current state more closely.Source: Plain Language Summary · p. 2
• The recurring organization suggests a reproducible description of cyclone information in GraphCast.Source: Plain Language Summary · p. 2

1 Introduction

• Weather models forecast complex atmospheric evolution, but their high-dimensional hidden-state computations are not directly interpretable in physical terms.Relating event attributes to hidden states offers physical reference points for interpretation.Source: §1 · p. 3
• Input–output interpretability studies do not directly reveal how physical information is organized within model hidden states.Source: §1 · p. 3
• SAEs provide analyzable components for identifying and comparing signals associated with physical information in hidden states.Each learned direction defines a feature, whose weight gives its activation.Source: §1 · p. 3
• Prior studies identify cyclone-related SAE features, but leave unresolved how features organize relations to multiple physical attributes of the same event.Studies reported features tracking cyclone position and intervention-related forecast changes, and atmospheric-river intensity features persisting across depth.Source: §1 · p. 4
• Because independent SAEs may recover different features, the study asks whether multi-attribute organization recurs across decompositions and model depth.It also examines how feature correspondence to current and forecast states changes across depth.Source: §1 · p. 4
• The study uses tropical cyclones in GraphCast to test relations between learned SAE features and multiple measurable event attributes.Cyclone footprints provide spatial targets, and lifecycles allow recurrence to be tested through time.Source: §1 · p. 4
• The analysis tests recurrence over time, maps attributes across the Layer-8 dictionary, checks independent decompositions, and examines changes across depth.Source: §1 · p. 4

2.1 GraphCast and Sparse Autoencoders

• GraphCast’s 16-layer processor states are decomposed by TopK SAEs into 4,096 features with 32 active per node; Layer 8 anchors the analysis, which extends across layers.The cyclone analysis uses 2018–2019 states, after SAE training on 1979–2017 states.Source: §2.1 · p. 5

2.2 Cyclone and Pressure References

• Cyclone targets use reported centres and 34-kt wind radii, merging simultaneous footprints; across 2018–2019, records comprise 2,115 cyclone-valid six-hourly timestamps, 4,118 storm–time records, and 205 cyclones.MSLP is taken from ERA5; the deepest negative MSLP anomaly within a cyclone serves as a pressure-based intensity proxy, while the spatial anomaly field provides the structure reference.Source: §2.2 · pp. 5–6

2.3 Cyclone–Feature Correspondence Measures

• Position correspondence uses median F1, intensity uses signed deviations between normalized pressure and feature-response amplitudes, and pressure structure uses Spearman correlation with reversed pressure anomalies.The pressure-structure comparison uses a local ±12° region around each cyclone and summarizes storm-level correlations by their temporal median.Source: §2.3 · pp. 6–8

2.4 Quantifying Cyclone-Attribute Organization and Its Evolution Across Model Depth

• Intensity-readout improvement x_D is quantified in median absolute intensity deviation relative to a zero-activation readout, whose common physical-intensity reference over shared evaluation timestamps has error L0 ≃ 0.252.x_D = 1 indicates zero median absolute deviation.Source: §2.4 · p. 8
• Five independent Layer-8 SAEs and five seeds at each of 16 layers are analyzed with feature selection performed independently and no feature alignment across fits or layers.The analysis also compares each fixed contemporaneous F1-leading readout against physical references at t and t+6 h, and summarizes feature breadth and F1–pressure-structure overlap.Source: §2.4 · pp. 8–9

3.1 Temporal Recurrence and Variability of Cyclone Attribute–Feature Relations

• Across 2,115 timestamps, f613’s median position F1, signed D, and pressure-structure correlation were 0.512, 0.180, and 0.644, respectively.The measures varied over Dorian’s lifecycle; on 1 September 2019 at 18 UTC, F1 was 0.846 and pressure-structure correlation was 0.794.Source: §3.1 · pp. 9–10

3.2 Cyclone-Attribute Organization Across SAE Features

• Cyclone-attribute relations span multiple SAE features, while position and pressure-structure relations overlap in a more concentrated subset.The dictionary-wide F1–pressure-structure overlap weights were more concentrated than either individual attribute distribution.Source: §3.2 · pp. 10–11
• Across fits, effective feature numbers ranged from 3.08–3.94 for position F1, 2.57–3.97 for D-proximity, 1.16–2.88 for positive pressure-structure correlation, and 1.39–2.88 for F1–pressure-structure overlap.Feature participation in pressure-structure relations and their overlap ranged from strong concentration to moderate distribution across about 1–3 features. This recurring pattern reproduced the distributed but partially bundled organization across SAE decompositions despite changes in feature identity and ranking.Source: §3.2 · p. 11

3.3 Organization and Physical-Time Alignment Across Processor Depth

• Across all 16 layers, F1-leading features retained positive median pressure-structure correlation, and the distributed but partially bundled organization remained detectable.In the reference seed, each layer’s leader was among the top two features by correlation in 13 of 16 layers.Source: §3.3 · pp. 11–12
• From Layer 8 onward, the median ∆F1 favored position six hours ahead, and from Layer 9 onward, the median ∆ρ favored pressure structure six hours ahead.The median signed ∆D remained close to zero across layers.Source: §3.3 · p. 12

4 Discussion and Conclusions

• Because independent SAEs need not recover the same features, the analysis compares recurring organization rather than treating one feature as a fixed unit.The study reports that the organization recurs across fits despite differences in feature identity.Source: §4 · p. 13
• The analysis begins with predefined position, intensity, and pressure-structure attributes, so it examines a selected physical description of the event.A complementary approach would start from model-derived features or subspaces and ask which event dimensions emerge.Source: §4 · p. 13
• The analysis covers GraphCast and two years of single-step states from 205 cyclones; readout correspondence alone does not establish that forecasts use these features.Testing whether GraphCast relies on them would require intervention or ablation experiments.Source: §4 · p. 13
• Independent SAEs at each layer do not trace information transfer between layers, and SAE training choices may alter observed feature splitting and aggregation.Aligning features or subspaces across layers could help test how information is carried and updated.Source: §4 · p. 13
• In the cyclone case, physically meaningful attribute relations provide a reproducible target for interpreting event-related information in GraphCast hidden states.The organization recurs across independent SAEs and layers without requiring individual feature alignment.Source: §4 · pp. 12–14

Open Research

• GraphCast was used at commit a64f793, with source code and pretrained model resources available through the WeatherNext repository.ERA5 input fields and MSLP references came from WeatherBench 2.Source: Open Research · p. 14
• Tropical-cyclone tracks and wind-radius data came from IBTrACS v4.01.The passage identifies Gahtan et al. (2024) as the source.Source: Open Research · pp. 14–15
• SAE checkpoints, analysis code, and processed results are archived at Zenodo.The passage gives DOI 10.5281/zenodo.23492703.Source: Open Research · p. 15