WxFM-XL: Adapting Univariate Foundation Models to Multi-Station Weather Forecasting

1 Introduction

• Univariate foundation models can forecast strongly with limited fine-tuning, but their univariate design leaves multi-station spatiotemporal relationships under-modeled.The models use temporal knowledge learned from large-scale, cross-domain data and can sometimes forecast in zero-shot settings.Source: §1 · p. 1
• Multivariate extensions often treat stations as generic variables, overlooking both geographic relationships and station-specific error correlations with the foundation model.An anchor station’s errors correlate strongly with distant coastal stations, beyond simple neighborhood relationships.Source: §1 · p. 1
• Spatial relationships are static, while foundation-model error patterns vary by model and frequency, making their joint integration challenging.The paper frames the task as integrating static station relationships with dynamic error priors.Source: §1 · p. 1
• WxFM-XL decomposes forecasts by frequency, builds band-specific error graphs, and fuses them dynamically with spatial relations for residual correction.Experiments across multiple datasets show that WxFM-XL outperforms state-of-the-art baselines.Source: §1 · pp. 1–2

2 Related Work

• Time-series foundation models transfer temporal knowledge from large cross-domain corpora, but most focus on univariate forecasting, limiting performance in multi-station weather settings.Examples include Timer [9], Sundial [11], Chronos [1], and TimesFM [5], with later work addressing scalability and generative capability.Source: §2 · p. 2
• Task-specific multi-station models represent station interactions, but generally do not directly exploit temporal priors from foundation-model pretraining.Examples include graph-based approaches such as StemGNN [4] and MSGNet [3], and long-range models such as Autoformer [15] and FEDformer [19].Source: §2 · pp. 2–3
• Multivariate foundation-model extensions such as Timer-XL [10] and AdaPTS [2] often treat stations as generic variables, without explicit spatial modeling or stationwise error-prior correlations.Moirai [14] supports heterogeneous frequencies and arbitrary numbers of variables; Timer-XL [10] and AdaPTS [2] use different multivariate adaptation strategies.Source: §2 · p. 3

3 Problem Formulation

• The forecasting task uses N geographically located stations, each with a fixed location represented by longitude, latitude, and elevation.The location collection is denoted L ∈ R^{N×3}.Source: §3 · p. 3
• Given observations from the past Tin timeframes at all stations, the model forecasts the target variable for the next Tout timeframes.The formulation can be extended to multiple weather variables by applying the same procedure to each variable.Source: §3 · p. 3
• The approach applies a univariate foundation model independently at each station, then seeks to improve those forecasts with dynamic fusion and a graph residual adapter.The independent stationwise forecasts are the starting point for the proposed cross-station refinement.Source: §3 · p. 3

4 The WxFM-XL Framework

• WxFM-XL has two components: a Dynamic Fusion Mechanism that combines spatial and error-prior graphs, and a Graph Residual Adapter that refines forecasts using the resulting graph.The adapter is designed to preserve the foundation model’s temporal priors.Source: §4 · p. 3

4.1 Dynamic Fusion Mechanism

• Forecast decomposition separates foundation-model outputs into low-, mid-, and high-frequency components for band-specific graph construction.The corresponding Fourier coefficients are retained and reconstructed in the time domain before constructing frequency-dependent error graphs.Source: §4.1 · p. 3
• For each frequency band, the error-prior graph uses correlations between stationwise forecast errors, while the spatial graph is built from fixed station locations.Spatial adjacency uses a Gaussian kernel; each graph is sparsified and symmetrized according to the described construction.Source: §4.1 · p. 4
• A learnable band-specific gate fuses the static spatial graph with dynamic error-prior graphs, balancing geographic constraints with nonlocal error relationships.The resulting graph guides residual refinement for the corresponding frequency component.Source: §4.1 · pp. 4–5

4.2 Graph Residual Adapter

• The Graph Residual Adapter embeds forecast patches at multiple temporal scales, propagates representations across stations, then refines them temporally and fuses the reconstructed residuals.A node gate adapts graph propagation weights, and sample-dependent weights combine residuals from different patch sizes.Source: §4.2 · p. 5
• The foundation model stays frozen while adapter parameters are jointly optimized using final-forecast MSE and energy-normalized RMSE across frequency bands.Energy normalization is used to avoid low-energy frequency components being overwhelmed during training.Source: §4.2 · pp. 5–6

5.1 Experimental Settings

• Evaluation uses three hourly weather-station datasets, chronological 6:1:3 train-validation-test splits, 96-hour input windows, and 24- or 96-hour forecast horizons.The datasets are MeteoNet, Hunan, and a global dataset; WxFM-XL uses three frequency bands and patch sizes {6, 12, 24}.Source: §5.1 · pp. 6–7

5.3 Ablation Study

• Removing frequency decomposition, error correlation priors, or the multi-scale temporal module worsens reported MSEs, supporting all three components.For example, removing frequency decomposition raises Hunan temperature MSE from 5.143 to 5.393; removing error priors raises v-wind MSE from 2.450 to 2.534 on Hunan and 4.501 to 4.672 on French. Figure 3 reports that patch sizes {6, 12, 24} achieve the lowest MSE and MAE across the three French regional variables.Source: Table 2 · p. 7, Figure 3 · p. 7, §5.3 · p. 8

5.2 Result Analysis

• Across all 16 dataset-variable-horizon settings, WxFM-XL (Timer) ranks first in nine and WxFM-XL (Sundial) in seven, with both outperforming their frozen backbones.The settings span regions, variables, and forecasting horizons.Source: §5.2 · p. 7
• On four French wind forecasting settings, the best WxFM-XL variant reduces MSE by 8.3%–12.6% relative to the strongest task-specific baseline.WxFM-XL generally performs better than STELLA, CDPNet, MSGNet, and Corrformer.Source: §5.2 · p. 7
• Fine-tuning improves average Timer-XL and Moirai performance but not uniformly; AdaPTS also generally underperforms WxFM-XL.Timer-XL fine-tuning raises French 24-hour v-wind MSE from 5.66 to 6.01 and Hunan 96-hour temperature MSE from 7.37 to 9.29.Source: §5.2 · p. 7

5.4 Visualization Study

• Error-prior graphs have heavier edge-distance tails than the spatial graph, and their Pearson correlation with it falls from 0.60 at low frequency to 0.24 at high frequency.Figure 4 compares edge-distance distributions, structural correlations, and weighted edge overlaps for the French v-wind example.Source: Figure 4 · p. 8, §5.4 · p. 8

• WxFM-XL decomposes foundation-model forecasts into frequency bands, dynamically integrates band-specific error priors with static spatial relations, and applies a lightweight graph residual adapter.The adapter performs structured cross-station correction while preserving the foundation model’s temporal priors.Source: §6 · p. 8
• Experiments on regional and global datasets demonstrate consistent improvements over existing baselines.Source: §6 · p. 8
• Future work will incorporate dependencies among meteorological variables for joint multivariate and multi-station forecasting.Source: §6 · p. 8

AI Use Statement

• Generative AI was used only for language polishing and grammar correction, not for research design, analysis, or implementation.Source: AI Use Statement · p. 9

Reproducibility Statement

• Appendices document methodological derivations, preprocessing, configurations, seeds, hardware, and evaluation protocols; source code and data-processing scripts are to be released upon acceptance.Source: Reproducibility Statement · p. 9

Appendix Overview

• The appendices provide methodological derivations, implementation details, experimental results, complete MAE and ablation results, and representative forecasting cases.Source: Appendix Overview · p. 11

A.1 Frequency Domain Decomposition

• Forecasts are decomposed into low-, mid-, and high-frequency components, and the same decomposition is applied to targets for band-specific errors and training.The disjoint masks partition the spectrum, allowing the original forecast to be recovered exactly.Source: §A.1 · pp. 11–12

A.2 Graph Residual Adapter Details

• For each frequency band, a frozen-backbone adapter processes band-limited forecasts through multi-scale patch embedding, graph propagation, temporal refinement, and reconstruction.The adapter makes frequency-specific residual corrections without updating foundation-model parameters.Source: §A.2 · p. 12

A.2.1 Multi-Scale Time Patch Embedding

• Shorter temporal patches preserve local fluctuations, while longer patches provide broader context for residual correction.Scale weights are inferred from refined representations because useful temporal scales can vary across samples and frequency bands.Source: §A.2.1 · p. 12, §A.2.5 · p. 14

A.2.2 Graph Normalization and Propagation

• Graph propagation aggregates residual information across stations connected by geographic relations, foundation-model error dependencies, or both.A learnable node gate refines the frequency-specific fusion graph to account for differing station receptivity.Source: §A.2.2 · p. 13

A.2.3 Time Attention and MLP

• After station-wise graph propagation, Time Attention models dependencies across temporal patches, and an MLP refines token features before residual reconstruction.For a 24-hour horizon, the patch size of 24 yields one token, so that branch does not perform cross-token attention.Source: §A.2.3 · pp. 13–14

A.2.5 Dynamic Multi-Scale Fusion

• Dynamic scale weights combine scale-specific residual forecasts, and a learnable band factor controls each frequency component’s correction magnitude.The residual formulation preserves the frozen foundation model’s temporal forecasts while adding structured corrections based on station dependencies.Source: §A.2.5 · p. 14

A.3 Training Objective

• Training combines final-forecast MSE with energy-normalized error for each frequency band, preventing high-energy low-frequency components from overwhelming optimization.The normalization maintains meaningful gradients for mid- and high-frequency adapters.Source: §A.3 · p. 15

A.4.8 Overall Online Complexity

• With fixed graph and adapter dimensions, online computation and activation memory grow linearly with station count, avoiding quadratic station dependence.The stated condition is that |F|, |P|, K, D, and T_p are fixed with respect to N.Source: §A.4.8 · p. 17

• The Hunan dataset contains hourly observations from 3,570 stations across an approximately 576 km × 576 km region in Hunan Province, China.It covers April 1 to September 30, 2023, and includes 2-meter temperature and 10-meter u- and v-wind components.Source: §B.1.1 · p. 17

B.1.2 French Regional Dataset

• The French regional dataset contains hourly observations from 234 stations in an approximately 252 km × 252 km area in northwestern France.It spans 2016 and evaluates 2-meter temperature and 10-meter u- and v-wind components; compared with Hunan, it has fewer and more sparsely distributed stations.Source: §B.1.2 · p. 17

B.1.3 Global Dataset

• The global dataset contains hourly observations from 3,850 stations worldwide during 2019–2020, with hourly temperature and averaged wind speed evaluated.Because global variables were scaled by 10 during preprocessing, reported MSE and MAE are divided by 100 and 10, respectively.Source: §B.1.3 · p. 17

B.1.4 Dataset Splits

• Each dataset and variable is split chronologically into training, validation, and test sets in a 6:1:3 ratio, without shuffling time.Prior graphs use training data only, and reported results are computed on the held-out test split.Source: §B.1.4 · p. 17

B.2 Baseline Details

• Baselines include zero-shot univariate foundation models, zero-shot and fine-tuned multivariate models, adapter variants, and task-specific multi-station forecasters.Timer and Sundial are evaluated independently at each station; Timer-XL and Moirai have fine-tuned variants, while CDPNet, MSGNet, Corrformer, and STELLA represent multi-station models.Source: §B.2.1 · p. 18, §B.2.2 · p. 18, §B.2.3 · p. 18

B.3 Implementation Details

• WxFM-XL uses frozen Timer or Sundial backbones, a 96-hour input window, three frequency bands, and dataset-specific graph neighborhood sizes.Trainable components include adapters and their graph, temporal, patch, and scale-scoring modules; experiments use three random seeds for trainable models.Source: §B.3 · pp. 18–19

B.4 Evaluation Metrics

• Forecasting accuracy is evaluated with MSE and MAE, for which lower values indicate better performance.For Fig. 4(c), weighted edge overlap compares sparsified, symmetrized spatial and error-prior graphs; it is undefined when its denominator is zero.Source: §B.4 · p. 19, §B.4.1 · p. 19

• Supplementary MAE results cover all datasets, variables, and forecasting horizons under the same settings as the main-paper MSE results.Representative weather sequences are also included to illustrate different methods’ forecasting behavior.Source: C Additional Experimental Results · p. 19
• Table 3 reports complete MAE results for the Hunan, French, and global datasets at 24- and 96-hour forecast horizons.Source: Table 3 · p. 20
• Table 4 reports ablation-study MAE results on the Hunan and French regional datasets at a 24-hour forecast horizon.Source: Table 4 · p. 20

C.1 Complete MAE Results

• Complete MAE results cover Hunan, French, and global datasets at 24- and 96-hour horizons, with mean and sample standard deviation over three runs for trainable models.Timer and Timer-XL use deterministic inference; variations for Sundial and Moirai are negligible relative to their means.Source: §C.1 · p. 20
• The complete model achieves the lowest MAE across variables on both regional datasets; removing frequency decomposition or the error-prior graph degrades accuracy, while removing the multi-scale temporal module degrades it most.The largest degradation is particularly evident for temperature and wind forecasting on the French regional dataset.Source: §C.1 · p. 20

C.2 Forecasting Case Studies

• Representative test cases span datasets, weather variables, and horizons, comparing ground truth with predictions from the frozen foundation model and WxFM-XL.The cases include smooth trends and rapid local fluctuations.Source: §C.2 · p. 20, Figure 5 · p. 22
• The cases support examining systematic forecast deviations, temporal-trend preservation, station-level local variations, and error accumulation at longer horizons.Source: §C.2 · p. 21