AsyncCouple-Flow: Asynchronous Cross-Modal Coupling and Flow Matching for Spatio-Temporal Forecasting

• MM-STF faces asynchronous sampling, deployment-time modality missingness, and long-horizon error accumulation, while common-grid interpolation can discard fast-stream detail or fabricate values for slow streams.The unified goal is to process different timestamps and spatial scales without a shared grid, tolerate absent streams, and predict the full trajectory non-autoregressively.Source: §1 · p. 2
• MATS tokenizes each modality at its native rate and selects top-k informative tokens into equal-length sequences, while ACCG uses graph edges encoding temporal, semantic, and physical relationships.ACCG represents each (modality, time, position) triplet as a node; absent modalities are masked nodes handled through graph message passing.Source: §1 · pp. 2–3
• The Flow-Matching Forecasting Head models a multi-step forecast as one conditional ODE integration, avoiding autoregressive recursion and its recursive feedback of prediction errors.Stochastic modality dropout during training exposes the model to varied missingness patterns.Source: §1 · p. 3
• AsyncCouple-Flow reports consistent improvements over strong specialized baselines on weather and traffic forecasting and robustness with up to two missing modalities at inference.The reported evaluations use ERA5+GOES+ISD weather forecasting and PEMS-BAY traffic forecasting.Source: §1 · p. 3

2.1 Spatio-Temporal Forecasting

• STGNNs such as DCRNN and Graph WaveNet model single-modality sensor data on a shared time grid; Earth-system Transformers often use autoregressive rollout.AsyncCouple-Flow is described as complementary because it models multi-source asynchrony and uses single-pass flow integration.Source: §2.1 · p. 3

2.2 Multi-Modal Fusion under Asynchrony and Missingness

• CrossViViT and AirFormer assume shared-grid, full-stream inference; static-input SMIL addresses missingness, whereas ACCG treats absent streams as masked graph nodes.ACCG represents each (modality, time, position) triplet as a node, with neighboring nodes handling message passing when modalities are absent.Source: §2.2 · p. 4

2.3 Generative Forecasting via Flow Matching

• Diffusion forecasting uses computationally heavy iterative reverse sampling; Flow Matching integrates a single conditional ODE, and this paper applies it to MM-STF.The paper states that, to its knowledge, Flow Matching had not previously been applied to MM-STF.Source: §2.3 · p. 4

3.2 Modality-Aware Token Sparsification (MATS)

• MATS uses modality-specific encoders and a shared scorer to retain top-k tokens per timestep, forming equal-length sequences without cross-modal interpolation.The encoder uses CNN patch embeddings for grid sources and a point-wise MLP with positional encoding for in-situ sensors; positional embeddings encode absolute time and relative spatial offset.Source: §3.2 · pp. 5–6

3.3 Asynchronous Cross-Modal Coupling Graph (ACCG)

• ACCG represents retained tokens as native-time graph nodes and weights edges by temporal offset, semantic similarity, and modality-pair physical priors.It limits connectivity to temporal neighbors within window W and cross-modal nearest-time neighbors, yielding O(|V| · W) edges rather than O(|V|^2).Source: §3.3 · p. 6
• When a modality is absent, ACCG omits its nodes and continues message passing over the remaining graph, with temporal decay reweighting farther-in-time observations.Target-time queries attend over the graph to form the context used for forecasting.Source: §3.3 · p. 6

3.4 Flow-Matching Forecasting Head

• The Flow-Matching head trains a conditional vector field on the displacement from Gaussian noise to the ground-truth trajectory and generates the H-step forecast by integrating the ODE once.The ACCG context conditions the vector field through cross-attention; 10–25 Euler steps yield forecasts reported as indistinguishable from finer discretizations, avoiding per-step error amplification inherent in autoregressive rollout.Source: §3.4 · p. 7

3.5 Training Objective and Modality Dropout

• Training adds a one-step reconstruction regularizer and token-budget loss to flow matching, and drops modalities stochastically after MATS to prepare for missing inputs at inference.Each iteration drops modalities with probability 0.3 and recomputes the graph on the resulting subgraph; ACCG costs O(L · V · W · d), while inference uses at most 25 ODE solver steps.Source: §3.5 · p. 7

4.1 Datasets and Setup

• The evaluation spans WeatherBench-MM weather forecasts using ERA5, GOES-16, and ISD inputs, and PEMS-BAY-MM traffic forecasts augmented with road-network and NOAA weather information.Weather targets are 2-m temperature and total precipitation at 6 h and 24 h; traffic flow is forecast at 15, 30, and 60 min.Source: §4.1 · p. 8

4.2 Main Results

• AsyncCouple-Flow achieves the best scores across both benchmarks and all four metrics.On WeatherBench-MM at 24 h, it reduces RMSE by 11.6% over CrossViViT and raises SSIM by 0.045. On PEMS-BAY-MM at 60 min, it reduces MAE by 16.6% over MTGNN and 17.4% over Graph WaveNet.Source: §4.2 · p. 8, Table 1 · p. 9

4.3 Ablation Study

• Learned token selection, all three ACCG edge factors, and the Flow-Matching head improve ablation performance.Mean pooling and random sampling raise MAE by 9.7% and 14.2%, respectively; removing δt, sim, or ϕphys raises MAE by 5.3%, 7.1%, or 3.5%, with semantic similarity having the largest effect. An L2 head raises RMSE by 13.0%; 50-step DDPM recovers roughly half the MAE gap but takes 4.5× the inference time.Source: §4.3 · pp. 8–9, Figure 2 · p. 9

4.4 Robustness to Missing Modalities

• With two modalities missing, the full model reaches MAE 1.22, beating every baseline with complete inputs, whose best MAE is 1.29.Adding training-time modality dropout further flattens the degradation curve; AsyncCouple-Flow without dropout is already more robust through ACCG masking.Source: Figure 3 · p. 10, §4.4 · p. 10

4.5 Long-Horizon Drift

• At 24 h on weather and 60 min on traffic, AsyncCouple-Flow reduces MAE by 12.4% and 10.1%, respectively, versus CrossViViT.Errors increase with horizon, including for non-autoregressive CrossViViT; AsyncCouple-Flow predicts the trajectory through one ODE integration while maintaining lower error throughout.Source: §4.5 · p. 10, Figure 4 · p. 11

4.6 Efficiency Analysis

• At 150 ms per H-step forecast on one A100 GPU, AsyncCouple-Flow is 1.4× and 3.1× faster than CrossViViT and 50-step PreDiff, respectively.It is slower than Earthformer (55 ms) but reduces MAE from 1.49 to 1.13. MATS cuts GNN input tokens by ∼6×, and the 25-step Euler solver count is independent of forecast horizon.Source: §4.6 · pp. 10–11, Figure 5 · p. 11

4.7 Spectral Fidelity

• Within ±10%, AsyncCouple-Flow’s predicted-to-ground-truth energy ratio stays across the full wavenumber range, while every baseline falls below 0.5 at k ≥40.Its spectrum follows ERA5 nearly to the smallest resolved scales; Earthformer, FengWu, and CrossViViT lose one to two orders of magnitude of energy at k ≥30.Source: Figure 6 · p. 12, §4.7 · p. 12

• Across two heterogeneous benchmarks and four metrics, AsyncCouple-Flow outperforms ten state-of-the-art baselines, tolerates up to two missing modalities, and improves high-frequency reconstruction rather than over-smoothing.Source: §5 · p. 12
• MATS compresses heterogeneous inputs into equal-length sequences, ACCG edges factor time offset, semantic similarity, and physical priors, and the Flow-Matching head predicts the entire trajectory in one ODE pass.Source: §5 · p. 12
• The framework jointly addresses asynchronous sampling rates, deployment-time modality missingness, and long-horizon error accumulation—obstacles that existing approaches typically tackle in isolation.Source: §5 · p. 12