PaperlayerView exact revision on arXiv

Source-linked AI summary

Extending Precipitation Nowcasting Horizons via Spectral Fusion of Radar Observations and Foundation Model Priors

Yuze Qin, Qingyong Li, Zhiqing Guo, Wen Wang, Yan Liu, Yangli-ao Geng

arXiv:2603.21768v3cs.LGcs.AI

TL;DR

Radar-only precipitation nowcasting degrades at longer lead times because it lacks large-scale atmospheric context, while existing multimodal methods do not reconcile radar–meteorological heterogeneity. PW-FouCast addresses this gap by fusing Pangu-Weather forecasts with radar observations in the frequency domain through spectral modulation, memory, and high-frequency reinjection. It achieves state-of-the-art performance on SEVIR and MeteoNet while maintaining structural fidelity.

  • Problem

    Radar-only nowcasting lacks large-scale atmospheric context at longer lead times, and existing multimodal fusion methods do not explicitly resolve radar–meteorological heterogeneities.

  • Method

    PW-FouCast uses a Fourier-domain backbone to fuse Pangu-Weather forecasts with radar observations through spectral modulation, Frequency Memory, and Inverted Frequency Attention.

  • Results

    PW-FouCast achieves state-of-the-art performance on SEVIR and MeteoNet, outperforming radar-only and standard multimodal baselines.

  • Takeaways & Limitations

    The framework extends reliable forecast horizons while preserving structural fidelity in long-lead precipitation nowcasting.

Abstract

from arXiv · show

Precipitation nowcasting is critical for disaster mitigation and aviation safety. However, radar-only models frequently suffer from a lack of large-scale atmospheric context, leading to performance degradation at longer lead times. While integrating meteorological variables predicted by weather foundation models offers a potential remedy, existing architectures fail to reconcile the profound representational heterogeneities between radar imagery and meteorological data. To bridge this gap, we propose PW-FouCast, a novel frequency-domain fusion framework that leverages Pangu-Weather forecasts as spectral priors within a Fourier-based backbone. Our architecture introduces three key innovations: (i) Pangu-Weather-guided Frequency Modulation to align spectral magnitudes and phases with meteorological priors; (ii) Frequency Memory to correct phase discrepancies and preserve temporal evolution; and (iii) Inverted Frequency Attention to reconstruct high-frequency details typically lost in spectral filtering. Extensive experiments on the SEVIR and MeteoNet benchmarks demonstrate that PW-FouCast achieves state-of-the-art performance, effectively extending the reliable forecast horizon while maintaining structural fidelity. Our code is available at https://github.com/Onemissed/PW-FouCast.

I. INTRODUCTION

Radar-only nowcasting can degrade at longer lead times because radar reflectivity lacks the atmospheric drivers underlying precipitation evolution. PW-FouCast addresses this gap by fusing Pangu-Weather priors with radar information in the frequency domain.

  • Motivation: Radar-only models often degrade at longer lead times because reflectivity captures precipitation outcomes rather than their atmospheric drivers.The cited drivers include temperature, humidity, wind speed, and pressure.
  • Motivation: Foundation-model forecasts provide multimodal atmospheric inputs, but conventional addition, concatenation, and cross-attention do not explicitly resolve radar–meteorological heterogeneities.The modalities differ in spatial scale, distribution, and representation.
  • Proposed Approach: PW-FouCast uses a Fourier-domain backbone to align and fuse spectral amplitude and phase information from radar observations and Pangu-Weather forecasts.The framework exploits shared phase representations between the modalities.
  • Contributions: The framework introduces a frequency-domain encoder–decoder designed to assimilate foundation-model priors and resolve heterogeneities between radar reflectivity and meteorological forecasts.These are presented as two primary contributions of the work.
  • Contributions: Frequency Memory stores and retrieves spectral features of diverse precipitation patterns to help maintain structural fidelity over time.The module is listed among the paper’s contributions alongside the multimodal fusion framework.
  • Contributions: PW-FouCast achieves state-of-the-art results on SEVIR and MeteoNet, outperforming radar-only and standard multimodal baselines.The authors report extensive benchmark experiments supporting this comparison.

III. PRELIMINARIES

The paper combines Pangu-Weather meteorological forecasts with radar sequences and processes them through Fourier-domain representations. Meteorological inputs are regridded and temporally resampled to match the radar data before prediction.

  • Pangu-Weather Model: Pangu-Weather supplies geopotential, humidity, temperature, and wind components as multimodal inputs for long-term precipitation nowcasting.These variables represent synoptic-scale trends and are used as physical constraints.
  • Adaptive Fourier Neural Operator: AFNO applies a forward Fourier transform to an input feature map for spatial mixing before adaptively mixing spectral channels with a shared MLP.The MLP uses block-diagonal complex-valued weights and shared parameters across spatial tokens.
  • Problem Formulation: The forecasting task uses an observed radar sequence together with N Pangu-Weather forecasts to predict K future radar frames.Radar frames have C channels and spatial resolution H × W, while meteorological forecasts contain M variables on an H′ × W′ grid.
  • Problem Formulation: A preprocessing operator spatially regrids and temporally resamples meteorological fields so they match the radar’s shape and cadence in latent space.The alignment addresses differences in spatial and temporal resolution before multimodal processing.
  • Problem Formulation: The model outputs a predicted radar frame ˆX_t at each future lead time.The predicted frame has C channels and spatial resolution H × W.

IV. METHODOLOGY

PW-FouCast fuses radar and meteorological information in the Fourier domain by aligning spectral amplitudes and phases, using Pangu-Weather phase features as structural guidance.

  • PW-FouCast integrates radar observations with Pangu-Weather forecasts through a frequency-domain encoder–decoder architecture.The framework is designed for spectral integration of multimodal inputs.
  • Pangu-Weather-guided Frequency Modulation: PFM steers spectral magnitudes and phases toward ground-truth properties using Pangu-Weather guidance.It aligns both amplitude and phase information in the frequency domain.
  • Pangu-Weather-guided Frequency Modulation: Radar amplitude combined with Pangu-Weather phase produces spatial patterns that similarly match observed radar reflectivity.This phase similarity motivates using meteorological phase features to correct hidden-layer representations.
  • Pangu-Weather-guided Frequency Modulation: Channelwise attention amplifies frequency components whose phase aligns with Pangu-Weather predictions, then learnable phasor interpolation fuses the phases.The fused phase is recombined with the reweighted amplitude to form complex frequency coefficients.
  • Pangu-Weather-guided Frequency Modulation: The two-stage fusion first aligns magnitudes and then refines phases, producing hidden-layer coefficients that better match ground truth in both properties.

C. Frequency Memory

Frequency Memory stores spectral patterns from ground-truth precipitation sequences and retrieves them to correct hidden-layer phase discrepancies while preserving structural evolution.

  • Frequency Memory records phase patterns from observed sequences to preserve fine-grained structural changes in forecasts.It addresses the diversity of precipitation movement, expansion, and contraction patterns that coarse meteorological priors cannot capture accurately.
  • Training and matching: Training uses storing and matching stages in which a memory bank is populated with ground-truth frequency-domain features and later used for phase alignment.The memory remains fixed during the second phase.
  • Memory matching: Ground-truth radar sequences are encoded and transformed with a discrete Fourier transform to obtain frequency features for memory matching.The resulting feature has embedding height, width, and channel dimensions.
  • Memory matching: Normalized ground-truth frequency features are compared with memory slots using complex inner products, producing one similarity vector per spatial location.The memory contains S slots, and similarities are converted into attention weights across slots.
  • Memory matching: Attention-weighted memory retrieval forms matched frequency features as convex combinations of memory slots, bounding their amplitude between 0 and 1.
  • Phase alignment: FM-PA uses recalled features to correct hidden-layer phases, retaining recalled amplitude information rather than normalizing it away.The phase-fusion weight increases as phase discrepancy grows, rotating hidden phases toward the matched phase.
  • Phase alignment: Input features are aligned to the memory channel dimension before the same matching procedure retrieves patterns for phase correction.

D. Inverted Frequency Attention

Inverted Frequency Attention restores high-frequency information attenuated by learned spectral filtering by extracting and gatedly reinjecting the discarded residual.

  • IFA enhances spectral diversity in hidden layers for temporal-feature extraction.
  • Motivation: The learned Fourier operator tends to attenuate small-amplitude coefficients, producing effective low-pass behavior.
  • Residual extraction: IFA computes the discarded high-frequency residual by subtracting the filtered output from the input coefficients.This operation effectively applies the inverse mask of the learned frequency operator.
  • Residual reinjection: A learnable channelwise gate controls reinjection of the high-frequency residual into the low-frequency component.The gate is broadcast across spatial frequency dimensions for elementwise multiplication.

E. Loss Function

The training objective combines spatial reconstruction accuracy with spectral fidelity to reduce spatial error and preserve high-frequency precipitation structure.

  • The loss is a weighted sum of spatial mean-squared error and spectral L1 loss.The weighting coefficient λ satisfies 0 ≤ λ ≤ 1.
  • The spatial MSE penalizes reconstruction error, while spectral L1 encourages accurate frequency-domain recovery and preservation of high-frequency echo structure.

A. Experimental Setup

The SEVIR experiments use five radar frames covering 50 minutes to predict the following 20 frames, spanning a 200-minute forecast horizon.

  • SEVIR experiments use five input frames covering 50 minutes and predict the next 20 frames covering 200 minutes.

1) Dataset:

The evaluation combines SEVIR and MeteoNet data, Pangu-Weather meteorological inputs, standardized preprocessing, and multiple complementary forecasting metrics.

  • 1) Dataset:: MeteoNet experiments use five input frames over 50 minutes to predict 20 frames over 200 minutes.The 2018 subset contains 5,381 training samples and 1,027 test samples.
  • 1) Dataset:: Pangu-Weather forecasts provide five upper-air variables, temporally aligned with radar and spatially resampled to 32 × 32 hidden-layer resolution.
  • 1) Dataset:: Models use AdamW with a 0.001 learning rate, four convolutional encoder-decoder modules, hidden depth L = 6, and 128×128 radar inputs.
  • 3) Evaluation Metrics:: Performance is evaluated with CSI and HSS, alongside MSE, MAE, PSNR, and SSIM for event, continuous, perceptual, and structural accuracy.

4) Hyperparameter Selection:

Hyperparameter sweeps select dataset-specific Frequency Memory sizes and loss weights, while benchmark results assess PW-FouCast against unimodal and multimodal baselines and long-term performance.

  • 4) Hyperparameter Selection:: Hyperparameter sweeps select S = 240 for SEVIR and S = 160 for MeteoNet, with loss weights λ = 0.57 and λ = 0.55, respectively.The study attributes SEVIR’s larger memory requirement to greater precipitation-pattern complexity.
  • B. Experimental Results: PW-FouCast reduces SEVIR MSE and MAE by 2.28% and 2.15% and raises average CSI and HSS by 6.84% and 7.28% over strongest baselines.
  • B. Experimental Results: On MeteoNet, PW-FouCast reduces MSE and MAE by 2.36% and 3.80% and improves CSI and HSS by 5.40% and 5.21%.Peak PSNR and SSIM also indicate superior pixel-level accuracy and structural fidelity.
  • B. Experimental Results: Spectral fusion aligns radar and meteorological magnitudes and phases, addressing heterogeneities that simplistic spatial fusion methods fail to reconcile.
  • B. Experimental Results: PW-FouCast maintains a performance lead at every SEVIR time step, with wider MAE and PSNR gaps as lead time increases.

C. Case Study

PW-FouCast maintains sharper and more physically consistent precipitation structures at long lead times on SEVIR and MeteoNet. Ablations attribute complementary gains to spectral alignment, phase correction, and high-frequency detail recovery.

  • Qualitative Case Studies: On SEVIR, PW-FouCast maintains sharp echo structures beyond two hours, where baselines typically degrade.This qualitative comparison is shown in Fig. 6.
  • Qualitative Case Studies: On MeteoNet, the model preserves well-defined structures beyond 120 minutes and outperforms unimodal and multimodal baselines.This comparison is shown in Fig. 7.
  • Ablation Study: PFM aligns hidden-layer spectral properties with meteorological priors, while Frequency Memory corrects phase information and primarily reduces spatial errors.The ablation study is reported in Table III.
  • Ablation Study: PFM and Frequency Memory jointly sharpen precipitation localization through complementary amplitude reweighting and phase correction.
  • Ablation Study: Inverted Frequency Attention recovers high-frequency details lost in conventional spectral attention, improving perceptual scores and echo-boundary realism.
  • Overall Findings: Across SEVIR and MeteoNet, PW-FouCast achieves state-of-the-art results while improving accuracy and structural fidelity.The framework uses phase-aware spectral fusion of Pangu-Weather priors.
Loading 2603.21768v3…